4 ms·
We don't expect this to be as good as the latest OpenAI GPT release. This is just to demonstrate that developing a conversation agent using an existing foundati
by falaki 4y ago
We don't expect this to be as good as the latest OpenAI GPT release. This is just to demonstrate that developing a conversation agent using an existing foundation model is not as hard as some may assume. Take a foundation model that is not capable of Q&A and tune it with a fairly small Q&A data and you get your in-house ChatGPT.
Disclaimer: I work at Databricks.
- mrg3_2013 4y agoThanks for the feedback. The potential edge with Dolly is huge. Building a firewalled model with custom corpus is a big deal. I have been experimenting with openai and even with public data (but really limiting to the domain), yields great improvements (openai may be stale because of cut off data). I am excited to see where Dolly goes.
- mrtranscendence 4y agoDolly appears to fundamentally be a tech demo advertising how you can use Databricks for compute. I honestly wouldn't expect them to take it that much further, particularly in the context of larger models that would be significantly more expensive to fine-tune. But I'm happy to be proven wrong.
- falaki 4y agoWe plan to continue working on it and invest more.
- theGnuMe 4y agoI imagine they will sell fine tuning as a service to Databricks customers. If I put all my data into their lake I too can get my own custom ChatGPT. That's compelling.
- epups 4y agoI also see that as the use case and would find it useful. However I feel this is somewhat low-budget so far coming from such a large company.
- Szpadel 4y agoyou are referring to the dolly model? I think the training set could achieve similar performance if we would fine tune similarly sized model