6 ms·
The ability to run it on your own private data and train/distill it from the larger models is very useful (see Orca for example: https://arxiv.org/abs/2306.0270
by emadm 3y ago
The ability to run it on your own private data and train/distill it from the larger models is very useful (see Orca for example: https://arxiv.org/abs/2306.02707 https://arxiv.org/abs/2306.02707).
Most of the valuable data in the world is private data unsuitable to send to GPT4 and other proprietary models.
- MuffinFlavored 3y ago> is very useful How? Is there not extreme correlation between "13B parameters (low/not a lot) most likely means lots of hallucinations"
- emadm 3y agoWhat’s better a 130bn model or 10 13bn models working together.
- jmiskovic 3y agoThe GPT4 cognitive leaps and breadth of knowledge seem impossible to replicate with any number of 13B models. 13B x10 might be the only feasible business model for your company, but that alone doesn't make it a superior option. Can you share more details on how the models work together?