3 ms·
Yes and no. Everybody to some degree needs open models based on public data. That's because part of the power of these models is that there needs to be a gene
by ravenstine 3y ago
Yes and no.
Everybody to some degree needs open models based on public data. That's because part of the power of these models is that there needs to be a general-purpose foundation that the LLMs can be fine tuned on for specific purposes. We teach children a variety of subjects because, if you train a child to do nothing except weld for 12 years, you might get an adult who can competently perform very specific welds but is otherwise a bumbling illiterate idiot who can't take custom instructions or do basic arithmetic when just holding a torch isn't enough. You might be able to train an LLM from scratch on a very specific corpus, but LLMs only really become useful when they have an amount of general purpose knowledge.
Plus there's the problem that training an LLM from scratch takes a ton of time.
But the siloing of specific knowledge can make a lot of sense. Specialist LLMs may be closed or semi-closed but still be based on a foundation of open data.