3 ms·
Completely missed the point, as usual. Consider the amount of world knowledge present in a 175B model vs a 7B model. You can elicit instruction following behavi
by Imnimo 4y ago
Completely missed the point, as usual. Consider the amount of world knowledge present in a 175B model vs a 7B model. You can elicit instruction following behavior with a relatively small number of samples , but you're never going to get that coverage of memorized facts. The Alpaca blog post gives the example that Alpaca doesn't know the capital of Tanzania, for example. The most is that depth of knowledge and understanding, not the surface level instruction following behavior.
- radu_floricica 4y agoThe point is that you can split the work in two: the one you mentioned, creating and training the model, and a separate step of shaping it with reinforcement learning. Which is quite expensive, because it uses human operators in one of its steps. But apparently you can duplicate easily enough this with an API of an already educated engine, which makes the second step not be a moat anymore.
- Imnimo 4y agoHow expensive do you imagine collecting human feedback data is compared to the cost of doing the original language modelling training?
- cubefox 4y agoBut that's not a new point?! The second, expensive step (RLHF) was already automated by Anthropic with their Constitutional AI approach several months ago.
- terafo 4y agoYou don't need to store it in weights. It would be too expensive to constantly update model with new data. It wouldn't be able to contain everything anyways. LLaMa 30B with access to 100TB of storage(you can store whole libgen plus arxiv and much much more in there) and trained to use it should be ahead of LLaMa 65B with no such access in terms of world knowledge. And it would cost less.
- Imnimo 4y agoYou can retrieve small amounts of specific information this way. But suppose the model was asked to write code in a new programming language that it doesn't have existing knowledge of. Could it do that given unguided access to the docs?
- terafo 4y agoIt might, but in this specific instance some kind of fine-tuning would be better.
- Imnimo 4y agoIt feels pretty obvious to me that it could not, but then again I've been wrong about LLM abilities before!
- webmaven 4y agoWe've already seen zero-shot learning of novel syntaxes (that is, chatting with an LLM to interactively devise a new language that is then used, without any actual retraining of the model).