12 ms·
Outperforming larger language models with less training data and smaller models
- xnx 3y agoThe amount of activity and progress in the LLM/ML/AI spaces is truly fantastic. Optimizations like this are particularly valuable when hardware (e.g. Nvidia) is so expensive.
- ziofill 3y agoIs it that a lot of capacity is unused in those behemoth LLMs, or that the smaller language model just mimics the reasoning task? (Mimics the mimicking?)
- famouswaffles 3y agoThere is no actual distinction between the "real" thing and "mimicking". The datasets behemoth LLMs are trained on include a lot of noise that derail progress. They also just contain a lot of irrelevant knowledge that the LLM has to learn or memorize so an obscene amount of parameters is required. When you're not trying to teach a language model the sum total of human knowledge and you provide a high quality curated dataset, the scale barrier is much lower. https://arxiv.org/abs/2305.07759 https://arxiv.org/abs/2305.07759
- behnamoh 3y agoWhy don't people just train LLMs on pure Wikipedia, arXiv, and other scientific websites? That would reduce the noise and improve hallucinations, no?
- vczf 3y agoI don't think it would know how to have a conversation if it's never been exposed to one before.
- taneq 3y agoThat’s what the fine tuning is about. It learns the language, concepts etc. from the main dataset and is then tweaked by continuing to train on a smaller, high quality, hand curated dataset. That’s how it learns to generate conversational responses by default instead of needing a complicated prompt.
- brucethemoose2 3y agoThat is essentially Phi. The results are promising. But LLMs do gain useful "emergent" properties from training on massive lower quality datasets.
- deleted 3y ago[deleted]
- astrange 3y agoIt has to learn the meaning of words, including implicit associations, and to do that it needs to see approximately all the English text ever. We don't know how to balance this with only feeding it useful knowledge.
- eru 3y agoIt doesn't necessarily have to see approximately all the English text ever. Real people don't learn English like that, for example. It's just that given what we know about neural networks, it's often easier and simpler and more effective to increase the amount of training data than to change anything else.
- basch 3y agoReal people might not speak every dialect of English. They may not be well versed in local grammatical oddities.
- marshray 3y agoDoesn't seem to be much of a problem in practice? If someone knew all the math and science in Wikipedia, for example, I think they'd probably be forgiven for not knowing every regionalism.
- astrange 3y agoUnfortunately models aren't always good at knowing what they don't know ("out of distribution data") so it could lead to confidently wrong answers if you leave something out. And if you want it to be superhuman then you're by definition not capable of knowing what's important, I guess.
- eru 3y agoBtw, models like GPT 4 can express that they are not confident. But that looks like a 'smeared' out probability distribution on the next token. Not like the text produced by an unsure human.
- msp26 3y agoIt's been done before. "Textbooks are all you need". https://arxiv.org/abs/2306.11644 https://arxiv.org/abs/2306.11644
- fulafel 3y agoThat question seems synonymous with "are those [current] behemoth LLMs close to optimal?" and it seems obvious that they are not. I wonder what ideas there are about how one could estimate the optimal size.
- sp332 3y agoThe large models generalize better. The small models are easier to train for specific tasks.
- sinuhe69 3y agoWhy the amount of the training data for LLM is less than for the distilled and task-specific models (in the first figure)? Or did the authors count the amount of training data for the LLMs to the required training data for the destined/task-specific models? https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeIs4yaBA3Ir55j869FMzdmRdf7OxiIjsWl05GU48ikYOHZGLk1H8tIHeKKBaY_xER0QITv5DUhADZvqS1os6mNA_nLQKqwW7DOXnwcnPl6BhsMJ_LKTvglGUrHR5_QC8MIe3K7i9zyfcWkwzvjPhXLifYijgkeeG_1yn9EMm-ol9eI9Cv_rz71wMyGfk2/s1570/image3.png https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj...
- swsieber 3y ago> Or did the authors count the amount of training data for the LLMs to the required training data for the destined/task-specific models? Yes. They are counting the amount of data you need to collect to solve your problem. I can grab a pretrained LLM, and the data I have to collect in that instance is what I need to fine tune it.
- threeseed 3y ago> For instance, serving a single 175 billion LLM requires at least 350GB of GPU memory using specialized infrastructure Apple ships the Mac Studio which support up to 144GB of usable GPU memory. Would be amusing if they were to release a Mac Pro with 300+ GB and dominate the LLM serving space.
- rubyn00bie 3y agoApple's hardware advantage is something I'm hoping to see really unleashed with the M3 generation. The fact the A17 Pro has ray tracing support is giving me hope they can catch up with the incumbents quickly. It's honestly the one thing that has kept me away from their newer hardware-- I primarily use my computer at a desk and PC hardware (GPU primarily) is leagues ahead of what their very best can do. I honestly can't justify spending nearly 4k when Linux works great for everything I do professionally AND when I'm done with work I can still play a game.
- behnamoh 3y ago3 most important things for Apple are: - HW design - UI - UX That Macs come with awesome VRAM is only because #3. Apple has no explicit incentive to make LLM-ready machines.
- rubyn00bie 3y agoI disagree. ML is eating the world by storm and one thing Apple loves is high margins. Folks like myself would gladly pay that margin to someone besides Nvidia. I paid $2k for my last GPU and was happy to do it. That doesn’t even factor the fact that businesses would pay even more when the alternative are machines that start in the tens of thousands for just the GPU ability. GPUs right now are carrying fat margins and that makes more than enough incentive for Apple IMHO.
- behnamoh 3y agoBut you wouldn't be able to upgrade an Apple Mac when the next generation of LLMs is out.
- greatpostman 3y agoStill waiting for google to release a model that matches gpt4. Until then, I’m assuming their presumed ai supremacy is marketing
- behnamoh 3y agoThere's AI-research supremacy and there's AI-product supremacy. Google has many brains (pun intended) on the former, but lacks incentive/structure for the latter.
- 6D794163636F756 3y agoThey're often also slow to market. They weren't the first search engine, they weren't the first email platform, they weren't the first video site, but now they're the biggest in those categories. I think Bard's growth will be very interesting to watch given that track record. Or it'll get dropped within a year.
- ttul 3y agoIt’s not getting dropped. If OpenAI’s models get good enough, search traffic will crater and Google will fall on extremely hard times for lack of ad revenue. At the same time, if they fail to integrate state of the art generative features into Workspace, people will go to Microsoft, where GPT-4 is presently handing Google their ass. Yes, Google Duet for Workspace totally sucks; I suspect their trial conversion rate approaches 0%. This is a make it or break it problem for Google and they will get it right or they won’t get it at all.
- pedrovhb 3y agoInteresting that they use T5 for the distilled model. I was under the impression that encoder-decoder architectures were on the way of the Dodo, but it seems they may still be relevant after all. Also interesting is that this isn't an inconceivably clever and out of the box idea. It shows there's still a lot of low hanging fruit to explore, and the future of LLMs isn't set in stone yet. Could be that the real deal is a mixture of experts trained in this style. It's exciting that it feels the holy grail is close to being achievable if only the right combination of ideas is tried.
- behnamoh 3y ago> I was under the impression that encoder-decoder architectures were on the way of the Dodo,... Why is that?
- janalsncm 3y agoDecoder-only is pretty good on some tasks. Translation for example was a classic encoded-decoder task but it seems decoders can handle it.
- mike_hearn 3y agoIf there's no encoder, then what is being decoded?
- joemoon 3y agoIt's definitely a very similar method but fundamentally different in that the 'Distilling step-by-step' approach is a multi-task model. As I understand it, rather that training the smaller model to produce the CoT/rationale as a part of the (decoded) response, it actually has two output layers. One output is for the label and the other is for the rationale. The other layers are shared, which is how/why the model is able to have an improved "understanding" of which nuances matter in the labeling task.
- euclaise 3y agoThey actually have a performance edge, but they aren't well suited to chat models because you can't do caching of past states like with decoder-only models
- p-e-w 3y ago> given the input question “Sammy wanted to go to where the people are. Where might he go? Answer Choices: (a) populated areas, (b) race track, (c) desert, (d) apartment, (e) roadblock”, distilling step-by-step provides the correct answer to the question, “(a) populated areas” Huh? My answer as a human would have been "race track", as that is probably "where the people are" (during a race). Did I fail? Am I a poor language model? Or is the whole thing just tea leaf reading to begin with?
- smeagull 3y agoA race track or an apartment are likely to be populated areas as well. "A" is a broader answer that encompasses more possibilities.
- Zuiii 3y agoA depending on the size of a "populated area", the density can be far lower than a crowded race track.
- kybernetikos 3y agoYeah "I felt lonely and wanted to be where the people are, so I wandered through the suburbs" really doesn't make sense. I think this is just a bad question.
- tuukkah 3y agoFurther, not all populated areas are known for their nightlife and social events.
- p-e-w 3y ago"Where the people are" doesn't simply mean "where a large number of people live". It also has connotations of "where it's happening", as in "where people are coming together (right now)". But my point is really that speaking of the correct answer with a question as vague and open to interpretation as this one is absurd.
- ilaksh 3y agoI am not a researcher, but it always seemed intuitive to me that the most effective models would be multimodal and trained with a core carefully tailored curriculum. I would want to ensure that the system gains and retains the fundamental structures and skills that you know it needs to effectively and accurately generalize. While maintaining those things you then feed it lots of diverse data to learn the exceptions and ways the skills can be combined. But somehow you need to ensure those core skills and knowledge throughout. Maybe you could do that just by including outputting those understandings or manipulations in addition to the final answer. Similar to what the paper does. For example, a code generation model might be required to output a state machine simulation of the requested program.
- eru 3y agoI agree that multimodal seems like the way to go, but it's not at all intuitive why we should expect the curriculum needing to be carefully tailored. Compare https://gwern.net/scaling-hypothesis https://gwern.net/scaling-hypothesis
- mcmoor 3y agoYeah so far looking at the progression since 70s, domain-specific attempts stagnant while just-increase-the-input-and-core attempts get us where we are now
- two_in_one 3y agoMy impression is that while domain specific models do better in their domain. They do not generalize well and cannot be transferred to different domains without retraining. Hence comes mixture of experts (domain specifics). The trick is to juggle them all. Which needs another quite capable 'manager' model. The good thing is that they can be trained separately. Also at inference there in no need to call them all. Existing models, like GPT-4, can be used to create datasets for experts, or enhance/balance existing datasets.
- isaacfung 3y agoLIMA: Less Is More for Alignment https://arxiv.org/abs/2305.11206 https://arxiv.org/abs/2305.11206 AlpaGasus: Training A Better Alpaca with Fewer Data https://arxiv.org/abs/2307.08701 https://arxiv.org/abs/2307.08701 Textbooks Are All You Need II: phi-1.5 technical report https://arxiv.org/abs/2309.05463 https://arxiv.org/abs/2309.05463 Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Models https://arxiv.org/abs/2307.14430 https://arxiv.org/abs/2307.14430
- avereveard 3y agoSo this https://arxiv.org/abs/2212.08410 https://arxiv.org/abs/2212.08410 but one year later
- sp332 3y agoImpressive gains, but 22% on GSM8K is maybe not an attention-grabbing end result.
- fbnbr 3y agoI think smaller expert models will dominate the majority of applications. there is an optimum and fine balance to strike when it comes to size and usability. There will be many mechanisms like demonstrated in the post to find that optimum and realize it.
- dtx1 3y agoAnd for a large general model just have multiple small expert models and a relay model that decides which domain specific model to ask
- sandpaper26 3y agoIt's not a likely solution given how loss functions work, but in theory a single model could learn to perform exactly the function you describe. When you say "just do X" where X is any function (in this case, a piecewise function), a large enough model could do it.
- sandpaper26 3y agoAfter some reflection, it's maybe more accurate to visualize this in reverse: all expert models see the problem and attempt a solution, and then some "manager" model decides which expert model has the best solution and outputs it.
- liquidpele 3y agoUntil the manager model decides to outsource the expert models.
- treprinum 3y agoIn theory you need two layers to model any function. In practice this is wildly different.
- sandpaper26 3y ago
- sourabh03agr 3y agoInteresting! Do you think RLHF would be a necessity for smaller models to perform as par as state-of-the-art LLMs? In my view, instruction tuning will resolve any isssues related to output structure, tonality or the domain understanding but will it be enough to improve the reasoning capabilities of the smaller model?
- scotty79 3y agoI wonder if facebook could train LLM on all chat histories of all of its users.