5 ms·
This is 2020-level stuff. These days with emergent abilities in LLMs trained with over 1T tokens like GPT-4 a single-shot chain-of-thought beats most fine-tunin
by treprinum 3y ago
This is 2020-level stuff. These days with emergent abilities in LLMs trained with over 1T tokens like GPT-4 a single-shot chain-of-thought beats most fine-tunings. I did research on transformer adapters i.e. parameter-efficient fine-tuning and that stuff is now completely obsolete outside of some restricted domains where small models can still perform well.
- jph00 3y agoI haven't seen any recent papers that show that fine-tuning is obsolete - I've only seen papers showing the opposite. I'd be very interested to see any papers that have demonstrated applications where fine-tuning is not effective nowadays, if you have any links. Here's an example of a paper that shows good results from fine-tuning for instance: https://arxiv.org/abs/2305.02301 https://arxiv.org/abs/2305.02301
- treprinum 3y agoThis Stanford seminar video can provide some references: https://www.youtube.com/watch?v=tVtOevLrt5U https://www.youtube.com/watch?v=tVtOevLrt5U
- jph00 3y agoThat video shows examples of new models with one-shot performance better than old models with fine-tuning. It doesn't compare new models with fine-tuning to new models without fine-tuning.
- treprinum 3y agoRight, the page 28 of this article mentions "On some benchmarks, task-general models (not explicitly trained to perform a task) surpass prior state-of-the-art performance held by a task-specific model" but they don't provide data for fine-tuning of their very large models: https://arxiv.org/pdf/2206.07682.pdf https://arxiv.org/pdf/2206.07682.pdf
- ssivark 3y agoWhat’s your take on the argument that fine tuning might be destroying model calibration and causing it to overfit to aspects of the finetuning dataset eg: causing it to appear more confident, rather than becoming more correct? See https://twitter.com/animaanandkumar/status/1681690654060187648 https://twitter.com/animaanandkumar/status/16816906540601876...
- jph00 3y agoThat tweet is about a specific type of fine tuning: instruction tuning and RLHF. It causes to get better at the thing that it's being fine tuned for, which is making users feel better about the answer. Whether that's actually desirable in all cases is the question -- but that's not really related to whether fine tuning is a useful tool in general.
- Der_Einzige 3y agoGosh you are so wrong. Literally every bit of fine tuning and fine tuning related work is more important than ever. Being able to fine tune a giant model like GPT-4 would be a game changer. I don’t get why people like to come on here and tell blatant lies like this.
- treprinum 3y agoWhat would be my incentive to lie? I am getting absolutely breathtaking results from multi-task single shot prompts with one >1T model where I would need to fine-tune dozens of models to get anywhere close, and likely with less reliability. While fine-tuning might be important to bring some benefits to smaller models, in the really big models CoT is often much better.
- chaxor 3y agoGeneral models are useful to the general public, and they're useful as a starting point to find tune to a smaller model. The purpose matters. Research and most products benefit heavily from having a much smaller model (speed) to do one thing very well on trillions of documents. General chatting from big models is mostly a gimmick, but they're still very useful to find tune.
- treprinum 3y agoThe thing is that those super large models with CoT are often much more reliable than specialized fine-tuned smaller models you'd like to use on trillions of documents, as I am observing right now. You are not trading just speed but also quality with smaller models. The only relevant use case for fine-tuning for me recently was to apply wizard dataset to LLaMA 2 for a less censored model and that was done using LoRA.
- byt143 3y agoWhat tasks?
- royal__ 3y agoI would agree with other commenters that fine tuning is very much not obsolete, and for another important reason: many people and domains do not have the resources or even desire to work with extremely large models like GPT-4. The world outside of OpenAIs monoliths is still very much important.
- lmeyerov 3y agoYep as soon as query per second matters, and cost per query... Definitely going through that on some work now, where we use GPT4 on subset and rest on GPT3 or tuned. Only slow user-facing and other low-volume / high-latency goes to GPT4..
- winddude 3y agoGPT-4 is a fine tuned model.
- Blahah 3y agoThis claim is meaningless without being specific about the tasks.
- cypress66 3y agoFinetuning is more relevant than ever now. People are fine tuning LLaMA every single day.