4 ms·
Am i reading it right that the fine tune a model using 20 examples and 5 epochs? That seems really weird for me
by tobbe2064 3y ago
Am i reading it right that the fine tune a model using 20 examples and 5 epochs? That seems really weird for me
- isoprophlex 3y agoCan't overfit when your learning rate is zero! insert smart thinking meme
- smegsicle 3y ago[flagged]
- riku_iki 3y agoLLMs are few shots learners, that's why many people put examples into prompt, this is the next step.
- ed 3y agoI don’t believe few shot performance dictates how quickly you can fine-tune. Most fine tunes will have much larger datasets (I am under the impression you want 10’s of thousands of examples for most runs). So I’m similarly impressed 20 examples would make such a big difference. But also note entity density decreases as example count increases. This is counterintuitive — maybe something else is going on here?
- jxnlco 3y agousually higher parameter models do better with less training data, seperate from few shot learners, but related in other ways.
- bravura 3y agohttps://github.com/huggingface/setfit https://github.com/huggingface/setfit gets good fine-tuned scores on some downstream tasks with just 8 labeled examples.
- deleted 3y ago[deleted]