3 ms·
LLMs are few shots learners, that's why many people put examples into prompt, this is the next step.
by riku_iki 3y ago
LLMs are few shots learners, that's why many people put examples into prompt, this is the next step.
- ed 3y agoI don’t believe few shot performance dictates how quickly you can fine-tune. Most fine tunes will have much larger datasets (I am under the impression you want 10’s of thousands of examples for most runs). So I’m similarly impressed 20 examples would make such a big difference. But also note entity density decreases as example count increases. This is counterintuitive — maybe something else is going on here?
- jxnlco 3y agousually higher parameter models do better with less training data, seperate from few shot learners, but related in other ways.
- bravura 3y agohttps://github.com/huggingface/setfit https://github.com/huggingface/setfit gets good fine-tuned scores on some downstream tasks with just 8 labeled examples.