3 ms·
Do you know why GPT-J is being used instead of NeoX or any of the other larger open source models?
by Taek 4y ago
Do you know why GPT-J is being used instead of NeoX or any of the other larger open source models?
- joshhart 4y ago7B is a sweet spot where you can do something with limited resources both for training and inference. Going beyond that you spill out of an A100 without tricks. We will continue iterating on this with other models.
- dragonwriter 4y agoIf fine tuning a small model, which can be run on consumer hardware once trained, provides quality results, why use a larger base model?