5 ms·
That sounds like an enormously expensive exercise.
by profsummergig 2mo ago
That sounds like an enormously expensive exercise.
- ronsor 2mo agoAt <50M parameters, training costs are completely trivial. You'll spend a lot more on your rent this month.
- HenryNdubuaku 2mo agoHaha, my rent is cheap lol
- salamo 2mo agoAs someone who's done something similar (https://blog.lukesalamone.com/posts/creating-tiny-semantic-search/ https://blog.lukesalamone.com/posts/creating-tiny-semantic-s...) the expensive part wasn't the training itself but the data curation and evaluation post-training. For this, getting a reasonable distribution of tool calls when the tool call can be anything isn't easy. Once you have that, the model is small enough batch sizes are probably enormous and training can probably be done on a consumer-grade GPU in a week or less. Or even faster on a bigger GPU.
- genxy 2mo agoYou can train a model of this size on your laptop in a day.
- profsummergig 2mo agoSo is the challenge, now, getting the right sort of data?
- kadoban 2mo agoTraining scales pretty badly, so smaller models like this are really not that bad in terms of cost.
- ComputerGuru 2mo agoAt this size, it certainly doesn’t have to be.