4 ms·
Hard to fully predict the future, of course, but changes are the answer is "no". Assuming you are comfortable with the current model performance rather than al
by cl42 2y ago
Hard to fully predict the future, of course, but changes are the answer is "no".
Assuming you are comfortable with the current model performance rather than always trying to have the most performant model...
1. Lots of researchers are looking at ways to reduce model size, inference time, etc... We see smaller models outperforming older benchmarks/achievements. Look into model distillation to see how this is done for specific benchmarks, or new approaches like Mixture of Experts (MOE) that reduce compute burden but get similar results as older models.
2. If your concern is cost of running a model, then GPU tech is also getting faster/better so even if compute requirements stay the same, the user will get an answer faster + will be cheaper to run due to the new hardware.
I hope that helps!
- jolieli 2y agosounds terrific. we can see the model companies are seeking ways to improve the model performance as the priority currently. More computing power is needed now. I do agree with your two ideas, but I am concerning a lot about whether the current GPUs are fully utilized or if they truly need more GPUs to improve the performance...
- jejeyyy77 2y agoAccording to Kaparthy, in his LLM talk, he mentions the limitation right now is compute and there are no signs otherwise. So in a sense, it is moores law all over again for GPUs + these models.