3 ms·
As someone else points out, training is slightly more involved, but I also find that these smaller models are next to worthless compared to the larger ones. Th
by smokel 2y ago
As someone else points out, training is slightly more involved, but I also find that these smaller models are next to worthless compared to the larger ones.
There are probably some situations where it suffices to use a small model, but for most purposes, I'd prefer to use the state of the art, and I'm eager for that state to progress a little more.
- vineyardmike 2y ago> but for most purposes, I'd prefer to use the state of the art I'm guessing in the future, we'll see a lot more automatic inference "on our behalf", and you won't care or notice if its using Llama5:3b or whatever comes out then. I'm betting that in a few years we'll see LLMs baked into a ton of stuff - lots of "simple" things mostly, like email summary/rewording, and similar "light touch" use cases. Maybe light multi-modal work like photo labeling. Probably expanded to be running in every other SaaS applications doing god knows what. For those, the small models would be more than enough, and much cheaper to run in large volumes. I'm guessing "chat with a bot to ask questions" will be a small amount of the inference that happens on our behalf, but will use the valuable SOTA model use case.