Inference is much more common than training just as using a word processor is much more common than writing one. Even from a training standpoint, there are a l

Inference is much more common than training just as using a word processor is much more common than writing one. Even from a training standpoint, there are a lot of interesting things you can do with DNN models that aren't at the large end (GPT-3/ChatGPT/DALL-E/Stable Diffusion/etc) of the spectrum. Right now there's not really a coherent open source stack to even run pre-trained models on a GPU even though most of the pieces are open source... so I find it a bit baffling that no one is even talking about trying to put the pieces together. The whole GPU compute world still basically needs to be side-loaded on many distros today with the core of it (the GPU compute drivers) still proprietary. There are things like RustICL coming along that give me some hope but I'm more lamenting what appears to be a lack of interest in the larger open source world in changing things even when all the pieces are there. If open source seems 'a bit' (lol) behind on mobile, that's nothing compared to how it's going to feel on the desktop in a few years.