4 ms·
I thought the whole deal with machine learning was that inference is easy, but training is hard. So creating the models from the data is what takes the billion
by m463 3y ago
I thought the whole deal with machine learning was that inference is easy, but training is hard.
So creating the models from the data is what takes the billions of transistors on many cards.
running the models doesn't need so much hardware.
also... with respect to powerful processing - I haven't seen graphics cards tapering off. There's a ravenous demand for better graphics hardware each year. For every technique that is commoditized, the next year there's a new way of doing things that is better. I remember things like lighting, or realistic hair, or physics or whatever making a new graphics card better. Why wouldn't AI stuff be any different?
- rahimnathwani 3y agoinference is easy, but training is hard That's often true for classification models (like image recognition), but it the generation models (SD, Llama and GPT-4) everyone is excited about use a lot of compute for inference.
- kramerger 3y agoIt may be "a lot" in relative terms, but still a few order of magnitude lower than training.