4 ms·
The processes of training and running use similar low-level operations, but training obviously requires a lot more of them. So it’s relevant to both, but one is
by xanderlewis 3y ago
The processes of training and running use similar low-level operations, but training obviously requires a lot more of them. So it’s relevant to both, but one is much more computation-heavy to begin with.
The point is that at the moment only a few of us are interested in training our own models, but this will change gradually as Apple introduces more machine learning models trained on our own local data behind the scenes. At that point, Macs with less support for this will start to slow down (well, or they’ll just not support those features). And the M1 chip will be worse in this regard than the M2.
- DeathArrow 3y agoAt that point, probably both M1 and M2 will be outdated. And people will sell their outdated laptops and buy M5 or M6 laptops. Worrying today about how your hardware will behave in 5 years from now wouldn't help you much.
- xanderlewis 3y agoSome of us only upgrade every ten years or so. ;-)
- eru 3y agoAs long as you don't mind your machine using lots of power when plugged in overnight (and training then), the old machines should be able to keep up for quite a while?
- adastra22 3y agoInference usually runs on much more quantized models, which don’t use bfloat16.