3 ms·
Counterargument: most people are not running directly the ML workloads, agreed - however Apple implements them as features of the OS (like the new GPT powered t
by weikju 3y ago
Counterargument: most people are not running directly the ML workloads, agreed - however Apple implements them as features of the OS (like the new GPT powered text predictions). So it WILL affect the regular users over time as new features are not made available for M1 users.
- candiodari 3y agoThe only reason you want to use bfloat16 is to train "compressed" models. Best inference performance tends to be at 4 bith quantization (and many people are trying to make it go down, even to 1 bit, and with sparsity, actually below that) So this just doesn't matter. Ironically, memory speed is what matters. And that's exactly the point where the M1 actually beats even the M3.
- weikju 3y agoThanks. I was going to rebut your point but I re-read the article and while one of the headings seem to rebut it, the actual content does not contradict anything, so I'll agree with your argument :)