3 ms·
This is probably a dumb question, that just shows my ignorance but I keep hearing on the consumer end of things that the M1-M4 chips are good at some AI. The
by ThinkBeat 3y ago
This is probably a dumb question, that just shows my ignorance
but I keep hearing on the consumer end of things that the M1-M4
chips are good at some AI.
The most important for me these days would be Photoshop, Resolve etc,
and I have seen those run a lot faster on Apple new proprietary chips
than on my older machines.
That may not translate well at all to what this chip can do or what
a H100s can do.
But does it translate at all?
Of course Apple is not selling their propritary chips either so for it
to be practical Apple would have to release some from of an external,
server something stuffed with their GPUs and AI chips
- singhrac 3y agoI’m also not quite an expert, but have benchmarked an M1 and various GPUs. The M* chips have unified memory and (especially Pro/Max/Ultra) have very high memory bandwidth even compared eg to a 1080 (an M1 Ultra has memory bandwidth between 2080 and 3090). At small batch sizes (including 1, like most local tasks), inference is bottlenecked by memory bandwidth, not compute ability. This is why people say the M* chips are good for ML. However H100s are used primarily for training (at enormous batch sizes) and require lots of interconnect to train large models. At that scale, arithmetic intensity is very high, and the M* chips aren’t very competitive (even if they could be networked) - they pick a different part of the Pareto power/efficiency curve than H100s which guzzle up power.