2 ms·
It's notable that they're so valuable because they feature 800Gbps of memory bandwidth. About twice what's available on the top end of M5, and exactly what make
by drawnwren 3mo ago
It's notable that they're so valuable because they feature 800Gbps of memory bandwidth. About twice what's available on the top end of M5, and exactly what makes llm inference fast.
- embedding-shape 3mo ago> because they feature 800Gbps of memory bandwidth. About twice what's available on the top end of M5 Ouch, about half of the memory bandwidth of a dedicated GPU though :/ Running LLMs on Apple hardware still doesn't make any sense to me.
- khalic 3mo agoM5 max has 614GB/s, you mean the m4?
- drawnwren 3mo agoOh, I was looking at every M5 except for the 40-core M5 Max. They have 460.