2 ms·
They do, but for inference at least, it's memory bandwidth that is the primary limiting factor for home LLMs right now, not raw compute.
by int_19h 3y ago
They do, but for inference at least, it's memory bandwidth that is the primary limiting factor for home LLMs right now, not raw compute.
- sroussey 3y agoWonder if the apple silicon ultra series will start using HBM3(e) on desktop in the future.