11 ms·
They are saying that AMD's new Epyc Venice CPU has 16 memory channels allowing up to 1.6Tb/s of bandwidth. Which is higher bandwidth than most non-HBM GPUs. So
by barbacoa 2mo ago
They are saying that AMD's new Epyc Venice CPU has 16 memory channels allowing up to 1.6Tb/s of bandwidth. Which is higher bandwidth than most non-HBM GPUs.
So full CPU local AI inference may become viable option in coming years.
- lallysingh 2mo agoThis is essentially guaranteed. There are lots of useful smaller models that we should be able to run locally. Over time they'll be more and more capable and require less API usage.
- calgoo 2mo agoIm wondering if we are finally seeing the end of the "hard disk" era, and are entering a new era of vast instant on systems.
- adgjlsfhk1 2mo agothe GPU competition is using 16 gpus, so the actual comparison is that the CPU has <1/10th the bandwidth