2 ms·
I agree that access is a problem now, but I think it is one that hardware improvements will solve very quickly. We are a few generations of Strix Halo type hard
by noelwelsh 1y ago
I agree that access is a problem now, but I think it is one that hardware improvements will solve very quickly. We are a few generations of Strix Halo type hardware away from effortlessly running very good LLMs locally. (It's already possible, but the hardware is about $2000 and the LLMs you can run are good but not very good.) AFAIK AMD have not released the roadmap for Medusa Halo, but the rumours [1] are increased CPU and GPU performance, and increased bandwidth. Another iteration or two of this will make Strix Halo hardware more affordable, and the top-of-the-line models will be beasts for local LLMs.
[1]: https://www.notebookcheck.net/Powerful-Zen-6-Medusa-Halo-iGPU-could-challenge-desktop-GPUs-with-20-more-CUs-and-50-wider-bus.971869.0.html https://www.notebookcheck.net/Powerful-Zen-6-Medusa-Halo-iGP...