4 ms·
I think there is a mismatch with Intel/Window's idea of an AI PC being a beefed up edge TPU core, while customers who do want AI want LLMs, which I'm under the
by jdboyd 1y ago
I think there is a mismatch with Intel/Window's idea of an AI PC being a beefed up edge TPU core, while customers who do want AI want LLMs, which I'm under the impression the Lunar Lake AI features aren't that great at accelerating (but are presumably good at non LLM vision and audio processing). If I'm misunderstanding this, I'd love a reply.
- hnuser123456 1y agoHaving a chatgpt style experience locally requires about 24GB of dedicated VRAM at a bandwidth of about 1 TB/s. These "AI PCs" are nothing close to this. They can only run much smaller neural networks for a small variety of data operations. Think of like, photoshop filters. Any AI chat experience on an AI PC is still being sent over network to a datacenter. Some new photoshop filters does not define a whole new category of computer.
- bigbadfeline 1y agoExactly. The problem is Intel's AI PC might be PCs but they are too under-powered for AI work. Apple makes real AI PCs and they sell well enough, the latest AMD chips are getting into that territory as well. I want to see Intel succeed but they repeatedly miss the boat by over-promising and under-delivering, as if they're afraid of providing too much value too soon.
- hnuser123456 1y agoIt gets me thinking though, what would an AI PC really be? It would be a category of laptop you could buy that could be guaranteed to run decent local LLMs at realistically usable speeds, and there's an app out of the box for it, which exposes and ever advertises an API endpoint and invites the user to write their own code to use their own computer to its capabilities, also allowing them to swap out the model. But of course, going from cloud to local is the opposite of what MS is trying to do.