4 ms·
As far as I can tell from the current preview releases, the Copilot stuff still runs in the cloud, and it would be infeasible to run a general GPT-like chatbot
by runjake 3y ago
As far as I can tell from the current preview releases, the Copilot stuff still runs in the cloud, and it would be infeasible to run a general GPT-like chatbot on consumer PC hardware for a good while.
I imagine the NPU stuff is for assist with misc local AI/ML tasks, like photo editing and typing auto-correct ala what Apple's doing with their OSes.
- yjftsjthsd-h 3y ago> it would be infeasible to run a general GPT-like chatbot on consumer PC hardware for a good while. It's lower quality and slower than cloud, but I can run ollama on my aging laptops today; I wouldn't be surprised if the addition of hardware acceleration allowed new PCs to run a chatbot locally.
- runjake 3y agoYou can run Ollama on your aging laptop today, but at how many tokens per second and at how many parameters? Most consumer PC hardware isn't going to have a very capable GPU. But, I certainly anticipate "AI accelerators" in consumer PCs, in the short term, ala AI PC NPUs. FWIW, I basically leave Ollama running on my M3 Max and 24/7 and use it regularly, and it's ridiculously fast, but: M3 Max with lots of RAM and a crazy memory busy.