3 ms·

Is Microsoft putting a local model onto each Windows 11 install now? Would an Intel NPU built into Intel's CPUs make copilot perform better? I though Copilot wa

Is Microsoft putting a local model onto each Windows 11 install now? Would an Intel NPU built into Intel's CPUs make copilot perform better? I though Copilot was exclusively ran through a Bing i-frame-esque window in Windows 11? Unless they're planning to make locally run AI models that will take advantage of Intel's NPU's in the future and this is a step to prepare for it? Very interesting.

runjake 3y ago
As far as I can tell from the current preview releases, the Copilot stuff still runs in the cloud, and it would be infeasible to run a general GPT-like chatbot on consumer PC hardware for a good while. I imagine the NPU stuff is for assist with misc local AI/ML tasks, like photo editing and typing auto-correct ala what Apple's doing with their OSes.
- yjftsjthsd-h 3y ago
  > it would be infeasible to run a general GPT-like chatbot on consumer PC hardware for a good while. It's lower quality and slower than cloud, but I can run ollama on my aging laptops today; I wouldn't be surprised if the addition of hardware acceleration allowed new PCs to run a chatbot locally.
  - runjake 3y ago
    You can run Ollama on your aging laptop today, but at how many tokens per second and at how many parameters? Most consumer PC hardware isn't going to have a very capable GPU. But, I certainly anticipate "AI accelerators" in consumer PCs, in the short term, ala AI PC NPUs. FWIW, I basically leave Ollama running on my M3 Max and 24/7 and use it regularly, and it's ridiculously fast, but: M3 Max with lots of RAM and a crazy memory busy.
rawgabbit 3y ago
Knowing Microsoft's history, their intent is to suck up all of our Copilot chat history and responses. They will slowly get enough information about us where they even simulate a virtual clone of our online activity.

3 ms·

Is Microsoft putting a local model onto each Windows 11 install now? Would an Intel NPU built into Intel's CPUs make copilot perform better? I though Copilot wa

by scohesc 3y ago

runjake 3y ago
As far as I can tell from the current preview releases, the Copilot stuff still runs in the cloud, and it would be infeasible to run a general GPT-like chatbot on consumer PC hardware for a good while. I imagine the NPU stuff is for assist with misc local AI/ML tasks, like photo editing and typing auto-correct ala what Apple's doing with their OSes.
- yjftsjthsd-h 3y ago
  > it would be infeasible to run a general GPT-like chatbot on consumer PC hardware for a good while. It's lower quality and slower than cloud, but I can run ollama on my aging laptops today; I wouldn't be surprised if the addition of hardware acceleration allowed new PCs to run a chatbot locally.
  - runjake 3y ago
    You can run Ollama on your aging laptop today, but at how many tokens per second and at how many parameters? Most consumer PC hardware isn't going to have a very capable GPU. But, I certainly anticipate "AI accelerators" in consumer PCs, in the short term, ala AI PC NPUs. FWIW, I basically leave Ollama running on my M3 Max and 24/7 and use it regularly, and it's ridiculously fast, but: M3 Max with lots of RAM and a crazy memory busy.
rawgabbit 3y ago
Knowing Microsoft's history, their intent is to suck up all of our Copilot chat history and responses. They will slowly get enough information about us where they even simulate a virtual clone of our online activity.