4 ms·
On device is still meh. Realistically Apple needs to put a few teams on it, and have built in support for running LLMs on OSX. I assume if you integrate this
by 999900000999 5mo ago
On device is still meh.
Realistically Apple needs to put a few teams on it, and have built in support for running LLMs on OSX.
I assume if you integrate this on an OS level you might be able to pull off some additional tricks. So far most of the tools for running LLMs locally have been driven by volunteers or very small startups.
Microsoft could also do this, but they're too busy trying to upsell everyone. Microsoft wants to sell you on paying for each token.
- CharlieDigital 5mo agoApple has the edge here since they fully control the hardware and can baseline their implementation. It feels like Microsoft's task is much harder because there's a much broader range of hardware and capabilities that they need to support.
- 999900000999 5mo agoMicrosoft sells hardware. Surface laptops. Microsoft has spent a lot of time telling people to buy AI ready PCs. I just don't think it's in their business model to offer on device LLM. They want you to subscribe to Copilot. Which encapsulates the entire enshitication of Windows. Microsoft demands more money. Subscribe to gamepass, subscribe to OneDrive!
- CharlieDigital 5mo agoI'm aware of the Surface line, but 1) they don't own the hardware stack nearly to the same degree as Apple does, 2) it makes up only a small slice of the ecosystem and is a more premium product as far as PCs go. Windows ecosystem has far more variance in terms of CPU + GPU capabilities. I can see Microsoft pushing the subscription angle, but I do think there is a place for on-device capabilities since those will always be more responsive for small tasks.
- 999900000999 5mo agoI agree with your points. While I'm just thinking out loud, I can imagine either anthro or open AI releasing hardware for on-prem solutions in the near future. I believe a few small companies have already specialized in this. But imagine a situation where maybe you pay a small subscription and Open AI allows you to download non open source models guaranteed to work to a degree. It's going to be an interesting decade.
- zihotki 5mo agoMS is more like assembling the surfaces, it's not the same as having a full vertical integration like Apple is trying to achieve with their contracts with Intel, in-house modem, etc.