4 ms·
From what’s presented this seems to be the lower end of Q&A and reasoning tasks and not long horizon agentic work. I agree that the search engine replacement AI
by throwthrowuknow 1mo ago
From what’s presented this seems to be the lower end of Q&A and reasoning tasks and not long horizon agentic work. I agree that the search engine replacement AI usage is something that can run anywhere (though it’s still better run in the cloud for speed, context length, sandboxing and convenience) but this isn’t the engine of AI growth.
Also, the average consumer is not going to be running a local model until they are built into the hardware they already buy and when they are, who is supplying the weights? They’ll likely be shipped as an ASIC (or MSIC) at that point anyways. Those will use a licensed model from the current leaders. The whole argument sounds like saying that cloud services shouldn’t be profitable because everyone has a computer at home or to meme “we have AI at home”.
- mcphage 1mo ago> Also, the average consumer is not going to be running a local model until they are built into the hardware they already buy and when they are, who is supplying the weights? Apple or Nvidia, presumably.
- throwthrowuknow 1mo agoHardware yes, weights? lol
- mcphage 1mo agoWhy not? Apple would have you download an updated set of weights with OSX / iOS updates. Nvidia could bundle them with driver updates. I don't think either would struggle to get people to keep their weights up to date.
- throwthrowuknow 1mo agoI’m sure it would be fine for simple routine tasks but it wouldn’t compete with OpenAI and Anthropic models
- mcphage 1mo agoMaybe, but that you’ll have to take up with the author of the article.
- ForHackernews 1mo ago> Those will use a licensed model from the current leaders. Says who? In the world of video decoding, H.265 is losing out to AV1 largely because it's not superior enough to H.264 to justify the expensive and complicated licensing. Do you really think chipmakers are going to pay a 25c per unit tax to get a model that is 15% better?
- esseph 1mo ago> Also, the average consumer is not going to be running a local model until they are built into the hardware they already buy and when they are, who is supplying the weights? They’ll likely be shipped as an ASIC (or MSIC) at that point anyways. Those will use a licensed model from the current leaders. I want somebody to make a local box: - router - wifi - thread/matter support - 1-3 nvme in RAID - GPU, running GPT-OSS OOTB with other models downloadable - running containers - allows people to share and store files with friends and family - runs a very simplified homekit - has phone app But with the prices of hardware right now... Probably out of reach for a general computing product.