3 ms·
I'd be kind of shocked if Nvidia isn't playing with this. I don't expect it's like super commercially viable today, but for sure things need to trend to radica
by pwarner 8mo ago
I'd be kind of shocked if Nvidia isn't playing with this.
I don't expect it's like super commercially viable today, but for sure things need to trend to radically more efficient AI solutions.
- saati 8mo agoThese are chips that become e-waste the second a better a model comes out, and nvidia is already limited by TSMC capacity.
- cyanydeez 8mo agoOnly in VC backed funding land. In the real world, theres talking refrigerators who dont need to know how to recite shakespeare.
- HPsquared 8mo agoOn the upside, Shakespeare isn't going to change soon.
- MarsIronPI 8mo agoSo you're saying we should burn Shakespeare onto a chip? /s
- throwthrowuknow 8mo agothese aren’t made for general chatbot use
- hamdingers 8mo agoThis is a ridiculous mindset. Llama 3.1 8B can do lots of things today and it'll still be able to do those things tomorrow. If you baked one of these into a smart speaker that could call tools to control lights and play music, it will still be able to do that when Llama 4 or 5 or 6 comes out.
- bigyabai 8mo agoIf you pay $1,500 for a Mistral ASIC that is beaten by a $15 Qwen ASIC that comes out six months later, you'd be feeling pretty dang ridiculous.
- hamdingers 8mo agoI'm equally capable of making up numbers to support my perspective but I don't see the point.
- bigyabai 8mo agoThe point is that the GP's mindset is not very ridiculous if you value things by a price/utility ratio. Software and hardware advancements will lead to buyer's remorse faster than people get an ROI from local inference.
- darkwater 8mo agoSW and HW advancements will bring this topic in the "good enough for vast majority" field, thus making GP point moot. You don't care if your LLM ASIC chip is not the latest one because it works for the use you purchased it for. The highly dynamical nature of LLM itself will make part of the advantage of upgradable software not that interesting anymorw. [1] [1] although security might be a big enough reason for upgrades to still be required
- dzhiurgis 8mo agoI'd pay for $100 chip that replaces anthropic sub and works 10x faster, even for 12 months. Edit: assuming model owners will let this happen, which they wont
- sowbug 8mo agoThey'll be perfect for an appliance like the Rick and Morty butter robot.