4 ms·
I hope this is the future. Offline, small ML models, running inference on ubiquitous, inexpensive hardware. Models that are easy to integrate into other things,
by nine_k 1y ago
I hope this is the future. Offline, small ML models, running inference on ubiquitous, inexpensive hardware. Models that are easy to integrate into other things, into devices and apps, and even to drive from other models maybe.
- rohan_joshi 1y agoyeah totally. the quality of these tiny models are only going to go up.
- divamgupta 1y agoThat is our vision too!
- WhyNotHugo 1y agoDedicated single-purpose hardware with models would be even less energy-intensive. It's theoretically possible to design chips which run neural networks and alike using just resistors (rather than transistors). Such hardware is not general-purpose, and upgrading the model would not be possible, but there's plenty of use-cases where this is reasonable.
- amelius 1y agoBut resistors are, even in theory, heat dissipating devices. Unlike transistors, which can in theory be perfectly on or off (in both cases not dissipating heat).
- divamgupta 1y agoThe thing is that the new models keep coming every day. So it’s economically not feasible to make chips for a single model
- regularfry 1y agoIt's theoretically possible but physical "neurons" is a terrible idea. The number of connections between two layers of an FF net is the product of the number of weights in each, so routing makes every other problem a rounding error.
- theshrike79 1y agoThis is what Apple is envisioning with their SLMs, like having a model specifically for managing calendar events. It doesn't need to have the full knowledge of all humanity in it - just what it needs to manage the calendar.
- koolala 1y agoIssue is their envisioning everyone only using Apple products.
- theshrike79 1y agoJust like Google wants everyone to use their products. That’s how companies work. The tech is still public and the research is available
- throwaway28733 1y agoApple's hardware is notoriously overpriced, so I don't think they're envisioning that at all.
- DrBenCarson 1y agoIs it? The base $600 Mac and $150 Apple TV are easily two of the best deals in their market
- theshrike79 1y agoI'm still betting on a future AppleTV model being a full-on local LLM machine. This way they could offload as much of the "LLM" work on a device that lives in the home, all family linked phones and devices could use it for local inference. It's way overpowered as is anyway, why not use it for something useful.
- discardedrefuse 1y agoHmm. A pay once (or not at all) model that can run on anything? Or a subscription model that locks you in, and requires hardware that only the richest megacorps can afford? I wonder which one will win out.
- tracker1 1y agoThe popular one.
- divamgupta 1y agoThis is our goal too.