3 ms·
That's exactly what the next generation of 'accelerators' will be like. Whether it will be co-located with a GPU for consumer hardware remains to be seen. The
by fooker 3y ago
That's exactly what the next generation of 'accelerators' will be like.
Whether it will be co-located with a GPU for consumer hardware remains to be seen.
The thing to determine is how essential running LLMs locally is for consumers.
BigTech is pushing hard to make their clouds the only place to run LLMs unfortunately, so unless there is a killer app that is just better locally (like games were for GPUs), this might not change.
- boppo1 3y ago> unless there is a killer app that is just better locally Therapy & relationship bots, like the movie 'Her'. It's ugly, but it's coming.
- fooker 3y agoThere's no technical reason it has to be run locally. Massive privacy implications for sure, but people do consume all sorts of adult material online. Games though, no one has been able to make it work as well as local so far.