3 ms·
For the first time in a long time, consumer hardware is not even close to being fast enough for a new type of mass applications: GenAI. People have stopped car
by aurareturn 3y ago
For the first time in a long time, consumer hardware is not even close to being fast enough for a new type of mass applications: GenAI.
People have stopped caring about consumer chips because they have been "good enough" for a long time.
Hardware is getting exciting again. We need significantly more compute, memory bandwidth, and memory capacity. There are huge bottlenecks everywhere.
- wvenable 3y agoFor GenAI, I wonder if we need consumer hardware to be that powerful. Isn't this exactly what "the cloud" is for?
- aurareturn 3y agoI think the best models will always require "the cloud" but there will be plenty of use cases where "good enough" local LLMs need to be on-device.
- kmeisthax 3y agoIt depends. If you're OpenAI, consumer hardware should never be powerful enough to run local models. If you're Apple, it should.