3 ms·
Why would a local model for a consumer device need 17k tok/s? Apple is better off building chips with generalizable TPUs (or equivalent) so they can upgrade/pa
by tyre 2mo ago
Why would a local model for a consumer device need 17k tok/s?
Apple is better off building chips with generalizable TPUs (or equivalent) so they can upgrade/patch models.
- MaxikCZ 2mo agoI cant shake the feeling of "640KB is enough for everybody". Imagine not one AI answering over 1 minute but a team of 100+ agents in hieararchical structure taking care of your request in seconds, checking each other.
- jeffybefffy519 2mo agoIt would open heaps of use cases, you could almost pass it over frames of images the camera sees in real time for example...
- deleted 2mo ago[deleted]