3 ms·
It’s not just about small models, that’s only one part of evolution Some groups are baking models into silicone, Deepmind has an example, it gets 18,000 tokens
by yieldcrv 3mo ago
It’s not just about small models, that’s only one part of evolution
Some groups are baking models into silicone, Deepmind has an example, it gets 18,000 tokens/sec on Llama 3.1, not sure about parameter size
- intrasight 3mo agoI think this is the future - at least it will be for on-device models. Apple, for instance, will "bake silicon" once a year for their current model, and use that chip in all their devices.
- fooker 3mo ago> Some groups are baking models into silicone While some other groups are baking silicone into models :)
- yieldcrv 3mo agoThose are doctors fooker! but yes misplaced e