3 ms·
Stupid question, but why aren't they doing it already? What's their bottleneck right now?
by jjjjj55555 3y ago
Stupid question, but why aren't they doing it already? What's their bottleneck right now?
- danielmarkbruce 3y agoTry shoving a couple trill (estimate) parameters in memory. Or, shuttling it back and forth from disk as required to generate a single token. And then shuttling that into the chip. Assuming they are using FP16 it's 4 TB give or take. They aren't kidding when they call them "Large".