Try shoving a couple trill (estimate) parameters in memory. Or, shuttling it back and forth from disk as required to generate a single token. And then shuttling

Try shoving a couple trill (estimate) parameters in memory. Or, shuttling it back and forth from disk as required to generate a single token. And then shuttling that into the chip. Assuming they are using FP16 it's 4 TB give or take. They aren't kidding when they call them "Large".