3 ms·
I'm curious to know what the cost is to switch contexts. Pure performance is amazing, but how long does it take to get a system going, to load the model and bui
by jauntywundrkind 1y ago
I'm curious to know what the cost is to switch contexts. Pure performance is amazing, but how long does it take to get a system going, to load the model and build context? What systems cans witch contexts while keeping the model non-destructively vs when is executing destructive?
I have a lot of questions about how models are run at scale; so curious to know more. With such a massive wafer as chip as Cerebras, it feels like perhaps switching might be even more consuming. Or maybe there's some brilliant strategy to have multiple contexts all loaded that it can flip between! Inventorying & using so much ram so spread out is it's own challenge!