3 ms·
Yeah, but it's not enough. The whole chip doesn't have enough SRAM to hold a tenth of GPT-3, let alone GPT-4. These are the kinds of models you would hope a chi
by modeless 6y ago
Yeah, but it's not enough. The whole chip doesn't have enough SRAM to hold a tenth of GPT-3, let alone GPT-4. These are the kinds of models you would hope a chip like this would help with, but as soon as the model gets too big to fit in the SRAM then the advantage goes way down. And I think the models of the future will continue to get much, much bigger.
- petra 6y agoThere are a few companies doing analog compute-in-memory for ML. Usually on flash processes. Maybe cerebras could fit well for that. As for the models of the future getting much bigger - is the GPT family representative ? Are we seeing the same growth is in other domains ? And isn't there a big niche for smaller than GPT models ?