I don't think they rely on SRAM very much for training. https://cerebras.ai/blog/the-complete-guide-to-scale-out-on-cerebras-wafer-scale-clusters https://cerebr

I don't think they rely on SRAM very much for training. https://cerebras.ai/blog/the-complete-guide-to-scale-out-on-cerebras-wafer-scale-clusters https://cerebras.ai/blog/the-complete-guide-to-scale-out-on-... outlines the memory architecture but it seems like they are able to keep most of the storage off wafer which is how they scale to 100s of GB of parameters with "only" 10s of GB of SRAM.