Thanks! Yes, I work as an HPC researcher, but not for Cerebras. I just watched the presentation of this paper at the SC20 Supercomputing conference last week.

Thanks! Yes, I work as an HPC researcher, but not for Cerebras. I just watched the presentation of this paper at the SC20 Supercomputing conference last week. I probably should have said, according to the specs, a CS-1 does have loads (TBs?) of off-chip DRAM to spill over into if the 18GiB isn't enough. That 18GiB is on-chip SRAM which you could think of as a giant 18GiB cache, which is astounding. Once you spill over that though, you're limited to how fast you can get data to the chip (probably multiple PCIe connections to a host CPU on the system with loads of RAM - I think the details are in the paper), so the results won't be nearly as impressive as this 10000x claim.