3 ms·
Along those lines, the graph didn't show Cerebras 2.6 TRILLION transistor wafer scale WSE-2 deep learning accelerator. Be interesting to see if wafer scale bec
by Nevermark 3y ago
Along those lines, the graph didn't show Cerebras 2.6 TRILLION transistor wafer scale WSE-2 deep learning accelerator.
Be interesting to see if wafer scale becomes more common. I would think wafer scale CPU core farms would help cloud providers deliver greater one-big-server goodness.
850,000 cores
40GB of ram distributed among cores
220Pb/s of memory bandwidth
Result: 7,500 trillion FP16 operations per second
Product page: https://www.cerebras.net/product-chip/ https://www.cerebras.net/product-chip/
Tech dive: https://www.techinsights.com/blog/cerebras-dives-wse-architecture https://www.techinsights.com/blog/cerebras-dives-wse-archite...
- saagarjha 3y agoIs it just me or is that very little RAM? What kind of workloads would you run on it?
- magicalhippo 3y agoThe accelerator contains 40GB of SRAM It's cache or scratchpad memory. Keep in mind this is "on chip", most systems, including this one, have additional memory off chip in the form of DRAM sticks. edit: For comparison, the NVIDIA A100 has up to 164kB of shared memory per SM while the wafer beast has about 47kB per core if you divide it out.