3 ms·
Slightly off-topic: I remember seeing news about the enormous chip Cerebras was/is selling (pdf https://f.hubspotusercontent30.net/hubfs/8968533/WSE-2%20Datash
by JamesCoyne 4y ago
Slightly off-topic:
I remember seeing news about the enormous chip Cerebras was/is selling (pdf https://f.hubspotusercontent30.net/hubfs/8968533/WSE-2%20Datasheet.pdf https://f.hubspotusercontent30.net/hubfs/8968533/WSE-2%20Dat...).
Has there been any indication that the LLMs released in the last few months use exotic hardware like this, or is it all "standard" hardware?
- wmf 4y agoOpenAI uses Nvidia GPUs and Google uses their TPUs.
- brucethemoose2 4y agoYou might see more training on Intel XPUs when they come out, since they have such enormous RAM pools. Maybe AMD MI300s and Intel Ponte Vecchio (both 128GB) in the shorter term, though I think they will mostly be in HPC supercomputers instead of cloud instances.
- ipsum2 4y agoEveryone except Google uses Nvidia for training. Cerebras, Gaudi, and other custom AI accelerators have unable to surpass Nvidia in performance/$ and performance/watt yet.