2 ms·
TPUs aren't transformer ASICs. The Ironwood TPU that Gemini was trained on was designed before LLMs became popular with ChatGPT's release. The architecture was
by zhemao 11mo ago
TPUs aren't transformer ASICs. The Ironwood TPU that Gemini was trained on was designed before LLMs became popular with ChatGPT's release. The architecture was general enough that it ended up being efficient for LLM training.
A special-purpose transformer inference ASIC would be like Etched's Sohu chip.
- mrinterweb 11mo ago> TPUs aren't transformer ASICs. https://cloud.google.com/tpu https://cloud.google.com/tpu > A TPU is an application-specific integrated circuit (ASIC) designed by Google for neural networks.