There is a fairly new ASIC named "Sohu" that is purpose-built for transformers. They have some bold claims that are impressive if true. I found a short discuss

There is a fairly new ASIC named "Sohu" that is purpose-built for transformers. They have some bold claims that are impressive if true. I found a short discussion[2] you may find useful. [1]: https://www.etched.com/ https://www.etched.com/ [2]: https://www.lesswrong.com/posts/qhpB9NjcCHjdNDsMG/new-fast-transformer-inference-asic-sohu-by-etched https://www.lesswrong.com/posts/qhpB9NjcCHjdNDsMG/new-fast-t...