4 ms·
https://en.wikipedia.org/wiki/Tensor_Processing_Unit https://en.wikipedia.org/wiki/Tensor_Processing_Unit
by jhrmnn 4y ago
https://en.wikipedia.org/wiki/Tensor_Processing_Unit https://en.wikipedia.org/wiki/Tensor_Processing_Unit
- seydor 4y agobut those are not for sale, and not transformer-specific. There must be some optimizations that can be done in hardware and transformers are several years old now
- ruuda 4y agoYou likely already bought one. https://blog.google/products/pixel/introducing-google-tensor/ https://blog.google/products/pixel/introducing-google-tensor... https://apple.fandom.com/wiki/Neural_Engine https://apple.fandom.com/wiki/Neural_Engine
- jhrmnn 4y agoComputation-wise, transformers are really just a bunch of matrix multiplications, nothing more to it. (Which is partially why they’re so efficient and scalable.) Also, Nvidia’s GPU architectures are moving in the TPU direction (https://www.nvidia.com/en-us/data-center/tensor-cores/ https://www.nvidia.com/en-us/data-center/tensor-cores/).
- shaklee3 4y agoNvidia had tensor cores before the tpu existed
- jhrmnn 4y agoRight, correct. I was already thinking ahead about the recent TensorFloat precision, which is further directly targeting matrix multiplication.
- dragonwriter 4y ago> Nvidia had tensor cores before the tpu existed TPUs were released in 2016, Tensor Cores in 2017. But Edge TPUs (that is, the ones that exist outside of Google's Cloud infrastructure) weren’t until 2018. So, Nvidia had their cores available for people to buy first, but not before TPUs existed.
- dragonwriter 4y agoGoogle TPU are for sale [0] as are Nvidia cards with their Tensor Cores. As are Apple mainboards with their Neural Engine. [0] https://coral.ai/products/ https://coral.ai/products/