3 ms·
TPUs (short for Tensor Processing Units) are Google’s custom AI accelerator hardware which are completely separate from GPUs. I remember that introduced them in
by JLO64 1y ago
TPUs (short for Tensor Processing Units) are Google’s custom AI accelerator hardware which are completely separate from GPUs. I remember that introduced them in 2015ish but I imagine that they’re really starting to pay off with Gemini.
https://en.wikipedia.org/wiki/Tensor_Processing_Unit https://en.wikipedia.org/wiki/Tensor_Processing_Unit
- jxjnskkzxxhx 1y agoBelieve it or not, I'm also familiar with Wikipedia. It reads that they're optimized for low precisio high thruput. To me this sounds like a GPU with a specific optimization.
- flebron 1y agoPerhaps this chapter can help? https://jax-ml.github.io/scaling-book/tpus/ https://jax-ml.github.io/scaling-book/tpus/ It's a chip (and associated hardware) that can do linear algebra operations really fast. XLA and TPUs were co-designed, so as long as what you are doing is expressible in XLA's HLO language (https://openxla.org/xla/operation_semantics https://openxla.org/xla/operation_semantics), the TPU can run it, and in many cases run it very efficiently. TPUs have different scaling properties than GPUs (think sparser but much larger communication), no graphics hardware inside them (no shader hardware, no raytracing hardware, etc), and a different control flow regime ("single-threaded" with very-wide SIMD primitives, as opposed to massively-multithreaded GPUs).
- jxjnskkzxxhx 1y agoThank you for the answer! You see, up until now I had never appreciated that a GPU does more than matmuls... And that first reference, what a find :-) Edit: And btw, another question that I had had before was what's the difference between a tensor core and a GPU, and based on your answer, my speculative answer to that would be that the tensor core is the part inside the GPU that actually does the matmuls.
- deleted 1y ago[deleted]
- crazygringo 1y agoI mean yes. But GPU's also have a specific optimization, for graphics. This is a different optimization.
- kgwgk 1y agoDid you also read just after that "without hardware for rasterisation/texture mapping"? Does that sound like a _G_PU?
- jibal 1y agoYou asked a question, people tried to help, and you lashed out at them in a way that makes you look quite bad.