3 ms·
How much detail do we know about the TPUs' design? Does Google disclose a block-diagram level? ISA details? Do they release a toolchain for low-level program
by wyldfire 9y ago
How much detail do we know about the TPUs' design? Does Google disclose a block-diagram level? ISA details? Do they release a toolchain for low-level programming or only higher-level functions like TensorFlow?
EDIT: I found [1] which describes "tensor cores", "vector/matrix units" and HBM interfaces. The design sounds similar in concept to GPUs. Maybe they don't have or need interpolation hw or other GPU features?
[1] https://cloud.google.com/tpu/docs/system-architecture https://cloud.google.com/tpu/docs/system-architecture
- elmarhaussmann 9y agoSo far only very few details are disclosed. Here are two presentations: https://supercomputersfordl2017.github.io/Presentations/ImageNetNewMNIST.pdf https://supercomputersfordl2017.github.io/Presentations/Imag... http://learningsys.org/nips17/assets/slides/dean-nips17.pdf http://learningsys.org/nips17/assets/slides/dean-nips17.pdf For the last version of the TPU, Google provided more detail, e.g., in this paper: https://arxiv.org/pdf/1704.04760.pdf https://arxiv.org/pdf/1704.04760.pdf Hopefully, Google will publish something similar for TPUv2, but I have no knowledge whether or when that might happen.
- jacksmith21006 9y agoGreat paper on the Generation 1 TPU. But Google has not shared much details on gen 2 and in some ways kind of hid information. Suspect we will need a gen 3 to get a paper on the gen 2. Here is the gen 1 paper and highly recommend. Pretty interesting using 65536 very simple cores. https://arxiv.org/ftp/arxiv/papers/1704/1704.04760.pdf https://arxiv.org/ftp/arxiv/papers/1704/1704.04760.pdf
- ndesaulniers 9y ago> Maybe they don't have or need interpolation hw or other GPU features? Definitely, no need to do any kind of rasterization here.