3 ms·

From the paper: > if the TPU were revised to have the same memory system as the K80 GPU, it would be about 30X - 50X faster than the GPU and CPU. Is it "hard"

From the paper: > if the TPU were revised to have the same memory system as the K80 GPU, it would be about 30X - 50X faster than the GPU and CPU. Is it "hard" to interface with GDDR5/HBM? Layout challenges? Or do they need the capacity more than the speed? Why wouldn't they have used faster memory than DDR3?

revelation 10y ago
Well HBM requires integrating the memory into the chip, presumably they didn't have the manufacturing capability or didn't want the spend required on a first version. No idea on GDDR5 vs DDR3, maybe they didn't like the latency of the first.
dom0 10y ago
Memory controllers are not so simple to do, and fast MCs also eat quite a bit of power. So a simple reason that they didn't do it could be either a) they did not want to license a more expensive, faster design b) while it would be faster, it would decrease efficiency to a point that did not meet their goals (for data centers, efficiency > absolute performance, within reasonable boundaries) c) like b) just with cost of memory d) GDDR5 and DDR3/4 have different design trade-offs. The former is optimized for sequential bandwidth (and low capacity; GDDR always was a point-to-point memory bus just to achieve the clock speeds), while DDR3/4 takes random read/write workloads into account (eh... to the amount possible with DRAM...) -- HBM requires a silicon interposer, which is basically like another complete chip (just without the FEOL parts, "just" metallization), that has to be significantly larger than the size of all chips combined. So unless you really need that performance or have a volume product it's unlikely to be a good deal.
- dom0 10y ago
  Looking at their block diagrams: They have two large "cache-like" structures: The unified buffer (24 MiB) and the accumulators (4 MiB). Bandwidth between these and the matrix multiply unit is high (167 GiB/s), bandwidth out of that complex is low. So it would seem that they just don't need a very large bandwidth out of that function complex.
hedgehog 10y ago
GDDR5: Getting the controller and PHY is difficult, not many companies have them and the ones that do (NVIDIA etc) don't have a strong interest in licensing them. The PHY is a mixed signal design so requires specialized expertise and likely some debugging on the target process so not only would it take a long time but it'd be expensive and very possibly flaky in rev 1. HBM: The interposer means you get a complex, and risky manufacturing process.

revelation 10y ago

Well HBM requires integrating the memory into the chip, presumably they didn't have the manufacturing capability or didn't want the spend required on a first version. No idea on GDDR5 vs DDR3, maybe they didn't like the latency of the first.

dom0 10y ago

Memory controllers are not so simple to do, and fast MCs also eat quite a bit of power. So a simple reason that they didn't do it could be either a) they did not want to license a more expensive, faster design b) while it would be faster, it would decrease efficiency to a point that did not meet their goals (for data centers, efficiency > absolute performance, within reasonable boundaries) c) like b) just with cost of memory d) GDDR5 and DDR3/4 have different design trade-offs. The former is optimized for sequential bandwidth (and low capacity; GDDR always was a point-to-point memory bus just to achieve the clock speeds), while DDR3/4 takes random read/write workloads into account (eh... to the amount possible with DRAM...) -- HBM requires a silicon interposer, which is basically like another complete chip (just without the FEOL parts, "just" metallization), that has to be significantly larger than the size of all chips combined. So unless you really need that performance or have a volume product it's unlikely to be a good deal.

dom0 10y ago
Looking at their block diagrams: They have two large "cache-like" structures: The unified buffer (24 MiB) and the accumulators (4 MiB). Bandwidth between these and the matrix multiply unit is high (167 GiB/s), bandwidth out of that complex is low. So it would seem that they just don't need a very large bandwidth out of that function complex.

hedgehog 10y ago

GDDR5: Getting the controller and PHY is difficult, not many companies have them and the ones that do (NVIDIA etc) don't have a strong interest in licensing them. The PHY is a mixed signal design so requires specialized expertise and likely some debugging on the target process so not only would it take a long time but it'd be expensive and very possibly flaky in rev 1. HBM: The interposer means you get a complex, and risky manufacturing process.