4 ms·

Not sure if you meant to laugh at a serious question. I am fully aware of my ignorance of the space. Since it appears you're in the deep learning hardware busi

Not sure if you meant to laugh at a serious question. I am fully aware of my ignorance of the space. Since it appears you're in the deep learning hardware business, what would be the impediment to using eDRAM or similar? eDRAM is too costly at those sizes for general purpose processors, but I imagine the reduced latency and increased bandwidth would be a huge win for a ridiculously parallel deep learning processor, and would definitely be a tradeoff worth making.

deepnotderp 10y ago
Sorry, that was more of a laugh at the state of deep learning model sizes than anything. Okay, so about eDRAM. There are two types of eDRAM: On-die and on-package. On-die eDRAM refers to manufacturing of DRAM cells on the logic die, which would be a big boon in terms of density since eDRAM cells can be almost 3x as dense as SRAM. The problem however, is that on-chip eDRAM has been impossible to scale beyond 40nm, which mitigates any advantages you would receive from using eDRAM. On-package eDRAM is more interesting but the primary cost in memory access is the physical transportation of the data, which is a physical limit and can't be circumvented. You can call it all sorts of fancy names such as "eDRAM", but the fact of the matter is that you're still moving data. For reference, the projected cost of movement of a 64-bit word on 10nm (ON CHIP) according to Lawrence Livermore national laboratories is ~1pJ, whereas the cost of a 64-bit FLOP is estimated to be 1pJ also. As you can see, the cost of data movement dwarfs the cost of computation. Of course, you gain a lot compared to DRAM, but HBM can offer the same efficiency gains of course. Didn't meant to be rude with the first response. Let me know if you have any other questions, I'd be happy to answer them :)

4 ms·

Not sure if you meant to laugh at a serious question. I am fully aware of my ignorance of the space. Since it appears you're in the deep learning hardware busi

by saosebastiao 10y ago

deepnotderp 10y ago
Sorry, that was more of a laugh at the state of deep learning model sizes than anything. Okay, so about eDRAM. There are two types of eDRAM: On-die and on-package. On-die eDRAM refers to manufacturing of DRAM cells on the logic die, which would be a big boon in terms of density since eDRAM cells can be almost 3x as dense as SRAM. The problem however, is that on-chip eDRAM has been impossible to scale beyond 40nm, which mitigates any advantages you would receive from using eDRAM. On-package eDRAM is more interesting but the primary cost in memory access is the physical transportation of the data, which is a physical limit and can't be circumvented. You can call it all sorts of fancy names such as "eDRAM", but the fact of the matter is that you're still moving data. For reference, the projected cost of movement of a 64-bit word on 10nm (ON CHIP) according to Lawrence Livermore national laboratories is ~1pJ, whereas the cost of a 64-bit FLOP is estimated to be 1pJ also. As you can see, the cost of data movement dwarfs the cost of computation. Of course, you gain a lot compared to DRAM, but HBM can offer the same efficiency gains of course. Didn't meant to be rude with the first response. Let me know if you have any other questions, I'd be happy to answer them :)