3 ms·

Yes, but then you end up competing directly with the price/performance of off-the-shelf hardware. PC memory bandwidth is the best deal in computing because it's

Yes, but then you end up competing directly with the price/performance of off-the-shelf hardware. PC memory bandwidth is the best deal in computing because it's amortized across so many units. I think it would be difficult for an FPGA to beat a $200 CPU and $60 MB that has 30-50 GiB/s RAM bandwidth. (Numbers are approximate)

dragontamer 9y ago
QDR-IV and RLDRAM3 are off-the-shelf parts. Go to any supplier like Digikey (https://www.digikey.com/product-detail/en/cypress-semiconductor-corp/CY7C4122KV13-106FCXC/CY7C4122KV13-106FCXC-ND/5802600 https://www.digikey.com/product-detail/en/cypress-semiconduc...) and you can order yourself some high-speed RAM today. An FPGA or ASIC would be needed to talk to these special RAM chips, but that's a problem that can be solved. > I think it would be difficult for an FPGA to beat a $200 CPU and $60 MB that has 30-50 GiB/s RAM bandwidth. Its not bandwidth that's the problem with a lot of these chips. Its latency. QDR-IV achieves 7.5 nanoseconds of latency, while L3-cache of Ryzen or Intel Skylake achieves 10-nanoseconds, while DDR4 has latency measured in 50nanoseconds to 100-nanoseconds, depending on the CPU that's accessing it. So yes. An off-the-shelf QDR-IV chip will achieve BETTER latency numbers than your off-the-shelf computer, by significant margins. Bandwidth is basically a solved problem with regards to ASICs or FPGAs: just add another channel. Talk to 10-chips, or 20-chips at a time to have 10x or 20x the bandwidth. Latency is the hard problem, which is why I'm focusing on it.
- marshray 9y ago
  Commodity DDR4 costs $10/GiB on Amazon. The part you linked costs $16,497/GiB at Digikey. That's 1600x more expensive, not even counting board fabrication costs. https://www.amazon.com/Crucial-Single-PC4-19200-288-Pin-Memory/dp/B019FRDFU0/ https://www.amazon.com/Crucial-Single-PC4-19200-288-Pin-Memo...
  - dragontamer 9y ago
    Something like Ethash would obviously use DDR4 to store the DAG. But something like Cryptonight (which hits 2MB EXTREMELY hard) would use QDR-IV or RLDRAM 3. An ASIC / FPGA designer can use whatever part they want, for whatever part of the algorithm they want, for as many "cores" as they want. If DDR4 is the best answer to some problem in cost/effectiveness (ie: Ethash's DAG storage), then DDR4 should be used. In the case of Ethash: there is a 16 MB inner loop, which would likely be best if it were run out of RLDRAM3. The DAG portion (which can be 4GBish, maybe 8GB to be safe) would be run from DDR4. EDIT: I initially thought it was 128kb inner loop. Double-checking the specs, its 16MB, which RLDRAM3 seems like a good bet on that size. Mix-and-match parts as you see fit and imagine the best combination. That's the advantage of FPGAs and ASICs: they literally can choose any part to better match a problem. Spend expensive memory only on the portions where it is necessary.
    - marshray 9y ago
      > there is a 16 MB inner loop, which would likely be best if it were run out of RLDRAM3 RLDRAM isn't stocked at Digikey, but it does list some prices. Far more reasonable at only 92x more expensive than commodity DDR4. Here is a 16 MiB RAM SRAM chip for $169 with lower random-access latency than you're likely to obtain going out to a separate package. As a bonus, it comes with its own CPU cores: AMD RYZEN 5 1500X 4-Core 3.5 GHz (3.7 GHz Turbo) Socket AM4 65W YD150XBBAEBOX I'd love to be proven wrong. What combination of FPGA and specialty RAM beats that? https://www.newegg.com/Product/Product.aspx?Item=N82E16819113436 https://www.newegg.com/Product/Product.aspx?Item=N82E1681911...
      - dragontamer 9y ago
        > Here is a 16 MiB RAM SRAM chip for $169 with lower random-access latency than you're likely to obtain going out to a separate package. As a bonus, it comes with its own CPU cores: AMD RYZEN 5 1500X 4-Core 3.5 GHz (3.7 GHz Turbo) Socket AM4 65W YD150XBBAEBOX Ryzen may have 16 MB of L3, but it can't actually hold a 16MB dataset. Ryzen's cache structure is 8MB + 8MB. So its physically impossible for even 4-cores to get more than 8MB in high-speed memory. I'm sorry, but 8MB+8MB does NOT allow for a 16MB dataset to exist. If somehow the 8MB+8MB were stored on L3 between two CCXs, you incur a major cross-CCX latency of 130 nanoseconds. In fact, I'm almost certain that CPU would rather pull from main-memory than attempt to keep cross-CCX L3 caches coherent. http://www.sisoftware.eu/2017/04/05/amd-ryzen-review-and-benchmarks-cache-and-memory/ http://www.sisoftware.eu/2017/04/05/amd-ryzen-review-and-ben... Some select quotes: > > Going inter-CCX increases the latency by 3 times to about 130ns thus if the threads are not properly scheduled you can end up with far higher latency and far less bandwidth. > > Within the same compute unit (sharing L3), the latency is ~45ns So lets be frank: Ryzen's 8MB+8MB L3 structure will NEVER hold a 16MB complete dataset. Its just insanely inefficient due to the 130ns cross-CCX penalty, and the memory controller would rather pull ~70ns to 95ns latency from main DDR4 instead (yeah, the cross-CCX penalty worse than an access to main memory!!) But if we drop the problem size down to 8MB, then we get a more reasonable 45ns latency from Ryzen's L3 cache. Because now we're truly executing out of L3. > I'd love to be proven wrong. What combination of FPGA and specialty RAM beats that? Consider that RLDRAM3 has a latency of 7.5ns (tRC worst-case to the same bank) and has a power-usage of 2 Watts. That tends to beat Ryzen's Cross-CCX latency of 130ns, DDR4 Latency of ~90ns, Ryzen L3 latency of 45ns, and power-usage of 65 Watts. Consider that RLDRAM3 is available in many different sizes, all the way up to 64MB per chip. 8 RLDRAM3 chips working together will also increase bandwidth and capacity by 8x. So Bandwidth isn't an issue either. Here's a price point btw if you want a readily available supply: https://www.arrow.com/en/products/mt44k16m36rb-125eatr/micron-technology https://www.arrow.com/en/products/mt44k16m36rb-125eatr/micro...

dragontamer 9y ago

QDR-IV and RLDRAM3 are off-the-shelf parts. Go to any supplier like Digikey (https://www.digikey.com/product-detail/en/cypress-semiconductor-corp/CY7C4122KV13-106FCXC/CY7C4122KV13-106FCXC-ND/5802600 https://www.digikey.com/product-detail/en/cypress-semiconduc...) and you can order yourself some high-speed RAM today. An FPGA or ASIC would be needed to talk to these special RAM chips, but that's a problem that can be solved. > I think it would be difficult for an FPGA to beat a $200 CPU and $60 MB that has 30-50 GiB/s RAM bandwidth. Its not bandwidth that's the problem with a lot of these chips. Its latency. QDR-IV achieves 7.5 nanoseconds of latency, while L3-cache of Ryzen or Intel Skylake achieves 10-nanoseconds, while DDR4 has latency measured in 50nanoseconds to 100-nanoseconds, depending on the CPU that's accessing it. So yes. An off-the-shelf QDR-IV chip will achieve BETTER latency numbers than your off-the-shelf computer, by significant margins. Bandwidth is basically a solved problem with regards to ASICs or FPGAs: just add another channel. Talk to 10-chips, or 20-chips at a time to have 10x or 20x the bandwidth. Latency is the hard problem, which is why I'm focusing on it.

marshray 9y ago
Commodity DDR4 costs $10/GiB on Amazon. The part you linked costs $16,497/GiB at Digikey. That's 1600x more expensive, not even counting board fabrication costs. https://www.amazon.com/Crucial-Single-PC4-19200-288-Pin-Memory/dp/B019FRDFU0/ https://www.amazon.com/Crucial-Single-PC4-19200-288-Pin-Memo...
- dragontamer 9y ago
  Something like Ethash would obviously use DDR4 to store the DAG. But something like Cryptonight (which hits 2MB EXTREMELY hard) would use QDR-IV or RLDRAM 3. An ASIC / FPGA designer can use whatever part they want, for whatever part of the algorithm they want, for as many "cores" as they want. If DDR4 is the best answer to some problem in cost/effectiveness (ie: Ethash's DAG storage), then DDR4 should be used. In the case of Ethash: there is a 16 MB inner loop, which would likely be best if it were run out of RLDRAM3. The DAG portion (which can be 4GBish, maybe 8GB to be safe) would be run from DDR4. EDIT: I initially thought it was 128kb inner loop. Double-checking the specs, its 16MB, which RLDRAM3 seems like a good bet on that size. Mix-and-match parts as you see fit and imagine the best combination. That's the advantage of FPGAs and ASICs: they literally can choose any part to better match a problem. Spend expensive memory only on the portions where it is necessary.
  - marshray 9y ago
    > there is a 16 MB inner loop, which would likely be best if it were run out of RLDRAM3 RLDRAM isn't stocked at Digikey, but it does list some prices. Far more reasonable at only 92x more expensive than commodity DDR4. Here is a 16 MiB RAM SRAM chip for $169 with lower random-access latency than you're likely to obtain going out to a separate package. As a bonus, it comes with its own CPU cores: AMD RYZEN 5 1500X 4-Core 3.5 GHz (3.7 GHz Turbo) Socket AM4 65W YD150XBBAEBOX I'd love to be proven wrong. What combination of FPGA and specialty RAM beats that? https://www.newegg.com/Product/Product.aspx?Item=N82E16819113436 https://www.newegg.com/Product/Product.aspx?Item=N82E1681911...
    - dragontamer 9y ago
      > Here is a 16 MiB RAM SRAM chip for $169 with lower random-access latency than you're likely to obtain going out to a separate package. As a bonus, it comes with its own CPU cores: AMD RYZEN 5 1500X 4-Core 3.5 GHz (3.7 GHz Turbo) Socket AM4 65W YD150XBBAEBOX Ryzen may have 16 MB of L3, but it can't actually hold a 16MB dataset. Ryzen's cache structure is 8MB + 8MB. So its physically impossible for even 4-cores to get more than 8MB in high-speed memory. I'm sorry, but 8MB+8MB does NOT allow for a 16MB dataset to exist. If somehow the 8MB+8MB were stored on L3 between two CCXs, you incur a major cross-CCX latency of 130 nanoseconds. In fact, I'm almost certain that CPU would rather pull from main-memory than attempt to keep cross-CCX L3 caches coherent. http://www.sisoftware.eu/2017/04/05/amd-ryzen-review-and-benchmarks-cache-and-memory/ http://www.sisoftware.eu/2017/04/05/amd-ryzen-review-and-ben... Some select quotes: > > Going inter-CCX increases the latency by 3 times to about 130ns thus if the threads are not properly scheduled you can end up with far higher latency and far less bandwidth. > > Within the same compute unit (sharing L3), the latency is ~45ns So lets be frank: Ryzen's 8MB+8MB L3 structure will NEVER hold a 16MB complete dataset. Its just insanely inefficient due to the 130ns cross-CCX penalty, and the memory controller would rather pull ~70ns to 95ns latency from main DDR4 instead (yeah, the cross-CCX penalty worse than an access to main memory!!) But if we drop the problem size down to 8MB, then we get a more reasonable 45ns latency from Ryzen's L3 cache. Because now we're truly executing out of L3. > I'd love to be proven wrong. What combination of FPGA and specialty RAM beats that? Consider that RLDRAM3 has a latency of 7.5ns (tRC worst-case to the same bank) and has a power-usage of 2 Watts. That tends to beat Ryzen's Cross-CCX latency of 130ns, DDR4 Latency of ~90ns, Ryzen L3 latency of 45ns, and power-usage of 65 Watts. Consider that RLDRAM3 is available in many different sizes, all the way up to 64MB per chip. 8 RLDRAM3 chips working together will also increase bandwidth and capacity by 8x. So Bandwidth isn't an issue either. Here's a price point btw if you want a readily available supply: https://www.arrow.com/en/products/mt44k16m36rb-125eatr/micron-technology https://www.arrow.com/en/products/mt44k16m36rb-125eatr/micro...