3 ms·

> If your core is 50% waiting on RAM, then SMT into another thread to perform work while waiting on RAM. If your core is 50% waiting on RAM, then SMT into anot

> If your core is 50% waiting on RAM, then SMT into another thread to perform work while waiting on RAM. If your core is 50% waiting on RAM, then SMT into another thread, and that other thread will want some memory to work on, so it will also wait on RAM. On Top of it, this second thread now puts extra pressure on the memory subsystem, might cause cache evictions for the other thread, etc etc etc The moment that you include the memory subsystem into the SMT picture, SMT goes from a "no brainer; waiting on memory? do other work" to a "uhhh... i don't know if this makes things better or worse".

dragontamer 5y ago
Not quite. DDR4 and DDR5 have 50ns (single socket) to 150ns (dual socket) latency. For a 3GHz processor, that's 150 to 450 cycles. On any latency bound problem, SMT helps. However, what you say is true on bandwidth bound problems. Given the shear number of pointer hopping that happens in typical OOP code these days (or python / JavaScript), I expect SMT to be of big help to typical applications. DDR5 will double bandwidth in the near future. But that's not enough: HBM and GDDR6 have a possible future because you can only solve the bandwidth problem with more hardware. No tricks like SMT can help.
- zozbot234 5y ago
  Memory bandwidth is just barely trying to keep up with cores, frequencies and IPC amounts. Bandwidth available per core is still going to drop. So newer development workflows that optimize for this bottleneck are going to be very relevant.
  - dragontamer 5y ago
    Memory bandwidth does improve at least. Latency hasn't improved for the last 30 years. Tricks like SMT which can help mitigate the latency issue seem like the way forward.
  - oscardssmith 5y ago
    One thing that I think this will mean is the end of blas. Blas served us well for about 50 years, but one of the big problems it has is that it leads to code that takes multiple passes over memory. I think the future lies in systems that do code generation to better fuse loops of arbitrary code together. LoopVectorization.jl for example is able to generate blas level code for arbitrary computation, and as such can often be faster than blas since you don't have to use one of the specific hand optimized kernels.

dragontamer 5y ago

Not quite. DDR4 and DDR5 have 50ns (single socket) to 150ns (dual socket) latency. For a 3GHz processor, that's 150 to 450 cycles. On any latency bound problem, SMT helps. However, what you say is true on bandwidth bound problems. Given the shear number of pointer hopping that happens in typical OOP code these days (or python / JavaScript), I expect SMT to be of big help to typical applications. DDR5 will double bandwidth in the near future. But that's not enough: HBM and GDDR6 have a possible future because you can only solve the bandwidth problem with more hardware. No tricks like SMT can help.

zozbot234 5y ago
Memory bandwidth is just barely trying to keep up with cores, frequencies and IPC amounts. Bandwidth available per core is still going to drop. So newer development workflows that optimize for this bottleneck are going to be very relevant.
- dragontamer 5y ago
  Memory bandwidth does improve at least. Latency hasn't improved for the last 30 years. Tricks like SMT which can help mitigate the latency issue seem like the way forward.
- oscardssmith 5y ago
  One thing that I think this will mean is the end of blas. Blas served us well for about 50 years, but one of the big problems it has is that it leads to code that takes multiple passes over memory. I think the future lies in systems that do code generation to better fuse loops of arbitrary code together. LoopVectorization.jl for example is able to generate blas level code for arbitrary computation, and as such can often be faster than blas since you don't have to use one of the specific hand optimized kernels.