5 ms·

Are there any applications that are RAM bandwidth-bound, though? The main bottleneck is supposed to be RAM latency. Going from single channel to dual channel o

by E6300 9y ago

Are there any applications that are RAM bandwidth-bound, though? The main bottleneck is supposed to be RAM latency. Going from single channel to dual channel offers like an 8% performance increase, IIRC. Is there any reason to expect any different with quad channel RAM?

dijit 9y ago
I can chime in and say that memory bandwidth is our primary bottleneck on servers at my current company.
- E6300 9y ago
  The only situation where I imagine that could happen is if you need to apply a small number of instructions to a massive data set that's fully loaded in memory. What sort of application are you running? If you can say, obviously.
  - dijit 9y ago
    That's exactly the case. I run dedicated gameservers. We preload "phases" which you can affect and transition to/from as a player.
  - merb 9y ago
    > The only situation There are many more: - Caches/Databases that should keep as much stuff in memory as possible (i.e. if you have a 32gb es instance you actually gain a lot from "fast memory") - In-Memory Databases - In-Memory RPC/message broker which of course has as a limitation factor the memory bandwidth. In all cases the memory bandwidth might be a important factor.
    - E6300 9y ago
      For all the cases you mention, the critical factor is the product of the average transaction size and the transaction count per second. As long as this value is smaller than the RAM bandwidth, the application will not be RAM bandwidth-bound. Generally speaking, databases are kept in memory to minimize latency, not maximize throughput. Bandwidth is not really a problem. Having to update 10 GB/s of a database would be highly unusual. Having to get data from random positions in a disk or SSD is much more common. As for the message broken, it's not clear to me why the bandwidth would "of course" be the limiting factor.
  - sliken 9y ago
    For sufficiently large values of small. An 8 core CPU @ 3 GHz runs 24 Billion clocks/sec. In any of which a CPU can execute multiple instructions. Two channel memory systems assuming DDR4@2400 can do about 40GB/sec. Thread ripper is about double that (of the skylake-x, but NOT the kabylake-x). The new skylake xeons are 6 channel (about 120GB/sec) and the new AMD Epyc is 8 channel (about 160GB/sec). Assuming a perfectly sequential access pattern and something simple like a=b+c (which reads 16 bytes and writes 8 bytes) you can run 1.6 billion of those a second. So to not be memory bound you need to run an extra 15 times more instructions... without adding any cache misses, just to execute one instruction per cycle (a fraction of the possible). If it's less you are memory bound. Now imagine it's not perfectly sequential, and instead you have to retrieve something from memory before you know where to go next. Like say a database index, binary tree, or linked list. Instead of getting 8 bytes @ 2400 Mhz you get 8 bytes per 70 ns. Keep in mind that's 8 bytes per 1/2.4 ns vs 70 or 168 times worse. Suddenly instead of needing 15 times more instructions you need 2500 instructions per memory load, all without a extra cache miss. So as you can see it can be quite easy to be memory limited. Sure some things do an amazing amount of calculations on very little data. But many things are data intensive, which justifies the large ram and large memory bandwidth machines that make up pretty much all servers shipped today. Memory bandwidth is expensive (CPU package, pins, sockets, motherboard traces, additional motherboard layers, power, etc), but well justified in many cases.
    - E6300 9y ago
      You are again conflating high bandwidth and low latency. > So to not be memory bound you need to run an extra 15 times more instructions... without adding any cache misses, just to execute one instruction per cycle (a fraction of the possible). If it's less you are memory bound. Yes, if you do operations in a lineal access pattern like this the performance will be bound by the bandwidth. This is the situation I was referring to above. > Now imagine it's not perfectly sequential, and instead you have to retrieve something from memory before you know where to go next. Like say a database index, binary tree, or linked list. Instead of getting 8 bytes @ 2400 Mhz you get 8 bytes per 70 ns. Keep in mind that's 8 bytes per 1/2.4 ns vs 70 or 168 times worse. > Suddenly instead of needing 15 times more instructions you need 2500 instructions per memory load, all without a extra cache miss. > So as you can see it can be quite easy to be memory limited. No, in this case you will not be limited by the bandwidth, but by the latency. Having more bandwidth will do nothing, because at 8 bytes per 70 ns you're only moving about 109 MiB/s. If 100% of the memory accesses are cache misses (they won't be) and the application uses all cores then yes, doubling the number of memory channels will double the multi-thread performance (unless channel count = core count), although the single-threaded performance will stay unchanged. Additionally, in this particular load you could get away with relatively low frequency RAM, which won't significantly affect the latency but will lower the total bandwidth (it will still be way higher than 109 MiB/s) and will be cheaper.
sliken 9y ago
Well what exactly do you mean by latency? Say 8 cores are randomly accessing memory. A quad channel system will have twice the throughput of a 2 channel because there can be twice as many cache misses being handled at once. For this reason many generations of HEDT and server chips have had 4 channels for many years and are quite justified. Take a 5 year old opteron or xeon for instance, or even an sandy bridge ( i7 like the i7-3820). Sandy bridge is 5 generations old, if 4 channels was justified then it's definitely justified today with today's faster and more numerous cores. The X-series is new branding, but Intel has been selling i7 chips with the LGA-2011 socket supporting 4 memory channels for years. So sure if you are cache friendly, great, as many cores as you can fit in a socket. But many applications aren't that cache friendly.
- E6300 9y ago
  > A quad channel system will have twice the throughput of a 2 channel because there can be twice as many cache misses being handled at once. Sure, that's the theory, but in practice it doesn't seem to make much of a difference, at least not for dual vs. single.
  - sliken 9y ago
    There exists cache friendly applications that see zero to minimal change with more bandwidth or more channels. There also exists cache unfriendly applications that see large changes with more bandwidth or more channels. Games generally are cache friendly, many easy benchmarks are cache friendly. But generally more aggressive use of a machine (which is presumably why you buy a top spec CPU) is generally less cache friendly. Also people notice worst case performance much more than average or best case. Audio skipping, user interface lag, etc. You can see this effect in action when you compare single thread performance to multithead performance using every CPU. L1 caches are generally note shared, so if it's less than N times faster for N CPUs you are seeing software overhead (the cost of synchronization) or cache misses (in L1, L2, or L3) or of course main memory bottlenecks. I've seen plenty of cases on older servers where running on all CPUs of single socket was FASTER than all CPUs of two sockets, but that's much less common these days because each socket has it's own memory system. I can assure you that the entire server market and high end desktop market isn't running 2 to 8 time the memory bandwidth just for fun. The bandwidth is expensive and justified.

by E6300 9y ago

dijit 9y ago

I can chime in and say that memory bandwidth is our primary bottleneck on servers at my current company.

E6300 9y ago
The only situation where I imagine that could happen is if you need to apply a small number of instructions to a massive data set that's fully loaded in memory. What sort of application are you running? If you can say, obviously.
- dijit 9y ago
  That's exactly the case. I run dedicated gameservers. We preload "phases" which you can affect and transition to/from as a player.
- merb 9y ago
  > The only situation There are many more: - Caches/Databases that should keep as much stuff in memory as possible (i.e. if you have a 32gb es instance you actually gain a lot from "fast memory") - In-Memory Databases - In-Memory RPC/message broker which of course has as a limitation factor the memory bandwidth. In all cases the memory bandwidth might be a important factor.
  - E6300 9y ago
    For all the cases you mention, the critical factor is the product of the average transaction size and the transaction count per second. As long as this value is smaller than the RAM bandwidth, the application will not be RAM bandwidth-bound. Generally speaking, databases are kept in memory to minimize latency, not maximize throughput. Bandwidth is not really a problem. Having to update 10 GB/s of a database would be highly unusual. Having to get data from random positions in a disk or SSD is much more common. As for the message broken, it's not clear to me why the bandwidth would "of course" be the limiting factor.
- sliken 9y ago
  For sufficiently large values of small. An 8 core CPU @ 3 GHz runs 24 Billion clocks/sec. In any of which a CPU can execute multiple instructions. Two channel memory systems assuming DDR4@2400 can do about 40GB/sec. Thread ripper is about double that (of the skylake-x, but NOT the kabylake-x). The new skylake xeons are 6 channel (about 120GB/sec) and the new AMD Epyc is 8 channel (about 160GB/sec). Assuming a perfectly sequential access pattern and something simple like a=b+c (which reads 16 bytes and writes 8 bytes) you can run 1.6 billion of those a second. So to not be memory bound you need to run an extra 15 times more instructions... without adding any cache misses, just to execute one instruction per cycle (a fraction of the possible). If it's less you are memory bound. Now imagine it's not perfectly sequential, and instead you have to retrieve something from memory before you know where to go next. Like say a database index, binary tree, or linked list. Instead of getting 8 bytes @ 2400 Mhz you get 8 bytes per 70 ns. Keep in mind that's 8 bytes per 1/2.4 ns vs 70 or 168 times worse. Suddenly instead of needing 15 times more instructions you need 2500 instructions per memory load, all without a extra cache miss. So as you can see it can be quite easy to be memory limited. Sure some things do an amazing amount of calculations on very little data. But many things are data intensive, which justifies the large ram and large memory bandwidth machines that make up pretty much all servers shipped today. Memory bandwidth is expensive (CPU package, pins, sockets, motherboard traces, additional motherboard layers, power, etc), but well justified in many cases.
  - E6300 9y ago
    You are again conflating high bandwidth and low latency. > So to not be memory bound you need to run an extra 15 times more instructions... without adding any cache misses, just to execute one instruction per cycle (a fraction of the possible). If it's less you are memory bound. Yes, if you do operations in a lineal access pattern like this the performance will be bound by the bandwidth. This is the situation I was referring to above. > Now imagine it's not perfectly sequential, and instead you have to retrieve something from memory before you know where to go next. Like say a database index, binary tree, or linked list. Instead of getting 8 bytes @ 2400 Mhz you get 8 bytes per 70 ns. Keep in mind that's 8 bytes per 1/2.4 ns vs 70 or 168 times worse. > Suddenly instead of needing 15 times more instructions you need 2500 instructions per memory load, all without a extra cache miss. > So as you can see it can be quite easy to be memory limited. No, in this case you will not be limited by the bandwidth, but by the latency. Having more bandwidth will do nothing, because at 8 bytes per 70 ns you're only moving about 109 MiB/s. If 100% of the memory accesses are cache misses (they won't be) and the application uses all cores then yes, doubling the number of memory channels will double the multi-thread performance (unless channel count = core count), although the single-threaded performance will stay unchanged. Additionally, in this particular load you could get away with relatively low frequency RAM, which won't significantly affect the latency but will lower the total bandwidth (it will still be way higher than 109 MiB/s) and will be cheaper.

sliken 9y ago

Well what exactly do you mean by latency? Say 8 cores are randomly accessing memory. A quad channel system will have twice the throughput of a 2 channel because there can be twice as many cache misses being handled at once. For this reason many generations of HEDT and server chips have had 4 channels for many years and are quite justified. Take a 5 year old opteron or xeon for instance, or even an sandy bridge ( i7 like the i7-3820). Sandy bridge is 5 generations old, if 4 channels was justified then it's definitely justified today with today's faster and more numerous cores. The X-series is new branding, but Intel has been selling i7 chips with the LGA-2011 socket supporting 4 memory channels for years. So sure if you are cache friendly, great, as many cores as you can fit in a socket. But many applications aren't that cache friendly.

E6300 9y ago
> A quad channel system will have twice the throughput of a 2 channel because there can be twice as many cache misses being handled at once. Sure, that's the theory, but in practice it doesn't seem to make much of a difference, at least not for dual vs. single.
- sliken 9y ago
  There exists cache friendly applications that see zero to minimal change with more bandwidth or more channels. There also exists cache unfriendly applications that see large changes with more bandwidth or more channels. Games generally are cache friendly, many easy benchmarks are cache friendly. But generally more aggressive use of a machine (which is presumably why you buy a top spec CPU) is generally less cache friendly. Also people notice worst case performance much more than average or best case. Audio skipping, user interface lag, etc. You can see this effect in action when you compare single thread performance to multithead performance using every CPU. L1 caches are generally note shared, so if it's less than N times faster for N CPUs you are seeing software overhead (the cost of synchronization) or cache misses (in L1, L2, or L3) or of course main memory bottlenecks. I've seen plenty of cases on older servers where running on all CPUs of single socket was FASTER than all CPUs of two sockets, but that's much less common these days because each socket has it's own memory system. I can assure you that the entire server market and high end desktop market isn't running 2 to 8 time the memory bandwidth just for fun. The bandwidth is expensive and justified.