3 ms·

Consider how GPUs & CPUs actually use memory: On a GPU, the memory access patterns are often very wide & easily controlled, such as fetching a block of texture

Consider how GPUs & CPUs actually use memory: On a GPU, the memory access patterns are often very wide & easily controlled, such as fetching a block of texture memory. This is like streaming reads for a disk, where you can actually just throw essentially RAID 0 at the problem to scale up really easily. So scaling all the way up to ridiculously wide 2048-bit bus widths (ex. AMD Vega 64) is completely viable since you're working with relatively bulk array data almost constantly. Now most GPUs aren't going quite that wide to hit the speeds they want, but consider that 192-bit is still considered "low-end" here and that's already a lot wider than you'll find on most CPUs. On a CPU, especially in languages like Java, JavaScript, etc..., memory access patterns are often very random and very small. Consider just something like a simple boolean accessor call `if (foo->isEnabled()) doThing();` - you're reading let's say 8 bytes (foo pointer) to in turn access 1 byte (a boolean). Now the CPU is already reading in chunks of 64-bytes typically (cache line width), so you're already blowing 2 memory fetches to get 128-bytes of data just to read 9 bytes total. Going wider, which is how GPUs scale bandwidth, is utterly pointless here. You'd just be fetching even more data you didn't want in the first place just to occupy precious cache space on something you're probably not going to use anyway. In fact this is why DDR5 actually essentially shrinks the bus-width, from 64-bit in prior DDR generations to 2x32-bit in DDR5

YetAnotherNick 3y ago
> especially in languages like Java, JavaScript, etc Not only there is sequential array in both the language, it is very often used. Things like iterating the array is one of the most common operation and is almost always memory bandwidth bound even for interpreted language.
- logicchains 3y ago
  >Not only there is sequential array in both the language, it is very often used In Java only primitive arrays are memory-efficient for iteration; ArrayLists are not because every element is boxed, so even when iterating sequentially a lookup of some random memory address is needed for every element.
dathinab 3y ago
> very random and very small a major part of JVM, JS Engines and even memory allocators, pools and similar is to make this patters less random, more predictable and more local. For example a common optimization for lisp like languages it to layout a linked list in a continuous way in memory. Stuff like that is needed even if you have more memory bandwidth as sometimes even more important then memory bandwidth is cache locality, which is WAY harder to scale then adding more memory channels. And branch prediction a major performance factor for modern CPUs also benefits quite a lot from cache locality. Furthermore having more bandwidth also only helps if you can bring the bandwidth to each core as needed. But if you ad more memory bandwidth it still has a limit per memory region which can be an issue. So in practice nearly all applications benefit from more bandwidth but the degree they do benefit is limited outside of highly paralelizable use-cases (like some graphics and AI use-case). So for most typical consumer use cases it's not worth the additional (high) cost this would impose to various aspects (especially motherboards). And for pro-sumer/server CPUs _we do have more channels_! But always a balance between added cost and expected need for more bandwidth. > In fact this is why DDR5 AFIK the reason they DDR5 has a split channel is way more complicated then that
- kllrnohj 3y ago
  > a major part of JVM, JS Engines and even memory allocators, pools and similar is to make this patters less random, more predictable and more local. They don't, though. The JVM memory model doesn't even really allow it. This is a big reason for value types, but that keeps getting perpetually delayed...
Zardoz84 3y ago
> On a CPU, especially in languages like Java, JavaScript, etc..., memory access patterns are often very random and very small. Consider just something like a simple boolean accessor call `if (foo->isEnabled()) doThing();` - you're reading let's say 8 bytes (foo pointer) to in turn access 1 byte (a boolean). Now the CPU is already reading in chunks of 64-bytes typically (cache line width), so you're already blowing 2 memory fetches to get 128-bytes of data just to read 9 bytes total. Going wider, which is how GPUs scale bandwidth, is utterly pointless here. You'd just be fetching even more data you didn't want in the first place just to occupy precious cache space on something you're probably not going to use anyway. In fact this is why DDR5 actually essentially shrinks the bus-width, from 64-bit in prior DDR generations to 2x32-bit in DDR5 I think that you are mixing ram channels with ram channel width. Channels, could be fetching totally different address at same time. On a multi core world this is very helpful. One core could being fetching a single 32 bit integer, and another core could being fetching a a struct of 128 bytes from another address. Also, server CPUs usually have many more RAM lanes that desktop CPUs . For example, EPYC server CPU could have 12 RAM channels!

YetAnotherNick 3y ago

> especially in languages like Java, JavaScript, etc Not only there is sequential array in both the language, it is very often used. Things like iterating the array is one of the most common operation and is almost always memory bandwidth bound even for interpreted language.

logicchains 3y ago
>Not only there is sequential array in both the language, it is very often used In Java only primitive arrays are memory-efficient for iteration; ArrayLists are not because every element is boxed, so even when iterating sequentially a lookup of some random memory address is needed for every element.

dathinab 3y ago

> very random and very small a major part of JVM, JS Engines and even memory allocators, pools and similar is to make this patters less random, more predictable and more local. For example a common optimization for lisp like languages it to layout a linked list in a continuous way in memory. Stuff like that is needed even if you have more memory bandwidth as sometimes even more important then memory bandwidth is cache locality, which is WAY harder to scale then adding more memory channels. And branch prediction a major performance factor for modern CPUs also benefits quite a lot from cache locality. Furthermore having more bandwidth also only helps if you can bring the bandwidth to each core as needed. But if you ad more memory bandwidth it still has a limit per memory region which can be an issue. So in practice nearly all applications benefit from more bandwidth but the degree they do benefit is limited outside of highly paralelizable use-cases (like some graphics and AI use-case). So for most typical consumer use cases it's not worth the additional (high) cost this would impose to various aspects (especially motherboards). And for pro-sumer/server CPUs _we do have more channels_! But always a balance between added cost and expected need for more bandwidth. > In fact this is why DDR5 AFIK the reason they DDR5 has a split channel is way more complicated then that

kllrnohj 3y ago
> a major part of JVM, JS Engines and even memory allocators, pools and similar is to make this patters less random, more predictable and more local. They don't, though. The JVM memory model doesn't even really allow it. This is a big reason for value types, but that keeps getting perpetually delayed...

Zardoz84 3y ago

> On a CPU, especially in languages like Java, JavaScript, etc..., memory access patterns are often very random and very small. Consider just something like a simple boolean accessor call `if (foo->isEnabled()) doThing();` - you're reading let's say 8 bytes (foo pointer) to in turn access 1 byte (a boolean). Now the CPU is already reading in chunks of 64-bytes typically (cache line width), so you're already blowing 2 memory fetches to get 128-bytes of data just to read 9 bytes total. Going wider, which is how GPUs scale bandwidth, is utterly pointless here. You'd just be fetching even more data you didn't want in the first place just to occupy precious cache space on something you're probably not going to use anyway. In fact this is why DDR5 actually essentially shrinks the bus-width, from 64-bit in prior DDR generations to 2x32-bit in DDR5 I think that you are mixing ram channels with ram channel width. Channels, could be fetching totally different address at same time. On a multi core world this is very helpful. One core could being fetching a single 32 bit integer, and another core could being fetching a a struct of 128 bytes from another address. Also, server CPUs usually have many more RAM lanes that desktop CPUs . For example, EPYC server CPU could have 12 RAM channels!