5 ms·

It sounds like this would be great for single user contexts, but really unpredictable for servers running multiple duplicate tasks. Can someone who knows bette

It sounds like this would be great for single user contexts, but really unpredictable for servers running multiple duplicate tasks. Can someone who knows better than me please comment on the Linux server scheduling issues with a CPU like this. At this stage I’m assuming I’d be better with a CPU with all cores the same.

mastax 4y ago
It's a pretty similar shape of problem to NUMA, which servers have managed for quite some time. (Perhaps more similar to big.LITTLE which is not so common in servers but Linux should have decent support for from ARM SoCs) Its a bit of an unnecessary hassle and annoyance, though. I'm pretty sure all the V-Cache EPYCs will have cache on all their dies so there won't be this issue.
MichaelZuo 4y ago
Linux multi-threading code isn't the most elegant or robust out there, but I doubt it would catastrophically fail, in the sense of performing worse on the newer AMD design, outside of a few exceptional cases.
- andrewstuart 4y ago
  >> Linux multi-threading code isn't the most elegant or robust out there Evidence?
  - MichaelZuo 4y ago
    You want evidence of my personal estimation of Linux?
  - secondcoming 4y ago
    How about Linux will move your process to another core for no good reason?
    - ilyt 4y ago
      It usually moves it for good reason, and most bad reasons were eventually weeded out of the scheduler. For example if task previously running on core is waiting for something, the choice is to allow something else to run there (and hope it finished before that other thread comes back from waiting), or not use it at all and waste CPU capacity. Obviously scheduler will choose to use that now-idle core, but that might cause the previous process (which scheduler have no idea when it will come back) to be resheduled somewhere else, usually on its sibling (as in closest in memory hierarchy) core. Or power saving. If say core was idle (thread was waiting on something), it might actually be quicker to run it on other core (especially one close in cache hierarchy) than to wake up the core and then ramp up the clock speed of it. Here is some interesting if a bit outdated info about the nitty gritty: https://www.usenix.org/system/files/login/articles/login_winter16_02_lozi.pdf https://www.usenix.org/system/files/login/articles/login_win...
    - chupasaurus 4y ago
      How about running your process in a cgroup attached to a cpuset with the cores/NUMA nodes you want? How about cgroups v2 which has the threaded mode so a process can put it's threads into different subgroups?
rektide 4y ago
I'd rather have more cache than less. There are some corner cases where things will go wrong that we can dig up I'm sure, but generally, most tasks will execute at least as fast as they would have without the lopsided extra cache, and some will operate faster. There's definitely a lot of iteration & growing in here we could do to get better here. Yet... it feels like searching for problems to worry about some tasks running faster than they would have before (but not all tasks). In most deployment models, it should be fine. I think, for now, this slightly favors two use cases: one big monolith running on all cores, which will let work get consumed as it's ready. Or divide your workers onto the different chiplets, and use a good load balancing strategy that is somehow utilization aware. Personally I'm lacking in Fear, Uncertainty or Doubt over this making things worse.
- jackmott42 4y ago
  The benchmarks show it making plenty of things worse in the non gaming domain on this particular CPU. On a server cpu that is running lower clocks anyway, probably near impossible for it to hurt.
- AnonCoward42 4y ago
  Just a small reminder that the CCD with V-Cache has way lower boost clocks and therefore performs worse in almost everything beside gaming. For casual use this is irrelevant I think, but it is still significant in many workloads. The only workload I know where V-Cache benefits more than it is hurt by lower boost clocks is gaming. The advantage of the dual CCD design is that this only/mostly hurts in highly threaded workloads at least.
brnt 4y ago
It sounds great when paired with an APU. No such part was announced though.
- andrewstuart 4y ago
  APUs are very slow compared to discrete GPUs.
  - wtallis 4y ago
    A lot of that is due to the low memory bandwidth on mainstream CPU sockets: 128-bit bus, going up to around 6GT/s currently. That's half the memory bandwidth of NVIDIA's current entry-level laptop discrete GPU. And more L3 cache on a CPU directly addresses that memory bandwidth limitation (provided it's accessible to the iGPU, which would certainly be the case if AMD put 3D cache on an APU).
- wmf 4y ago
  The GPU cannot access the L3 though so V-cache would not help APUs. Maybe in the future AMD will have V-Infinity Cache.
  - brnt 4y ago
    What a pity!
- AnthonyMouse 4y ago
  All of the Ryzen 7000 series are APUs.
  - Dalewyn 4y ago
    With AMD finally taking a page from Intel with regards to providing an iGPU to everyone, Intel expected to integrate ARC into their CPUs with Meteor Lake (14th gen), and discrete GPUs from Nvidia and AMD being as bloody expensive as they are, I wonder if we're on the verge of a turning point in desktop computing as discrete GPUs go down the path once trodden by sound cards?
sudosysgen 4y ago
This platform exposes each CCX ( group of 4 cores which share the L3 between them) as a NUMA domain if you want. This means that if your workload really takes a huge performance penalty from the ~10% lower (still very high) clock speed, or only part of it really enjoys the cache, you can manually tell the OS to stay where you want. Scheduling for this kind of chip is not super, but it might improve. Meanwhile, you can enjoy almost all of the performance for the specific workloads where it matters by doing this.
wmf 4y ago
Recent discussion on that topic: https://news.ycombinator.com/item?id=35656374 https://news.ycombinator.com/item?id=35656374
toast0 4y ago
> Can someone who knows better than me please comment on the Linux server scheduling issues with a CPU like this. The scheduler was more or less designed around symmetric multi processing. BIG/little asymmetric systems will still have obviously preferred cores; if you're optimizing for throughput, add one task to each fast core first, then to each slow core, then maybe move tasks around to fit policies, etc. With the 7900X3D and the 7950X3D, it's trickier, because one chiplet has a lower clock speed but more cache. Tasks that fit into the smaller cache will do better on the less cache chiplet, and tasks that fit into the larger cache but not the smaller cache will do better on the larger cache chiplet, and tasks that don't fit into either task will probably do better on the faster chiplet, but it kind of depends. In order to make good decisions, the scheduler would need more information about the task's memory access patterns, and I don't think that's something schedulers tend to keep track of; but if this type of chip is common in the future, it will need to happen. For Epyc, AMD's server processor line, I believe the plan for X3D is to add cache to all the chiplets, keeping them roughly symmetric; there's still the modern situation that some cores will boost higher than others, and moving tasks to different cores can be very expensive if the process has memory still resident in cache on the old core, and the new core doesn't share that cache, etc.
ilyt 4y ago
Kinda depends on whatever is running on it. I can imagine just... putting different workload on cores with spicy cache vs normal ones. ... but overall it would be far easier to just buy 5 faster machines and 10 slower ones instead of having 15 "half fast half slow" and fuck around with scheduling The aforementioned 7950X3D is a very weird proposition. 7800X3D (8 cores all with spicy cache) have game performance that's near the same in most games so I guess value proposition is "you can just run other workload" (streaming) on other ones but that's not many people's use case...

mastax 4y ago

It's a pretty similar shape of problem to NUMA, which servers have managed for quite some time. (Perhaps more similar to big.LITTLE which is not so common in servers but Linux should have decent support for from ARM SoCs) Its a bit of an unnecessary hassle and annoyance, though. I'm pretty sure all the V-Cache EPYCs will have cache on all their dies so there won't be this issue.

MichaelZuo 4y ago

Linux multi-threading code isn't the most elegant or robust out there, but I doubt it would catastrophically fail, in the sense of performing worse on the newer AMD design, outside of a few exceptional cases.

andrewstuart 4y ago
>> Linux multi-threading code isn't the most elegant or robust out there Evidence?
- MichaelZuo 4y ago
  You want evidence of my personal estimation of Linux?
- secondcoming 4y ago
  How about Linux will move your process to another core for no good reason?
  - ilyt 4y ago
    It usually moves it for good reason, and most bad reasons were eventually weeded out of the scheduler. For example if task previously running on core is waiting for something, the choice is to allow something else to run there (and hope it finished before that other thread comes back from waiting), or not use it at all and waste CPU capacity. Obviously scheduler will choose to use that now-idle core, but that might cause the previous process (which scheduler have no idea when it will come back) to be resheduled somewhere else, usually on its sibling (as in closest in memory hierarchy) core. Or power saving. If say core was idle (thread was waiting on something), it might actually be quicker to run it on other core (especially one close in cache hierarchy) than to wake up the core and then ramp up the clock speed of it. Here is some interesting if a bit outdated info about the nitty gritty: https://www.usenix.org/system/files/login/articles/login_winter16_02_lozi.pdf https://www.usenix.org/system/files/login/articles/login_win...
  - chupasaurus 4y ago
    How about running your process in a cgroup attached to a cpuset with the cores/NUMA nodes you want? How about cgroups v2 which has the threaded mode so a process can put it's threads into different subgroups?

rektide 4y ago

I'd rather have more cache than less. There are some corner cases where things will go wrong that we can dig up I'm sure, but generally, most tasks will execute at least as fast as they would have without the lopsided extra cache, and some will operate faster. There's definitely a lot of iteration & growing in here we could do to get better here. Yet... it feels like searching for problems to worry about some tasks running faster than they would have before (but not all tasks). In most deployment models, it should be fine. I think, for now, this slightly favors two use cases: one big monolith running on all cores, which will let work get consumed as it's ready. Or divide your workers onto the different chiplets, and use a good load balancing strategy that is somehow utilization aware. Personally I'm lacking in Fear, Uncertainty or Doubt over this making things worse.

jackmott42 4y ago
The benchmarks show it making plenty of things worse in the non gaming domain on this particular CPU. On a server cpu that is running lower clocks anyway, probably near impossible for it to hurt.
AnonCoward42 4y ago
Just a small reminder that the CCD with V-Cache has way lower boost clocks and therefore performs worse in almost everything beside gaming. For casual use this is irrelevant I think, but it is still significant in many workloads. The only workload I know where V-Cache benefits more than it is hurt by lower boost clocks is gaming. The advantage of the dual CCD design is that this only/mostly hurts in highly threaded workloads at least.

brnt 4y ago

It sounds great when paired with an APU. No such part was announced though.

andrewstuart 4y ago
APUs are very slow compared to discrete GPUs.
- wtallis 4y ago
  A lot of that is due to the low memory bandwidth on mainstream CPU sockets: 128-bit bus, going up to around 6GT/s currently. That's half the memory bandwidth of NVIDIA's current entry-level laptop discrete GPU. And more L3 cache on a CPU directly addresses that memory bandwidth limitation (provided it's accessible to the iGPU, which would certainly be the case if AMD put 3D cache on an APU).
wmf 4y ago
The GPU cannot access the L3 though so V-cache would not help APUs. Maybe in the future AMD will have V-Infinity Cache.
- brnt 4y ago
  What a pity!
AnthonyMouse 4y ago
All of the Ryzen 7000 series are APUs.
- Dalewyn 4y ago
  With AMD finally taking a page from Intel with regards to providing an iGPU to everyone, Intel expected to integrate ARC into their CPUs with Meteor Lake (14th gen), and discrete GPUs from Nvidia and AMD being as bloody expensive as they are, I wonder if we're on the verge of a turning point in desktop computing as discrete GPUs go down the path once trodden by sound cards?

sudosysgen 4y ago

This platform exposes each CCX ( group of 4 cores which share the L3 between them) as a NUMA domain if you want. This means that if your workload really takes a huge performance penalty from the ~10% lower (still very high) clock speed, or only part of it really enjoys the cache, you can manually tell the OS to stay where you want. Scheduling for this kind of chip is not super, but it might improve. Meanwhile, you can enjoy almost all of the performance for the specific workloads where it matters by doing this.

wmf 4y ago

Recent discussion on that topic: https://news.ycombinator.com/item?id=35656374 https://news.ycombinator.com/item?id=35656374

toast0 4y ago

> Can someone who knows better than me please comment on the Linux server scheduling issues with a CPU like this. The scheduler was more or less designed around symmetric multi processing. BIG/little asymmetric systems will still have obviously preferred cores; if you're optimizing for throughput, add one task to each fast core first, then to each slow core, then maybe move tasks around to fit policies, etc. With the 7900X3D and the 7950X3D, it's trickier, because one chiplet has a lower clock speed but more cache. Tasks that fit into the smaller cache will do better on the less cache chiplet, and tasks that fit into the larger cache but not the smaller cache will do better on the larger cache chiplet, and tasks that don't fit into either task will probably do better on the faster chiplet, but it kind of depends. In order to make good decisions, the scheduler would need more information about the task's memory access patterns, and I don't think that's something schedulers tend to keep track of; but if this type of chip is common in the future, it will need to happen. For Epyc, AMD's server processor line, I believe the plan for X3D is to add cache to all the chiplets, keeping them roughly symmetric; there's still the modern situation that some cores will boost higher than others, and moving tasks to different cores can be very expensive if the process has memory still resident in cache on the old core, and the new core doesn't share that cache, etc.

ilyt 4y ago

Kinda depends on whatever is running on it. I can imagine just... putting different workload on cores with spicy cache vs normal ones. ... but overall it would be far easier to just buy 5 faster machines and 10 slower ones instead of having 15 "half fast half slow" and fuck around with scheduling The aforementioned 7950X3D is a very weird proposition. 7800X3D (8 cores all with spicy cache) have game performance that's near the same in most games so I guess value proposition is "you can just run other workload" (streaming) on other ones but that's not many people's use case...