5 ms·

Computational Fluid Dynamics and implicit Finite Elements Analysis are still severely memory bandwidh limited on our 128 cores zen3 dual sockets servers. More b

by bobim 3y ago

Computational Fluid Dynamics and implicit Finite Elements Analysis are still severely memory bandwidh limited on our 128 cores zen3 dual sockets servers. More bandwidth is always welcome.

kookamamie 3y ago
Zen 3, as in Milan? Upgrade to Genoa and you'll see a very nice bump in performance due to DDR5 support with 12 memory channels, plus due to the added AVX512 instruction set.
- bayindirh 3y ago
  The problem is, unless you reorder and optimize your matrices to make the prefetcher happy, you end up throwing a lot of data, because you don't access the memory in the linear fashion where the prefetchers assume. So, having more channels is always nice, but this "more cowbells" approach doesn't bring in the performance you expect in some cases.
  - markhahn 3y ago
    more channels (and the fact that each ddr5 dimm is basically two channels) is a major win that doesn't depend on prefetching. it is perhaps underappreciated that CPU cores have gotten a lot more "thready" even in the absence of SMT, since each core has hundreds of instructions in flight. you know, kind of like how GPU cores (CUs, SMs) do...
    - bayindirh 3y ago
      It depends on prefetching very much. I have a similar simulation code, which can saturate the cores pretty efficiently (It was running with >1.7M integrations/sec/core and scaling linearly on last decade's hardware). But when you access to square submatrices inside a 3000x3000 matrix, and CPU prefetcher brings the whole n rows, and you discard most of the rows to get another part of the matrix, that bandwidth is wasted. Instead, you can rearrange your matrices to make the prefetcher happy, and do not throw what prefetcher brings, because you make it bring which is going to be used next. When I was doing last analysis runs, it was running with ~20% cache trash meaning I'm wasting 20% of the data I'm pulling in regardless. This is a huge loss, considering I was saturating memory bandwidth before the cores.
      - mastax 3y ago
        Which is an interesting aside, but unless I'm missing something the amount of wastage due to the prefetcher isn't at all proportional to the number of available memory channels so it's non sequitur.
        bayindirh 3y ago
        No, the wastage is due to because I'm not accessing data in a "linear" fashion if I lay out the matrices naively (actually pulling n by n square submatrices from the big matrix), and prefetcher brings in data with its baked in locality rules, but its assumptions doesn't fit my access pattern. So 20 percent of the data is thrown away. If I change how I fill and lay my matrices out, I can align my data with how prefetcher assumes, and way less data will be trashed during the process. I didn't do that because it needed a codebase-wide refactorization, and my code was already 30x faster than reference implementation with better accuracy, so we didn't pursue that avenue.
        bobim 3y ago
        Interesting, but is it even possible to optimize access to very sparse and unstructured matrices from finite elements?
        bayindirh 3y ago
        My matrices are dense. I’m working with boundary elements. But, there are sparse optimized libraries for efficient storage and access to sparse matrices (e.g.: Eigen has both sparse and dense sub-libraries). Also, there’s a whole literature concerned with sparse matrices and operations on those. However, my knowledge is limited on sparse side. If you want, I might try to dig into it a bit.
- bobim 3y ago
  I guess money is the rate determining step here. Secondary issue is that cores are also getting faster and then need more bandwidth. Solution is a low core count, maybe 2-3 per channel, more servers and serious interconnect, but cost…
Scene_Cast2 3y ago
I've heard that running CFD on thr CPU is still quite popular and prevalent, despite GPU implementations being widely available. What is it about the algorithm that makes GPUs less suitable? Model sizes? Branching? Data (non-)locality?
- bobim 3y ago
  Memory is one limiting factor, needing over 512 Gb is not uncommon. But complex physics like chemical reactions, particle tracking, any user field functions, is probably what is tricky to implement on a GPU.

by bobim 3y ago

Computational Fluid Dynamics and implicit Finite Elements Analysis are still severely memory bandwidh limited on our 128 cores zen3 dual sockets servers. More bandwidth is always welcome.

kookamamie 3y ago

Zen 3, as in Milan? Upgrade to Genoa and you'll see a very nice bump in performance due to DDR5 support with 12 memory channels, plus due to the added AVX512 instruction set.

bayindirh 3y ago
The problem is, unless you reorder and optimize your matrices to make the prefetcher happy, you end up throwing a lot of data, because you don't access the memory in the linear fashion where the prefetchers assume. So, having more channels is always nice, but this "more cowbells" approach doesn't bring in the performance you expect in some cases.
- markhahn 3y ago
  more channels (and the fact that each ddr5 dimm is basically two channels) is a major win that doesn't depend on prefetching. it is perhaps underappreciated that CPU cores have gotten a lot more "thready" even in the absence of SMT, since each core has hundreds of instructions in flight. you know, kind of like how GPU cores (CUs, SMs) do...
  - bayindirh 3y ago
    It depends on prefetching very much. I have a similar simulation code, which can saturate the cores pretty efficiently (It was running with >1.7M integrations/sec/core and scaling linearly on last decade's hardware). But when you access to square submatrices inside a 3000x3000 matrix, and CPU prefetcher brings the whole n rows, and you discard most of the rows to get another part of the matrix, that bandwidth is wasted. Instead, you can rearrange your matrices to make the prefetcher happy, and do not throw what prefetcher brings, because you make it bring which is going to be used next. When I was doing last analysis runs, it was running with ~20% cache trash meaning I'm wasting 20% of the data I'm pulling in regardless. This is a huge loss, considering I was saturating memory bandwidth before the cores.
    - mastax 3y ago
      Which is an interesting aside, but unless I'm missing something the amount of wastage due to the prefetcher isn't at all proportional to the number of available memory channels so it's non sequitur.
      - bayindirh 3y ago
        No, the wastage is due to because I'm not accessing data in a "linear" fashion if I lay out the matrices naively (actually pulling n by n square submatrices from the big matrix), and prefetcher brings in data with its baked in locality rules, but its assumptions doesn't fit my access pattern. So 20 percent of the data is thrown away. If I change how I fill and lay my matrices out, I can align my data with how prefetcher assumes, and way less data will be trashed during the process. I didn't do that because it needed a codebase-wide refactorization, and my code was already 30x faster than reference implementation with better accuracy, so we didn't pursue that avenue.
        bobim 3y ago
        Interesting, but is it even possible to optimize access to very sparse and unstructured matrices from finite elements?
        bayindirh 3y ago
        My matrices are dense. I’m working with boundary elements. But, there are sparse optimized libraries for efficient storage and access to sparse matrices (e.g.: Eigen has both sparse and dense sub-libraries). Also, there’s a whole literature concerned with sparse matrices and operations on those. However, my knowledge is limited on sparse side. If you want, I might try to dig into it a bit.
bobim 3y ago
I guess money is the rate determining step here. Secondary issue is that cores are also getting faster and then need more bandwidth. Solution is a low core count, maybe 2-3 per channel, more servers and serious interconnect, but cost…

Scene_Cast2 3y ago

I've heard that running CFD on thr CPU is still quite popular and prevalent, despite GPU implementations being widely available. What is it about the algorithm that makes GPUs less suitable? Model sizes? Branching? Data (non-)locality?

bobim 3y ago
Memory is one limiting factor, needing over 512 Gb is not uncommon. But complex physics like chemical reactions, particle tracking, any user field functions, is probably what is tricky to implement on a GPU.