4 ms·

This benchmark makes no sense. I was looking for some reference points and found forum topic[1] with few PC benchmark results and they are `strange`. Only RTX 3

by terafo 5y ago

This benchmark makes no sense. I was looking for some reference points and found forum topic[1] with few PC benchmark results and they are `strange`. Only RTX 3090 benchmark result is around 12k in raster, while RTX 3080's result is 13k. And there are 2060's that run like 1050Ti(both giving result around 5k). And 1660 Super that outputs 10k with 3070 and 2070 giving similar results. There's something seriously wrong with this benchmark. [1] https://forum.affinity.serif.com/index.php?/topic/124022-benchmark-1900-results/ https://forum.affinity.serif.com/index.php?/topic/124022-ben...

klodolph 5y ago
Isn’t it possible that this is an unusual workload? That’s what the thread seems to be claiming.
- terafo 5y ago
  Performance doesn't scale with memory bandwidth, SM count or any other spec that is known to me. RTX 3090 and 3080 are not just similar cards from the same generation. They are the same GA102 die with some configuration differences : 3090 does have 24gb VRAM while 3080 has only 10. On 3090 82 SMs of initial 84 are active and on 3080 it's 68 SMs. And they both perform only marginally better than 1660 Super, while usually the difference between this cards is in 3x range. It's likely just poorly written software.
  - klodolph 5y ago
    “It’s likely just poorly written software” sounds completely glib to me, it sounds like an intuitive leap rather than a reasoned-out conclusion. Again, it sounds like an unusual workload. I don’t see why performance must scale with some spec in any particular straightforward way—I think we all remembered seeing workloads that would get faster once you added more CPU cores, then slower once you kept adding CPU cores—because back in the day, most of us didn’t have experience with systems that had lots of CPUs. GPUs are optimized to suit common workloads and, quite predictably, aren’t optimized to suit uncommon workloads. Modern graphics uses a mix of CPU and GPU work. The back-and-forth, especially returning results from the GPU to the CPU, has historically caused all sorts performance problems. We don’t have the explanation for why here, but I see no reason to conclude that it must be poorly written software just because the performance characteristics are unintuitive.
    - terafo 5y ago
      > GPUs are optimized to suit common workloads and, quite predictably, aren’t optimized to suit uncommon workloads. That doesn't explain why there are such drastic performance differences between GPUs on same uarch. It shouldn't perform better on clearly worse GPUs. > We don’t have the explanation for why here, but I see no reason to conclude that it must be poorly written software just because the performance characteristics are unintuitive. I'm not saying that it `must` be poorly written software. I'm just saying that it's `likely`, given niche nature of this software. If performance is affected by some unusual factor they have to clearly indicate that.
  - _kbh_ 5y ago
    Could the transfer speed over PCIE to the video card be the limiting factor then?. It would make sense why the M1 GPU bench very high as well as it likely has 200GB/s read to the shared memory compared to the ~32GB/s given by a PCIE4.0 16x slot.
    - terafo 5y ago
      Mac Pro with w6900x uses PCIe gen 3, which is 2x worse than gen 4. And given other results in that thread, I think it's unlikely to be the bottleneck. If it was, people with ryzen CPUs and 30 series cards would have much higher performance than they have on practice.
      - _kbh_ 5y ago
        That is true, perhaps its a difference in drivers and operating systems?. it hard to tell if the benchmark is bad without having the same setup to test against because anything could impact it really.
plorkyeran 5y ago
The thread specifically says that they bet on fast transfer on and off of the gpu becoming common, and then the exact opposite of that happened. When you care about a performance characteristic that no one else cares about, you're going to get very weird performance results and it's not a given that a gpu better for normal workloads will be better for you.
- terafo 5y ago
  Transfer on and off the GPU is limited mostly by PCIe and the latest Mac Pro that performs almost like M1 Max in rasterization has only PCIe3.0, while PCIe gen 4 with double the bandwidth is available on many PCs. If you look closely at the benchmarks performance doesn't correlate with PCIe bandwidth, and even if it did, it doesn't explain nearly 3x difference between 3090 and W6900X.
  - MobiusHorizons 5y ago
    IIUC on the M1, transfer happens using unified memory. This should mean the bandwidth is not limited by pcie but rather by the memory bandwidth, which was quite high if I remember correctly. I
    - terafo 5y ago
      Memory bandwidth in w6900x is roughly 500gb/s. In 3090 it's more than 900gb/s. Yet, 3090 performs like 5500M(which is laughable). In M1 Max bandwidth is 400gb/s(but there is 200gb/s version). I'm comparing only raster performance which should be heavily GPU dependent.
      - MobiusHorizons 5y ago
        The difference is that with the w6900x, the memory bandwidth is not used to transfer data between the dGPU and main memory. That happens over PCIe instead.
berkut 5y ago
Anyone know what GPU "codegen" it's using on PC? Tech specs say GPU needs Direct3D 12.0 capable card, but I'm not convinced that's for GPU compute, but maybe it is a compute shader?... I'd assume OpenCL for cross-platform (but then if they're specialising Metal for MacOS maybe they don't care about that, and are using Direct3D.
terafo 5y ago
Because I was confused with these benchmark results I tried to run it myself. And learned that current benchmark version and benchmark in affinity forum topic are not comparable. But still there results for that older benchmark version are quite strange. And for the reference: my 1080Ti outputs 13k, but there is huge variance run to run.
pjspycha 5y ago
The forum features people running multiple GPU's using onboard graphics as a secondary GPU and improving performance drastically, yet a few cases where the onboard GPU really hurt the benchmark. Some other people are seeing modest improvement, others are doubling their scores with this multi-gpu feature. Yeah... I'd take this benchmark with a grain of salt.

by terafo 5y ago

klodolph 5y ago

Isn’t it possible that this is an unusual workload? That’s what the thread seems to be claiming.

terafo 5y ago
Performance doesn't scale with memory bandwidth, SM count or any other spec that is known to me. RTX 3090 and 3080 are not just similar cards from the same generation. They are the same GA102 die with some configuration differences : 3090 does have 24gb VRAM while 3080 has only 10. On 3090 82 SMs of initial 84 are active and on 3080 it's 68 SMs. And they both perform only marginally better than 1660 Super, while usually the difference between this cards is in 3x range. It's likely just poorly written software.
- klodolph 5y ago
  “It’s likely just poorly written software” sounds completely glib to me, it sounds like an intuitive leap rather than a reasoned-out conclusion. Again, it sounds like an unusual workload. I don’t see why performance must scale with some spec in any particular straightforward way—I think we all remembered seeing workloads that would get faster once you added more CPU cores, then slower once you kept adding CPU cores—because back in the day, most of us didn’t have experience with systems that had lots of CPUs. GPUs are optimized to suit common workloads and, quite predictably, aren’t optimized to suit uncommon workloads. Modern graphics uses a mix of CPU and GPU work. The back-and-forth, especially returning results from the GPU to the CPU, has historically caused all sorts performance problems. We don’t have the explanation for why here, but I see no reason to conclude that it must be poorly written software just because the performance characteristics are unintuitive.
  - terafo 5y ago
    > GPUs are optimized to suit common workloads and, quite predictably, aren’t optimized to suit uncommon workloads. That doesn't explain why there are such drastic performance differences between GPUs on same uarch. It shouldn't perform better on clearly worse GPUs. > We don’t have the explanation for why here, but I see no reason to conclude that it must be poorly written software just because the performance characteristics are unintuitive. I'm not saying that it `must` be poorly written software. I'm just saying that it's `likely`, given niche nature of this software. If performance is affected by some unusual factor they have to clearly indicate that.
- _kbh_ 5y ago
  Could the transfer speed over PCIE to the video card be the limiting factor then?. It would make sense why the M1 GPU bench very high as well as it likely has 200GB/s read to the shared memory compared to the ~32GB/s given by a PCIE4.0 16x slot.
  - terafo 5y ago
    Mac Pro with w6900x uses PCIe gen 3, which is 2x worse than gen 4. And given other results in that thread, I think it's unlikely to be the bottleneck. If it was, people with ryzen CPUs and 30 series cards would have much higher performance than they have on practice.
    - _kbh_ 5y ago
      That is true, perhaps its a difference in drivers and operating systems?. it hard to tell if the benchmark is bad without having the same setup to test against because anything could impact it really.

plorkyeran 5y ago

The thread specifically says that they bet on fast transfer on and off of the gpu becoming common, and then the exact opposite of that happened. When you care about a performance characteristic that no one else cares about, you're going to get very weird performance results and it's not a given that a gpu better for normal workloads will be better for you.

terafo 5y ago
Transfer on and off the GPU is limited mostly by PCIe and the latest Mac Pro that performs almost like M1 Max in rasterization has only PCIe3.0, while PCIe gen 4 with double the bandwidth is available on many PCs. If you look closely at the benchmarks performance doesn't correlate with PCIe bandwidth, and even if it did, it doesn't explain nearly 3x difference between 3090 and W6900X.
- MobiusHorizons 5y ago
  IIUC on the M1, transfer happens using unified memory. This should mean the bandwidth is not limited by pcie but rather by the memory bandwidth, which was quite high if I remember correctly. I
  - terafo 5y ago
    Memory bandwidth in w6900x is roughly 500gb/s. In 3090 it's more than 900gb/s. Yet, 3090 performs like 5500M(which is laughable). In M1 Max bandwidth is 400gb/s(but there is 200gb/s version). I'm comparing only raster performance which should be heavily GPU dependent.
    - MobiusHorizons 5y ago
      The difference is that with the w6900x, the memory bandwidth is not used to transfer data between the dGPU and main memory. That happens over PCIe instead.

berkut 5y ago

Anyone know what GPU "codegen" it's using on PC? Tech specs say GPU needs Direct3D 12.0 capable card, but I'm not convinced that's for GPU compute, but maybe it is a compute shader?... I'd assume OpenCL for cross-platform (but then if they're specialising Metal for MacOS maybe they don't care about that, and are using Direct3D.

terafo 5y ago

Because I was confused with these benchmark results I tried to run it myself. And learned that current benchmark version and benchmark in affinity forum topic are not comparable. But still there results for that older benchmark version are quite strange. And for the reference: my 1080Ti outputs 13k, but there is huge variance run to run.

pjspycha 5y ago

The forum features people running multiple GPU's using onboard graphics as a secondary GPU and improving performance drastically, yet a few cases where the onboard GPU really hurt the benchmark. Some other people are seeing modest improvement, others are doubling their scores with this multi-gpu feature. Yeah... I'd take this benchmark with a grain of salt.