5 ms·

I can write SIMD code that's 2x the speed of the non-vectorized code but I can also rewrite that same SIMD code so that 2x becomes 6x. Point being, not only th

I can write SIMD code that's 2x the speed of the non-vectorized code but I can also rewrite that same SIMD code so that 2x becomes 6x. Point being, not only that you can get 2x on top of initial 2x SIMD implementation but usually much more. Whether or not you see SIMD in the codegen is not a testament of how good the implementation really is. IPC is the relevant measure here.

adgjlsfhk1 2y ago
IPC looks like 3.5 instructions per clock. (and the speed for bigger inputs will be memory bound rather than compute bound).
- menaerus 2y ago
  If true this would be between ok-ish and decent result but since you didn't say anything about the experiment or how you ran it or anything about the HW it's hard to say more. I assume you didn't use https://github.com/miguelraz/StagedFilters.jl/blob/master/test/benchmarks.jl https://github.com/miguelraz/StagedFilters.jl/blob/master/te... since that benchmark is pretty much flawed for more than a few reasons: - Operates with the dataset that cannot fit even in the L3 cache on most consumer machines: ~76MB (float64) and ~38MB (float32) - Intermixes python with the julia code - Not even a single repetition - no statistical significance - Does not show what happens with different data distributions, e.g. random, but only uses a monotonically increasing sequence
  - ChrisRackauckas 2y ago
    Did you link the wrong script? The script you show runs everything to statistical significance using Chairmarks.@b. Also, I don't understand what the issue would be with mixing Python and Julia code in the benchmark script. The Julia side JIT compiles the invocations which we've seen removes pretty much all non-Python overhead and actually makes the resulting SciPy calls faster than doing so from Python itself in many cases, see for example https://github.com/SciML/SciPyDiffEq.jl?tab=readme-ov-file#measuring-overhead https://github.com/SciML/SciPyDiffEq.jl?tab=readme-ov-file#m... where invocations from Julia very handily outperform SciPy+Numba from the Python REPL. Of course, that is a higher order function so it's benefiting from the Julia JIT in other ways as well, but the point is in previous benchmarks we've seen the overhead floor as so low (~100ns IIRC) that it didn't effect benchmarks negatively for Python and actually improved many Python timings in practice. Though it would be good to show in this case what exactly the difference is in order to isolate any potential issue, I would be surprised if it's more than a 100ns overhead for an invocation like this and with 58ms being the benchmark size, that cost is well below the noise floor). Though trying different datasets is of course valid. There's no reason to reject a benchmark just because it doesn't fit into L3 cache, there's many use cases for that. But it does not mean that all use cases are likely to see such a result.
    - gauge_field 2y ago
      [dead]
    - menaerus 2y ago
      If it doesn't fit into the L3 cache then what the benchmark is measuring is not the CPU efficiency of your SIMD approach but also inherently includes different (and complex) effects of L1+L2+L3 cache hierarchy, memory prefetchers and main memory latency. Latency of accessing the RAM is alone ~two to ~three orders of magnitude larger then the latency of executing the single instruction. Per-se this isn't wrong but you may get the skewed picture of your algorithm efficiency because you're measuring multiple things at once, and perhaps the ones you may not be interested in or even knew that they even exist or are included in the numbers of your experiment, e.g. how good the memory prefetcher understands the access patterns of your experiment. For the same reason, I think it's better to avoid mixing many different things into the same benchmark run. I stand corrected wrt statistical significance since I am not familiar with Julia. I missed to understand that part.

adgjlsfhk1 2y ago

IPC looks like 3.5 instructions per clock. (and the speed for bigger inputs will be memory bound rather than compute bound).

menaerus 2y ago
If true this would be between ok-ish and decent result but since you didn't say anything about the experiment or how you ran it or anything about the HW it's hard to say more. I assume you didn't use https://github.com/miguelraz/StagedFilters.jl/blob/master/test/benchmarks.jl https://github.com/miguelraz/StagedFilters.jl/blob/master/te... since that benchmark is pretty much flawed for more than a few reasons: - Operates with the dataset that cannot fit even in the L3 cache on most consumer machines: ~76MB (float64) and ~38MB (float32) - Intermixes python with the julia code - Not even a single repetition - no statistical significance - Does not show what happens with different data distributions, e.g. random, but only uses a monotonically increasing sequence
- ChrisRackauckas 2y ago
  Did you link the wrong script? The script you show runs everything to statistical significance using Chairmarks.@b. Also, I don't understand what the issue would be with mixing Python and Julia code in the benchmark script. The Julia side JIT compiles the invocations which we've seen removes pretty much all non-Python overhead and actually makes the resulting SciPy calls faster than doing so from Python itself in many cases, see for example https://github.com/SciML/SciPyDiffEq.jl?tab=readme-ov-file#measuring-overhead https://github.com/SciML/SciPyDiffEq.jl?tab=readme-ov-file#m... where invocations from Julia very handily outperform SciPy+Numba from the Python REPL. Of course, that is a higher order function so it's benefiting from the Julia JIT in other ways as well, but the point is in previous benchmarks we've seen the overhead floor as so low (~100ns IIRC) that it didn't effect benchmarks negatively for Python and actually improved many Python timings in practice. Though it would be good to show in this case what exactly the difference is in order to isolate any potential issue, I would be surprised if it's more than a 100ns overhead for an invocation like this and with 58ms being the benchmark size, that cost is well below the noise floor). Though trying different datasets is of course valid. There's no reason to reject a benchmark just because it doesn't fit into L3 cache, there's many use cases for that. But it does not mean that all use cases are likely to see such a result.
  - gauge_field 2y ago
    [dead]
  - menaerus 2y ago
    If it doesn't fit into the L3 cache then what the benchmark is measuring is not the CPU efficiency of your SIMD approach but also inherently includes different (and complex) effects of L1+L2+L3 cache hierarchy, memory prefetchers and main memory latency. Latency of accessing the RAM is alone ~two to ~three orders of magnitude larger then the latency of executing the single instruction. Per-se this isn't wrong but you may get the skewed picture of your algorithm efficiency because you're measuring multiple things at once, and perhaps the ones you may not be interested in or even knew that they even exist or are included in the numbers of your experiment, e.g. how good the memory prefetcher understands the access patterns of your experiment. For the same reason, I think it's better to avoid mixing many different things into the same benchmark run. I stand corrected wrt statistical significance since I am not familiar with Julia. I missed to understand that part.