5 ms·

sounds intriguing and way more flexible, but the example is for video? i don't know anything about this, but if i had to guess i would say that any runtime code

by iammyIP 10y ago

sounds intriguing and way more flexible, but the example is for video? i don't know anything about this, but if i had to guess i would say that any runtime code generation is worse performance wise compared to a mix of pod structs and function pointers to functions which are all known at compile time.

vardump 10y ago
> runtime code generation is worse performance wise compared to a mix of pod structs and function pointers to functions which are all known at compile time Having "a mix of pod structs and function pointers" is analogous to having an interpreter for some scripting language. Do you consider interpreted scripting languages fast? Interpreters are slow pretty much for that reason. Having to chase pointers. Runtime code generating JITs wipe floor with interpreters, being usually 10-100x faster. Now I'm not saying your application will be 10-100x faster, but done right, it'll absolutely be faster if your performance critical "inner loop" / filter chain needs to currently follow pointers. It's just a question of how much and whether it matters for the use case.
- iammyIP 10y ago
  [Having "a mix of pod structs and function pointers" is analogous to having an interpreter for some scripting language.] Yes, it might be similar at the core, but i would guess the usual audio pod-struct/function mix is many times less complex than a full jit language interpreter. What does chasing pointer mean to slowness then? My understanding is that its similar to data pointers, that a certain region of memory needs to be looked up and be fed into the instruction cache. If the process is layed out so that instruction cache misses are minimised, whats so bad about it?
  - vardump 10y ago
    > whats so bad about it? Other than executing a lot of useless instructions, causing pipeline stalls due to indirection, potentially mispredicted branches, etc. I guess nothing. Just remember if the actual operation is just a few instructions, the price for all that extra work can be 1-2 orders of magnitude. If you only need to chase pointers say a million times per second or less, go for it. It won't matter so much. In worst cases you might lose 10% (of total cpu core capacity) performance, if even that. If it's in tens of millions or more, it's pretty likely to be a significant bottleneck. What does your profiler say? What kind of ILP you're getting?
    - iammyIP 10y ago
      The current code with the function pointers and pods works ok for me, its about in the million per second range and i didn't profile any alternative to it. How would you implement such an alternative with jit code generation into this pod/function c code setup without major rewrites? Seems to me like a completely different system which is bound to llvm (i use gcc)? edit: since you were mentioning jit optimisation and code generation - this is a field i do not have any experience, but my impression so far is that this runtime re-arrangement of code can yield quite a big performance increase for higher level languages (jvm does this afaik). Is the link you provided the same idea for statically compiled c/c++ code?
      - vardump 10y ago
        If the chain of operations change rarely, you might even get away with calling clang or whatever compiler and actually compiling a chain of filter source code mashed together.
        iammyIP 10y ago
        So how would you raw-structure a performance critical (audio) program that needs some flexible path for the data processing, like with a 128 sample buffer and about 3-10 choices along the processing path. Processing path should be switchable per buffer frequency. The function pointers were the best solution i could come up with, but maybe that doesnt really eliminate branching cache misses and stalling and its impact on performance, but just hides it in plain-text.
        vardump 10y ago
        How many permutations? If not too many, maybe just compile them all. Sometimes you do need to use function pointers. It's just if it's in a performance critical path, you might want to consider alternatives.
        iammyIP 10y ago
- LnxPrgr3 10y ago
  > Interpreters are slow pretty much for that reason. Having to chase pointers. Here's a test I did on the Raspberry Pi 3 (with source so anyone can repeat it on more Intel-y hardware): https://gist.github.com/LnxPrgr3/31eaf5648a9956f5c576f70d8768e6e6 https://gist.github.com/LnxPrgr3/31eaf5648a9956f5c576f70d876... Calls through a function pointer are technically more expensive, but it's a factor of ~1.1, not 10-100. Interpreters are slow because they chase pointers and otherwise branch in a manner the CPU can't predict. Calling the same function through the same pointer over and over again isn't that.
  - vardump 10y ago
    Different compilation unit prevents inlining. Function pointers to small functions can be a performance problem for exactly that reason -- except for very specific special cases (when compiler can actually figure out your function pointer calls always end up calling same function), function pointer calls cannot be inlined. C/C++ compiler abstracts the difference between inlining and actual call. For best results, cross compilation unit optimizations should be enabled so that inlining can occur across object and library boundaries. > Interpreters are slow because they chase pointers and otherwise branch in a manner the CPU can't predict. Calling the same function through the same pointer over and over again isn't that. A lot of function pointer usage is also data dependent. Data dependency and too complicated dependency chains are what breaks prediction.
    - LnxPrgr3 10y ago
      Data-dependent audio pipeline callbacks? In the schemes I'm aware of, they rarely change after setup--hence my thinking effectively fixed targets were the representative case. (After an update, pipeline stalls plague you until the predictor catches up.) Breaking inlining is probably the real cost to this. Still, we're talking ~9 cycles per indirect call on this hardware. If you call each callback per buffer instead of per sample (which everyone seems to do), you're down to an amortized cost below 1 cycle per sample for a filter chain 14 deep with only 128 sample buffers. 14 filters probably swamp that cost. Edit: A good compiler will completely remove my trivial function if it's inlined into call.cc's code. It has no side effects, and neither does the loop calling it 100,000,000 times. The resulting timings would be meaningless: doing nothing is cheap!

by iammyIP 10y ago

vardump 10y ago

> runtime code generation is worse performance wise compared to a mix of pod structs and function pointers to functions which are all known at compile time Having "a mix of pod structs and function pointers" is analogous to having an interpreter for some scripting language. Do you consider interpreted scripting languages fast? Interpreters are slow pretty much for that reason. Having to chase pointers. Runtime code generating JITs wipe floor with interpreters, being usually 10-100x faster. Now I'm not saying your application will be 10-100x faster, but done right, it'll absolutely be faster if your performance critical "inner loop" / filter chain needs to currently follow pointers. It's just a question of how much and whether it matters for the use case.

iammyIP 10y ago
[Having "a mix of pod structs and function pointers" is analogous to having an interpreter for some scripting language.] Yes, it might be similar at the core, but i would guess the usual audio pod-struct/function mix is many times less complex than a full jit language interpreter. What does chasing pointer mean to slowness then? My understanding is that its similar to data pointers, that a certain region of memory needs to be looked up and be fed into the instruction cache. If the process is layed out so that instruction cache misses are minimised, whats so bad about it?
- vardump 10y ago
  > whats so bad about it? Other than executing a lot of useless instructions, causing pipeline stalls due to indirection, potentially mispredicted branches, etc. I guess nothing. Just remember if the actual operation is just a few instructions, the price for all that extra work can be 1-2 orders of magnitude. If you only need to chase pointers say a million times per second or less, go for it. It won't matter so much. In worst cases you might lose 10% (of total cpu core capacity) performance, if even that. If it's in tens of millions or more, it's pretty likely to be a significant bottleneck. What does your profiler say? What kind of ILP you're getting?
  - iammyIP 10y ago
    The current code with the function pointers and pods works ok for me, its about in the million per second range and i didn't profile any alternative to it. How would you implement such an alternative with jit code generation into this pod/function c code setup without major rewrites? Seems to me like a completely different system which is bound to llvm (i use gcc)? edit: since you were mentioning jit optimisation and code generation - this is a field i do not have any experience, but my impression so far is that this runtime re-arrangement of code can yield quite a big performance increase for higher level languages (jvm does this afaik). Is the link you provided the same idea for statically compiled c/c++ code?
    - vardump 10y ago
      If the chain of operations change rarely, you might even get away with calling clang or whatever compiler and actually compiling a chain of filter source code mashed together.
      - iammyIP 10y ago
        So how would you raw-structure a performance critical (audio) program that needs some flexible path for the data processing, like with a 128 sample buffer and about 3-10 choices along the processing path. Processing path should be switchable per buffer frequency. The function pointers were the best solution i could come up with, but maybe that doesnt really eliminate branching cache misses and stalling and its impact on performance, but just hides it in plain-text.
        vardump 10y ago
        How many permutations? If not too many, maybe just compile them all. Sometimes you do need to use function pointers. It's just if it's in a performance critical path, you might want to consider alternatives.
        iammyIP 10y ago
LnxPrgr3 10y ago
> Interpreters are slow pretty much for that reason. Having to chase pointers. Here's a test I did on the Raspberry Pi 3 (with source so anyone can repeat it on more Intel-y hardware): https://gist.github.com/LnxPrgr3/31eaf5648a9956f5c576f70d8768e6e6 https://gist.github.com/LnxPrgr3/31eaf5648a9956f5c576f70d876... Calls through a function pointer are technically more expensive, but it's a factor of ~1.1, not 10-100. Interpreters are slow because they chase pointers and otherwise branch in a manner the CPU can't predict. Calling the same function through the same pointer over and over again isn't that.
- vardump 10y ago
  Different compilation unit prevents inlining. Function pointers to small functions can be a performance problem for exactly that reason -- except for very specific special cases (when compiler can actually figure out your function pointer calls always end up calling same function), function pointer calls cannot be inlined. C/C++ compiler abstracts the difference between inlining and actual call. For best results, cross compilation unit optimizations should be enabled so that inlining can occur across object and library boundaries. > Interpreters are slow because they chase pointers and otherwise branch in a manner the CPU can't predict. Calling the same function through the same pointer over and over again isn't that. A lot of function pointer usage is also data dependent. Data dependency and too complicated dependency chains are what breaks prediction.
  - LnxPrgr3 10y ago
    Data-dependent audio pipeline callbacks? In the schemes I'm aware of, they rarely change after setup--hence my thinking effectively fixed targets were the representative case. (After an update, pipeline stalls plague you until the predictor catches up.) Breaking inlining is probably the real cost to this. Still, we're talking ~9 cycles per indirect call on this hardware. If you call each callback per buffer instead of per sample (which everyone seems to do), you're down to an amortized cost below 1 cycle per sample for a filter chain 14 deep with only 128 sample buffers. 14 filters probably swamp that cost. Edit: A good compiler will completely remove my trivial function if it's inlined into call.cc's code. It has no side effects, and neither does the loop calling it 100,000,000 times. The resulting timings would be meaningless: doing nothing is cheap!