6 ms·

While I somewhat agree with you, the real answer is assume nothing and profile. Always. But there's a catch. Things are getting a bit out of hand, there are jus

by vardump 11y ago

While I somewhat agree with you, the real answer is assume nothing and profile. Always. But there's a catch. Things are getting a bit out of hand, there are just too many architectures and configurations. I think modern consumer-oriented code should run at least well on Intel Nehalem - Skylake, AMD K10 - K12, ARM Cortex A7/A8/A9/A15/A53/A57/A72, Qualcomm Krait/Kryo. Focus on 64-bit, but some attention should still be paid to 32-bit performance. Number of actual targets is even worse, because cache and memory architecture vary so much. At least all those have 64-byte cache lines. In high performance code, there's often the choice between memory and compute load balancing. What memory access patterns and layouts you're going to use - where do you want your bottlenecks? You can often reduce bandwidth requirements by computing more and vice versa. Take Sandy Bridge for example: While L1 and L2 cache sizes are fixed on Sandy Bridge, L3 cache varies between 1 - 20 MB. Clock speeds vary between 1.0 to 3.6 GHz. There are 1-4 memory channels, I believe they're usually arranged 64 byte granularity round robin. Say your access pattern is access every second 64-byte cache line vs. access every line. On 1 and 3 channel systems, both perform about same. When skipping 64 byte cache line on 2 channel system, effective memory bandwidth is equal to just 1 channel system. 4 channel system will behave like 2 channel one. The code might be bandwidth bottlenecked on 1-2 channel system and compute bound on 3-4 channel system. Then you're going to hit DRAM page miss every... 1-8 kB? System dependant, of course. To make the best choice for Sandy Bridge based systems, among other things, you need to know number of memory channels + their arrangement, amount of L3 cache and clock speed. On some Sandy Bridge CPUs you have a lot of time to do complicated data packing and unpacking, trying to just keep memory bandwidth usage down. That was just one architecture. Now add all the other platforms in the mix. There are pretty many different instruction set extensions. If you don't use them properly, you can lose an order of magnitude of performance. I think we'll need runtime code generation in the future. On system JIT/AOT also for traditionally compiled languages, such as C and C++.

acqq 11y ago
> the real answer is assume nothing and profile. They did profile, but what? Their "don't trust folklore" ("just 3% speedup") conclusion is valid only for one specific architecture and only for the most recent CPUs. As you say > modern consumer-oriented code should run at least well on Intel Nehalem - Skylake, AMD K10 - K12, ARM Cortex A7/A8/A9/A15/A53/A57/A72, Qualcomm Krait/Kryo. If I understand correctly, they themselves measure 30% speedup even only one generation before on Intel and they didn't measure any ARM or AMD. It looks to me that "don't trust folklore" claim was possible only by ignoring all the processors you name. And you are right, they also didn't show that they are aware of the differences inside of one generation (does the cheapest processor of the same generation behave the same?)
- loup-vaillant 11y ago
  > Their "don't trust folklore" ("just 3% speedup") conclusion is valid only for one specific architecture and only for the most recent CPUs. For now. In 5 years from now, you can be sure most new architectures will have predictors that are as good. So, while it might be good to reach for the low hanging fruit (jump threading), heavier approaches such as super-instruction replication are probably no longer a good idea.
  - acqq 11y ago
    Writing the code in a way not to be dependent on the most advanced hardware solution in existence is better choice unless you target only the specific hardware or the solution is of the "use once then throw away" kind.
    - loup-vaillant 11y ago
      That would be true, if we were speaking of using this optimization instead of that optimisation. We're not. Here, they're saying that in this particular architecture, a number of optimisations are effectively useless. They're saying that naive code is almost as fast. They're saying that some "optimizations" have even become counter-productive. The lesson I get from this? Unless I know a fair bit about the target platforms, the naive approach is better, because any performance model I have in mind might be invalidated anyway.
      - acqq 11y ago
        They actually benchmark the Python interpreter. They show that only on Haswell the gain from using the goto label is low. For me, that's not an argument to never use goto label, especially not to remove it from the Python implementation which certainly should run on many other CPUs than Haswell. Edit: therefore I don't agree with your "unless I know a fair bit about the target platforms, the naive approach is better." I'd agree however with "it the speed (or the battery use) actually doesn't matter, the naive approach is better." Sure.
astrobe_ 11y ago
The fact that the performance of an implementation varies significantly depending on the actual CPU could already be deduced from some of the Ertl papers. Moreover, given that one uses specifically a bytecode interpreter and not a JIT is generally the sign that the interpreter is to be used on multiple platforms. "Optimizing" in this context simply means (almost) nothing. The obvious conclusion is to not care too much about raw performance and focus instead on ease of interfacing with libraries, and to make it easy to add primitives/instructions to your VM. With the idea that if your interpreter is too slow, you just rewrite the critical parts in native code.
- mafribe 11y ago
  given that one uses specifically a bytecode interpreter and not a JIT is generally the sign that the interpreter is to be used on multiple platforms. Well, you could JIT the bytecode. Or you could use a meta-tracer (see e.g. [1]). Both (kind-of / sort-of) circumvent the platform dependency problem. [1] http://tratt.net/laurie/research/pubs/html/bolz_tratt__the_impact_of_metatracing_on_vm_design_and_implementation/ http://tratt.net/laurie/research/pubs/html/bolz_tratt__the_i...
- vardump 11y ago
  > you just rewrite the critical parts in native code. Rewrite critical parts of interpreter in native code? My point was there's no optimal native code anymore. Pure interpreters are falling out of fashion anyways. For JITted systems, there's a significant cost for calling native code. At least until JITs actually inline natively called code, possibly even through dynamic library (.so, .dylib, .dll, etc.) call. We need a lightweight, thin profile guided JIT/AOT engine. No standard library, memory management agnostic. Something that can target different architectures, memory & cache configurations and instruction set extensions that may have been unknown at design phase. A compilation target for some C/C++/Rust/etc. and a runtime that takes care of the architecture details.
  - astrobe_ 11y ago
    > Rewrite critical parts of interpreter in native code? My point was there's no optimal native code anymore. No, I mean rewrite the critical parts of the "interpreted" program as interpreter primitives/instructions: the best way to eliminate interpreter overhead is to merge N primitives into one. But I realise we have a different perspective. You seem to consider interpreter VMs as off the shelf components, when I consider VMs as custom components because I'm working mainly with embedded systems. In the embedded world, JIT is often simply not an option either because they are not available for your particular target, or because the resource constrains don't allow it. But still you sometimes want to trade resources for ease of development and/or flexibility.

by vardump 11y ago

acqq 11y ago

> the real answer is assume nothing and profile. They did profile, but what? Their "don't trust folklore" ("just 3% speedup") conclusion is valid only for one specific architecture and only for the most recent CPUs. As you say > modern consumer-oriented code should run at least well on Intel Nehalem - Skylake, AMD K10 - K12, ARM Cortex A7/A8/A9/A15/A53/A57/A72, Qualcomm Krait/Kryo. If I understand correctly, they themselves measure 30% speedup even only one generation before on Intel and they didn't measure any ARM or AMD. It looks to me that "don't trust folklore" claim was possible only by ignoring all the processors you name. And you are right, they also didn't show that they are aware of the differences inside of one generation (does the cheapest processor of the same generation behave the same?)

loup-vaillant 11y ago
> Their "don't trust folklore" ("just 3% speedup") conclusion is valid only for one specific architecture and only for the most recent CPUs. For now. In 5 years from now, you can be sure most new architectures will have predictors that are as good. So, while it might be good to reach for the low hanging fruit (jump threading), heavier approaches such as super-instruction replication are probably no longer a good idea.
- acqq 11y ago
  Writing the code in a way not to be dependent on the most advanced hardware solution in existence is better choice unless you target only the specific hardware or the solution is of the "use once then throw away" kind.
  - loup-vaillant 11y ago
    That would be true, if we were speaking of using this optimization instead of that optimisation. We're not. Here, they're saying that in this particular architecture, a number of optimisations are effectively useless. They're saying that naive code is almost as fast. They're saying that some "optimizations" have even become counter-productive. The lesson I get from this? Unless I know a fair bit about the target platforms, the naive approach is better, because any performance model I have in mind might be invalidated anyway.
    - acqq 11y ago
      They actually benchmark the Python interpreter. They show that only on Haswell the gain from using the goto label is low. For me, that's not an argument to never use goto label, especially not to remove it from the Python implementation which certainly should run on many other CPUs than Haswell. Edit: therefore I don't agree with your "unless I know a fair bit about the target platforms, the naive approach is better." I'd agree however with "it the speed (or the battery use) actually doesn't matter, the naive approach is better." Sure.

astrobe_ 11y ago

The fact that the performance of an implementation varies significantly depending on the actual CPU could already be deduced from some of the Ertl papers. Moreover, given that one uses specifically a bytecode interpreter and not a JIT is generally the sign that the interpreter is to be used on multiple platforms. "Optimizing" in this context simply means (almost) nothing. The obvious conclusion is to not care too much about raw performance and focus instead on ease of interfacing with libraries, and to make it easy to add primitives/instructions to your VM. With the idea that if your interpreter is too slow, you just rewrite the critical parts in native code.

mafribe 11y ago
given that one uses specifically a bytecode interpreter and not a JIT is generally the sign that the interpreter is to be used on multiple platforms. Well, you could JIT the bytecode. Or you could use a meta-tracer (see e.g. [1]). Both (kind-of / sort-of) circumvent the platform dependency problem. [1] http://tratt.net/laurie/research/pubs/html/bolz_tratt__the_impact_of_metatracing_on_vm_design_and_implementation/ http://tratt.net/laurie/research/pubs/html/bolz_tratt__the_i...
vardump 11y ago
> you just rewrite the critical parts in native code. Rewrite critical parts of interpreter in native code? My point was there's no optimal native code anymore. Pure interpreters are falling out of fashion anyways. For JITted systems, there's a significant cost for calling native code. At least until JITs actually inline natively called code, possibly even through dynamic library (.so, .dylib, .dll, etc.) call. We need a lightweight, thin profile guided JIT/AOT engine. No standard library, memory management agnostic. Something that can target different architectures, memory & cache configurations and instruction set extensions that may have been unknown at design phase. A compilation target for some C/C++/Rust/etc. and a runtime that takes care of the architecture details.
- astrobe_ 11y ago
  > Rewrite critical parts of interpreter in native code? My point was there's no optimal native code anymore. No, I mean rewrite the critical parts of the "interpreted" program as interpreter primitives/instructions: the best way to eliminate interpreter overhead is to merge N primitives into one. But I realise we have a different perspective. You seem to consider interpreter VMs as off the shelf components, when I consider VMs as custom components because I'm working mainly with embedded systems. In the embedded world, JIT is often simply not an option either because they are not available for your particular target, or because the resource constrains don't allow it. But still you sometimes want to trade resources for ease of development and/or flexibility.