5 ms·

Thanks for giving an actual example of such optimizations. In my personal experience my C++ (and Rust) code was often outperformed by the JVM's optimizations so

Thanks for giving an actual example of such optimizations. In my personal experience my C++ (and Rust) code was often outperformed by the JVM's optimizations so I've found it hard to relate to the tradeoffs C++ developers assume are obvious to the rest of us.

grumpyprole 3y ago
+1. Part of the problem is that x86 assembly is hardly programming the metal anymore. The performance characteristics of processors has changed over the years also, compilers can be more up-to-date than humans.
- overgard 3y ago
  x86 assembly doesn't represent the actual opcodes the CPU executes anymore, but it's still the low level "API" we have to the CPU. Even if assembly isn't programming to the metal, it's definitely more to the metal than C, and C is more to the metal than Java, etc. Metalness is a gradient
  - grumpyprole 3y ago
    Lol, I like the word "metalness".
  - SubjectToChange 3y ago
    Metalness is a gradient It would be better to say that "Metalness" is a sort of "Feature Set". IMO, most programmers would tend to agree that C++ is far closer to Java than C is, yet C++ is every bit as low level as C is. Indeed, even a managed language like C# supports raw pointers and even inline assembly if one is willing to get their hands dirty.
    - kaba0 3y ago
      There is expressivity and abstraction-ability which are language features, and there is “levelness” or metalness as is it being referred to. Low and high level languages have two definitions I have seen, one is more objective, but utterly useless: only assemblies are low-level, everything else is high. The other one is more about what can be controlled in a given language’s idiomatic usage, I believe that’s what most people actually refer to intuitively - e.g. C#/Java have ways to manipulate pointers, but you would generally not say that. In that regard, C, C++, Rust are quite close to the metal with the latter two actually being even closer as they have native control over SIMD datatypes as well, while C doesn’t have that. Note, this is only a partial ordering, as e.g. C#/Java also have control over vector instructions, yet I wouldn’t go around claiming them closer to the metal as C. Going back to expressivity, C++ is very expressive, while C is very inexpressive, but that is a completely orthogonal axis.
lbrandy 3y ago
I struggle to resonate with what you are saying, as my experience is the opposite. I'm curious where this discrepancy is rooted. Reckless hypothesis: are you working on majority latency or majority throughput sensitive systems? I have seen so, so, so many examples of systems where latencies, including and especially tail latencies, end up mattering substantially and where java becomes a major liability. In my experience, actually, carefully controlling things like p99 latency is actually the most important reason C++ is preferred rather than the more vaguely specified "performance".
- ianlevesque 3y ago
  The specific example that comes to mind was translating a Java application doing similarity search on a large dataset into fairly naive Rust doing the same. Throughput I guess. It may be possible to optimize Rust to get there but it’s also possible to (and in this case did) end up with less understandable code that runs at 30% the speed. Edit: And probably for that specific example it’d be best to go all the way to some optimized library like FAISS, so maybe C++ still wins?
- grumpyprole 3y ago
  I've seen C++ systems that are considerably slower than equivalent Java systems, despite the lack of stack allocation and boxing in Java. It's mostly throughput, malloc is slow, the C++ smart pointers cause memory barriers and the heap fragments. Memory management for complex applications is hard and the GC often gives better results.
  - vvanders 3y ago
    I've seen so may flat profiles due to shared_ptr. Rust has done a lot of things right but one thing it really did well was putting a decent amount of friction into std::sync::Arc<T>(and offering std::rc::Rc<T> when you don't want atomics!) vs &mut T or even Box<T>. Everyone reaches for shared_ptr when 99% of the time unique_ptr is the correct option.
    - memefrog 3y ago
overgard 3y ago
I think the reason to use C/C++ over java has less to do with the various optimizations and more to do with control over memory layout (and thus at least indirect control over cache usage and so on). Plus you remove a lot of "noise" in terms of performance (GC hiccups, JIT translation, VM load time, etc.).
kllrnohj 3y ago
It would be fascinating if you could give any such comparison examples. The only time I've seen JVM come anywhere close to C++ in normal usage is if the C++ code was written like it was Java - that is, lots of shared_ptr heap allocations for nearly everything. Or perhaps you're one of the rare few that write Java code like it was C instead? You can definitely get something fast like that, but it seems all too rare.
deterministic 3y ago
There is no universe where well written and optimised C++ code will be out-performed by Java code. You might get similar performance in Java if you create zero objects (except during initialisation) and then manually manage the usage of objects at runtime. However only if the matching C++ code is written naively.

grumpyprole 3y ago

+1. Part of the problem is that x86 assembly is hardly programming the metal anymore. The performance characteristics of processors has changed over the years also, compilers can be more up-to-date than humans.

overgard 3y ago
x86 assembly doesn't represent the actual opcodes the CPU executes anymore, but it's still the low level "API" we have to the CPU. Even if assembly isn't programming to the metal, it's definitely more to the metal than C, and C is more to the metal than Java, etc. Metalness is a gradient
- grumpyprole 3y ago
  Lol, I like the word "metalness".
- SubjectToChange 3y ago
  Metalness is a gradient It would be better to say that "Metalness" is a sort of "Feature Set". IMO, most programmers would tend to agree that C++ is far closer to Java than C is, yet C++ is every bit as low level as C is. Indeed, even a managed language like C# supports raw pointers and even inline assembly if one is willing to get their hands dirty.
  - kaba0 3y ago
    There is expressivity and abstraction-ability which are language features, and there is “levelness” or metalness as is it being referred to. Low and high level languages have two definitions I have seen, one is more objective, but utterly useless: only assemblies are low-level, everything else is high. The other one is more about what can be controlled in a given language’s idiomatic usage, I believe that’s what most people actually refer to intuitively - e.g. C#/Java have ways to manipulate pointers, but you would generally not say that. In that regard, C, C++, Rust are quite close to the metal with the latter two actually being even closer as they have native control over SIMD datatypes as well, while C doesn’t have that. Note, this is only a partial ordering, as e.g. C#/Java also have control over vector instructions, yet I wouldn’t go around claiming them closer to the metal as C. Going back to expressivity, C++ is very expressive, while C is very inexpressive, but that is a completely orthogonal axis.

lbrandy 3y ago

I struggle to resonate with what you are saying, as my experience is the opposite. I'm curious where this discrepancy is rooted. Reckless hypothesis: are you working on majority latency or majority throughput sensitive systems? I have seen so, so, so many examples of systems where latencies, including and especially tail latencies, end up mattering substantially and where java becomes a major liability. In my experience, actually, carefully controlling things like p99 latency is actually the most important reason C++ is preferred rather than the more vaguely specified "performance".

ianlevesque 3y ago
The specific example that comes to mind was translating a Java application doing similarity search on a large dataset into fairly naive Rust doing the same. Throughput I guess. It may be possible to optimize Rust to get there but it’s also possible to (and in this case did) end up with less understandable code that runs at 30% the speed. Edit: And probably for that specific example it’d be best to go all the way to some optimized library like FAISS, so maybe C++ still wins?
grumpyprole 3y ago
I've seen C++ systems that are considerably slower than equivalent Java systems, despite the lack of stack allocation and boxing in Java. It's mostly throughput, malloc is slow, the C++ smart pointers cause memory barriers and the heap fragments. Memory management for complex applications is hard and the GC often gives better results.
- vvanders 3y ago
  I've seen so may flat profiles due to shared_ptr. Rust has done a lot of things right but one thing it really did well was putting a decent amount of friction into std::sync::Arc<T>(and offering std::rc::Rc<T> when you don't want atomics!) vs &mut T or even Box<T>. Everyone reaches for shared_ptr when 99% of the time unique_ptr is the correct option.
  - memefrog 3y ago

overgard 3y ago

I think the reason to use C/C++ over java has less to do with the various optimizations and more to do with control over memory layout (and thus at least indirect control over cache usage and so on). Plus you remove a lot of "noise" in terms of performance (GC hiccups, JIT translation, VM load time, etc.).

kllrnohj 3y ago

It would be fascinating if you could give any such comparison examples. The only time I've seen JVM come anywhere close to C++ in normal usage is if the C++ code was written like it was Java - that is, lots of shared_ptr heap allocations for nearly everything. Or perhaps you're one of the rare few that write Java code like it was C instead? You can definitely get something fast like that, but it seems all too rare.

deterministic 3y ago

There is no universe where well written and optimised C++ code will be out-performed by Java code. You might get similar performance in Java if you create zero objects (except during initialisation) and then manually manage the usage of objects at runtime. However only if the matching C++ code is written naively.