5 ms·

surprised to see a garbage collected language project (C# for Garnet) beat redis/dragonfly

latchkey 3y ago
I think it is rare, but isn't impossible. I remember in the 2000's watching someone do a demo of Java/JVM beating the pants off a C++ application. If I remember correctly, it was something about how the JIT was doing an optimization that you would have had to write assembly in order to optimize it to the same level.
- nurettin 3y ago
  There were coding competition benchmarks at the time putting javac against the underdeveloped gcc 2.95 of the time. The trick they did with java programs was to allocate a region of memory only once and reuse that whenever they needed more, simulating a stack. Then the programs were benchmarked as hot start and cold start. Hot start timings were used to bench against C++ programs. If the algorithm they used was better, this sometimes resulted in head to head or better performance.
  - latchkey 3y ago
    I remember that one too, but this one was not just the hot memory trick, which seems relatively easy. It was entirely JIT based, where the JIT happened to pick a better path than gcc.
- thayne 3y ago
  There are at least two reasons I can think of why a JIT language with GC can outperform C/c++: 1. Memory management with a GC often has higher throughput. With the downside that you can have high latency when a garbage collection occurs. 2. The JIT compilation can potentially do a better job of optimizing, because it has information about how the code has been run so far. It is possible for c or c++ (or rust) to get get the first using alternative memory management strategies, and the second by hand optimizing, and/or using profile guided optimization.
  - pjmlp 3y ago
    3. The runtime also supports C++, and one emits the same MSIL code sequence that the C++ compiler would generate. Which is why WASM isn't that great novelty for grey beards.
  - monocasa 3y ago
    I've also seen GC reliant concurrent algorithms that would require a lot more synchronization to keep track of when to actually free an object. GCed languages can just forget about the object and rely on the GC later to figure out what threads have access to valid objects. C and C++ need to figure out if it's safe to free.
blackoil 3y ago
The investment in optimization in CLR or JVM can be huge as they impact millions of applications. While each C / C++ code will have to be hand optimized. Also limits on number of people in given time who can write optimal C code vs C# will also make managed code better.
- rfoo 3y ago
  Plus that it's easier to have unmanaged code in C# than in Java.
  - larodi 3y ago
    Is it possible to have unmanaged code in Java at all… or you mean linking libs and exposing them through Java API in the lang?
    - rfoo 3y ago
      Ah, I meant Java code manipulating off-heap memory. For Java I don't think there's a way to mix managed and unmanaged code like you do in C#/C++ CLR.
      - neonsunset 3y ago
        In C# you usually don't want to mix in C++. Manipulating off-heap memory is indeed easy - nowadays pointers to both object interiors, stack and unmanaged memory can be represented as `ref T` where T is byte, int, etc. These are then subsequently wrapped by `Span<T>` and `ReadOnlySpan<T>` respectively. This way a span can be a slice of memory that can have any origin: var fromStack1 = (stackalloc byte[32]); var fromStack2 = (Span<byte>)[0x20, 0x20, 0x20, 0x20]; var fromHeap = new byte[32].AsSpan(); unsafe { var ptr = NativeMemory.Alloc(32); var fromMalloc = new Span<byte>(ptr, 32); // Don't forget to free :) }
      - pjmlp 3y ago
        Panama has helped into that direction, it isn't like C# though, for feature parity we will eventually need Valhalla, which since they want to keep existing JARs working, it is taking its time. Ignoring the type systems from Eiffel, Objective-C, Oberon and Modula-3 linage, even though they were inspirations for Java, has been shown to have been a bad decision in hindsight.
fulafel 3y ago
Seems it wins by using more sophisticated data structures & algorithms, instead of "tight code".
pjmlp 3y ago
Not all garbage collected languages are made equal, some of them, like C# and .NET, do provide all the performace knobs that are needed for C++ like coding. People only have to learn how to use them, instead of placing all garbage collected languages into the same basket. In this specific case, MSIL and .NET were designed to support C++ as well, and languages like C# and F# do have ways to access those features, and even if some feature isn't exposed at the language grammar level, you can emit the same MSIL that C++/CLI would generate.
orthoxerox 3y ago
There's a series of blog posts by Oren Eini that shows that C# can be sufficiently fast even when you don't really optimize anything. Beating Redis in every benchmark is a whole another level, of course. https://ayende.com/blog/197412-B/high-performance-net-building-a-redis-clone-naively https://ayende.com/blog/197412-B/high-performance-net-buildi...
Thaxll 3y ago
Because it's not the language that matters here, the performance comes from the architecture of the storage among other things. I've seen Go code best Rust one, really when comparing languages you should look at the same implementation, if the design is completely different its not really comparable.
- surajrmal 3y ago
  While true, gc tends to affect tail latencies as it can increase variance. Obviously not always the case, but without sufficient planning it's likely. Languages with less stop the world problems tend to trade it for throughput, however if the application is not CPU bound it doesn't matter. The same can happen in a non gc language where a similar lack of allocation planning is done, but it tends to be considered more carefully because those languages really put allocations in your face.
astral303 3y ago
Manual memory management vs garbage collection trades speed of allocation for speed of freeing memory. Freeing memory is expensive in a garbage collected language, but allocation is basically free. Modern garbage collectors are very good and handle many common use cases rather efficiently, with minimal or zero pauses, such as freeing short-lived objects. Many GC operations are done in parallel threads without stopping the application (you just pay the CPU overhead cost). Also JITs in both the CLR and the JVM perform optimizations such as escape analysis, which stack-allocate objects that never escape a function’s scope. These objects thus do not have to be GC’d. So really with a GC’d language, you mostly have to worry about pauses and GC CPU overhead. Most GCs can be tuned for a predictable workload. (A bigger challenge for a GC is a variable workload.)
- neonsunset 3y ago
  Correction: JVM implementations perform escape analysis, in particular, because Java does not have structs. .NET does not perform escape analysis for objects and all attempts to experiment with it so far has shown greater impact on compilation speed with little to no profit in allocation rate. However, you will find average allocation traffic much lower in .NET because C# already puts way more data on stack through structs, non-boxing operations (where Java boxes), stack allocated buffers, non-copying slicing with spans and object pooling (which is common in Java too).
  - astral303 3y ago
    Thank you, I missed the stack allocation design doc stating it’s on the roadmap. (https://github.com/dotnet/runtime/blob/main/docs/design/coreclr/jit/object-stack-allocation.md https://github.com/dotnet/runtime/blob/main/docs/design/core...) Appreciate the detail about the stack allocated bits in .NET.
    - neonsunset 3y ago
      Yeah, it kind of is. There are quite a few of experiments that are conducted to see if they show promise in the prototype form and then are taken further for proper integration if they do. Unfortunately, object stack allocation was not one of them even though DOTNET_JitObjectStackAllocation configuration knob exists today, enabling it makes zero impact as it almost never kicks in. By the end of the experiment[0], it was concluded that before investing effort in this kind of feature becomes profitable given how a lot of C# code is written, there are many other lower hanging fruits. To contrast this, in continuation to green threads experiment, a runtime handled tasks experiment[1] which moves async state machine handling from IL emitted by Roslyn to special-cased methods and then handling purely in runtime code has been a massive success and is now being worked on to be integrated in one of the future version of .NET (hopefully 10?) [0] https://github.com/dotnet/runtime/issues/11192 https://github.com/dotnet/runtime/issues/11192 [1] https://github.com/dotnet/runtimelab/blob/feature/async2-experiment/docs/design/features/runtime-handled-tasks.md https://github.com/dotnet/runtimelab/blob/feature/async2-exp...

I think it is rare, but isn't impossible. I remember in the 2000's watching someone do a demo of Java/JVM beating the pants off a C++ application. If I remember correctly, it was something about how the JIT was doing an optimization that you would have had to write assembly in order to optimize it to the same level.

nurettin 3y ago
There were coding competition benchmarks at the time putting javac against the underdeveloped gcc 2.95 of the time. The trick they did with java programs was to allocate a region of memory only once and reuse that whenever they needed more, simulating a stack. Then the programs were benchmarked as hot start and cold start. Hot start timings were used to bench against C++ programs. If the algorithm they used was better, this sometimes resulted in head to head or better performance.
- latchkey 3y ago
  I remember that one too, but this one was not just the hot memory trick, which seems relatively easy. It was entirely JIT based, where the JIT happened to pick a better path than gcc.
thayne 3y ago
There are at least two reasons I can think of why a JIT language with GC can outperform C/c++: 1. Memory management with a GC often has higher throughput. With the downside that you can have high latency when a garbage collection occurs. 2. The JIT compilation can potentially do a better job of optimizing, because it has information about how the code has been run so far. It is possible for c or c++ (or rust) to get get the first using alternative memory management strategies, and the second by hand optimizing, and/or using profile guided optimization.
- pjmlp 3y ago
  3. The runtime also supports C++, and one emits the same MSIL code sequence that the C++ compiler would generate. Which is why WASM isn't that great novelty for grey beards.
- monocasa 3y ago
  I've also seen GC reliant concurrent algorithms that would require a lot more synchronization to keep track of when to actually free an object. GCed languages can just forget about the object and rely on the GC later to figure out what threads have access to valid objects. C and C++ need to figure out if it's safe to free.

blackoil 3y ago

The investment in optimization in CLR or JVM can be huge as they impact millions of applications. While each C / C++ code will have to be hand optimized. Also limits on number of people in given time who can write optimal C code vs C# will also make managed code better.

rfoo 3y ago
Plus that it's easier to have unmanaged code in C# than in Java.
- larodi 3y ago
  Is it possible to have unmanaged code in Java at all… or you mean linking libs and exposing them through Java API in the lang?
  - rfoo 3y ago
    Ah, I meant Java code manipulating off-heap memory. For Java I don't think there's a way to mix managed and unmanaged code like you do in C#/C++ CLR.
    - neonsunset 3y ago
      In C# you usually don't want to mix in C++. Manipulating off-heap memory is indeed easy - nowadays pointers to both object interiors, stack and unmanaged memory can be represented as `ref T` where T is byte, int, etc. These are then subsequently wrapped by `Span<T>` and `ReadOnlySpan<T>` respectively. This way a span can be a slice of memory that can have any origin: var fromStack1 = (stackalloc byte[32]); var fromStack2 = (Span<byte>)[0x20, 0x20, 0x20, 0x20]; var fromHeap = new byte[32].AsSpan(); unsafe { var ptr = NativeMemory.Alloc(32); var fromMalloc = new Span<byte>(ptr, 32); // Don't forget to free :) }
    - pjmlp 3y ago
      Panama has helped into that direction, it isn't like C# though, for feature parity we will eventually need Valhalla, which since they want to keep existing JARs working, it is taking its time. Ignoring the type systems from Eiffel, Objective-C, Oberon and Modula-3 linage, even though they were inspirations for Java, has been shown to have been a bad decision in hindsight.

fulafel 3y ago

Seems it wins by using more sophisticated data structures & algorithms, instead of "tight code".

pjmlp 3y ago

Not all garbage collected languages are made equal, some of them, like C# and .NET, do provide all the performace knobs that are needed for C++ like coding. People only have to learn how to use them, instead of placing all garbage collected languages into the same basket. In this specific case, MSIL and .NET were designed to support C++ as well, and languages like C# and F# do have ways to access those features, and even if some feature isn't exposed at the language grammar level, you can emit the same MSIL that C++/CLI would generate.

orthoxerox 3y ago

There's a series of blog posts by Oren Eini that shows that C# can be sufficiently fast even when you don't really optimize anything. Beating Redis in every benchmark is a whole another level, of course. https://ayende.com/blog/197412-B/high-performance-net-building-a-redis-clone-naively https://ayende.com/blog/197412-B/high-performance-net-buildi...

Thaxll 3y ago

Because it's not the language that matters here, the performance comes from the architecture of the storage among other things. I've seen Go code best Rust one, really when comparing languages you should look at the same implementation, if the design is completely different its not really comparable.

surajrmal 3y ago
While true, gc tends to affect tail latencies as it can increase variance. Obviously not always the case, but without sufficient planning it's likely. Languages with less stop the world problems tend to trade it for throughput, however if the application is not CPU bound it doesn't matter. The same can happen in a non gc language where a similar lack of allocation planning is done, but it tends to be considered more carefully because those languages really put allocations in your face.

astral303 3y ago

Manual memory management vs garbage collection trades speed of allocation for speed of freeing memory. Freeing memory is expensive in a garbage collected language, but allocation is basically free. Modern garbage collectors are very good and handle many common use cases rather efficiently, with minimal or zero pauses, such as freeing short-lived objects. Many GC operations are done in parallel threads without stopping the application (you just pay the CPU overhead cost). Also JITs in both the CLR and the JVM perform optimizations such as escape analysis, which stack-allocate objects that never escape a function’s scope. These objects thus do not have to be GC’d. So really with a GC’d language, you mostly have to worry about pauses and GC CPU overhead. Most GCs can be tuned for a predictable workload. (A bigger challenge for a GC is a variable workload.)

neonsunset 3y ago
Correction: JVM implementations perform escape analysis, in particular, because Java does not have structs. .NET does not perform escape analysis for objects and all attempts to experiment with it so far has shown greater impact on compilation speed with little to no profit in allocation rate. However, you will find average allocation traffic much lower in .NET because C# already puts way more data on stack through structs, non-boxing operations (where Java boxes), stack allocated buffers, non-copying slicing with spans and object pooling (which is common in Java too).
- astral303 3y ago
  Thank you, I missed the stack allocation design doc stating it’s on the roadmap. (https://github.com/dotnet/runtime/blob/main/docs/design/coreclr/jit/object-stack-allocation.md https://github.com/dotnet/runtime/blob/main/docs/design/core...) Appreciate the detail about the stack allocated bits in .NET.
  - neonsunset 3y ago
    Yeah, it kind of is. There are quite a few of experiments that are conducted to see if they show promise in the prototype form and then are taken further for proper integration if they do. Unfortunately, object stack allocation was not one of them even though DOTNET_JitObjectStackAllocation configuration knob exists today, enabling it makes zero impact as it almost never kicks in. By the end of the experiment[0], it was concluded that before investing effort in this kind of feature becomes profitable given how a lot of C# code is written, there are many other lower hanging fruits. To contrast this, in continuation to green threads experiment, a runtime handled tasks experiment[1] which moves async state machine handling from IL emitted by Roslyn to special-cased methods and then handling purely in runtime code has been a massive success and is now being worked on to be integrated in one of the future version of .NET (hopefully 10?) [0] https://github.com/dotnet/runtime/issues/11192 https://github.com/dotnet/runtime/issues/11192 [1] https://github.com/dotnet/runtimelab/blob/feature/async2-experiment/docs/design/features/runtime-handled-tasks.md https://github.com/dotnet/runtimelab/blob/feature/async2-exp...