6 ms·

From the benchmark performance charts(https://microsoft.github.io/garnet/docs/benchmarking/results-resp-bench https://microsoft.github.io/garnet/docs/benchmarki

by west0n 3y ago

From the benchmark performance charts(https://microsoft.github.io/garnet/docs/benchmarking/results-resp-bench https://microsoft.github.io/garnet/docs/benchmarking/results...), the throughput of the GET command exceeds that of Dragonfly by more than tenfold. While 50% latency is slightly higher than Dragonfly, the 99th percentile is slightly lower than Dragonfly. Both the throughput and latency of Garnet and Dragonfly are far better than Redis, indicating that Redis may require a significant performance optimization.

west0n 3y ago
What surprises me the most is that this project is developed in C#, while Dragonfly is developed in C++, and Redis is in C.
- amir734jj 3y ago
  If you look at the store code you see a lot of "unsafe" C# code (i.e pointer manipulation)
  - zigzag312 3y ago
    Shows how flexible C# is. You can trade high-level expressiveness for low-level control where needed and you don't have to deal with any FFI to do it.
    - CharlieDigital 3y ago
      C# and .NET are both truly underrated in the wider community -- I think -- because of some early snafus. The late adoption of an OSS model, Windows-centric in the .NET Framework days, too dependent on Visual Studio for a very long time. Nowadays, I think it's probably the most natural language and platform for teams that need to move on from TypeScript rather than Go or Rust given the similar constructs and idioms.
      - mdaniel 3y ago
        For a while there I had high hopes that one could pull in Java libraries via their JVM interop on top of the Common Language Runtime, but it doesn't seem to have caught on and I'm not in that ecosystem enough to know why. But yes, for the many excellent reasons you cited it means the library ecosystem is nowhere near that of the JVM, and thus I haven't once considered .net for a new project regardless of how much I love C# the language. Well, that, and my experience with the observability and introspection components of the JVM are second to none
        CharlieDigital 3y ago
        > the library ecosystem I don't really see many gaps. There tends to be fewer libraries, but the libraries available generally feel more complete and well thought out because users tend to cluster around the known libraries. Many of the first party libraries are really, really good. EF Core is a prime example of possibly one of the best ORMs on the market right now in terms of productivity, ergonomics, and performance.
        mdaniel 3y ago
        > I don't really see many gaps isn't that the "works on my machine" of this discussion? What's the Apache Tika for .net then? I don't mean, pdf parsing, I don't mean .docx parsing, I mean a framework for interacting with all their supported types <https://tika.apache.org/2.9.1/formats.html https://tika.apache.org/2.9.1/formats.html> with one surface area?
        CharlieDigital 3y ago
        https://github.com/KevM/tikaondotnet https://github.com/KevM/tikaondotnet ?
        mdaniel 3y ago
        > https://github.com/KevM/tikaondotnet/releases/tag/v1.17.1 https://github.com/KevM/tikaondotnet/releases/tag/v1.17.1 - Apr 3, 2018 You're right, how silly of me, I'll install some rando's 6 year old build of it right away. But in seriousness, that readme did remind me of the thing I was thinking of: http://www.ikvm.net/userguide/ikvmc.html http://www.ikvm.net/userguide/ikvmc.html
- declaredapple 3y ago
  I assume it's similar to the 1 billion row challenge with java - C# and Java have sufficiently advanced VM's that they can compete with C/CPP performance - assuming you go out of your way to optimize for it. https://github.com/gunnarmorling/1brc https://github.com/gunnarmorling/1brc
  - neonsunset 3y ago
    Comparison between languages: https://hotforknowledge.com/2024/01/13/1brc-in-dotnet-among-fastest-on-linux-my-optimization-journey/#results https://hotforknowledge.com/2024/01/13/1brc-in-dotnet-among-...
whimsicalism 3y ago
surprised to see a garbage collected language project (C# for Garnet) beat redis/dragonfly
- latchkey 3y ago
  I think it is rare, but isn't impossible. I remember in the 2000's watching someone do a demo of Java/JVM beating the pants off a C++ application. If I remember correctly, it was something about how the JIT was doing an optimization that you would have had to write assembly in order to optimize it to the same level.
  - nurettin 3y ago
    There were coding competition benchmarks at the time putting javac against the underdeveloped gcc 2.95 of the time. The trick they did with java programs was to allocate a region of memory only once and reuse that whenever they needed more, simulating a stack. Then the programs were benchmarked as hot start and cold start. Hot start timings were used to bench against C++ programs. If the algorithm they used was better, this sometimes resulted in head to head or better performance.
    - latchkey 3y ago
      I remember that one too, but this one was not just the hot memory trick, which seems relatively easy. It was entirely JIT based, where the JIT happened to pick a better path than gcc.
  - thayne 3y ago
    There are at least two reasons I can think of why a JIT language with GC can outperform C/c++: 1. Memory management with a GC often has higher throughput. With the downside that you can have high latency when a garbage collection occurs. 2. The JIT compilation can potentially do a better job of optimizing, because it has information about how the code has been run so far. It is possible for c or c++ (or rust) to get get the first using alternative memory management strategies, and the second by hand optimizing, and/or using profile guided optimization.
hipadev23 3y ago
> Redis may require a significant performance optimization. Redis is single-threaded, it’s simple and effective. I’m not sure it needs optimization, and we have 3 alternatives here. Garnet however is the first alternative to actually outperform Redis at both low and high levels of concurrency, which is remarkable. I can’t wait to try it out.
- gigatexal 3y ago
  Redis for 99% of the intended use-cases and companies will be just fine. It has always been rock solid when we used it. It was the opposite of DNS: it was never a problem.
  - moritonal 3y ago
    Could you expand on "It was the opposite of DNS: it was never a problem."? Feel like I'm missing some interesting history here.
    - klohto 3y ago
      isitdns.com
      - takeda 3y ago
        This is lame, it should say No, as you need DNS to load that page :P
    - akvadrako 3y ago
      It's weird since DNS is one of the most rock solid systems we have, with redundancy at every level. 1. Clients query a list of servers (IPs) and handle failover when you don't quickly get a reply. 2. Most of those servers at the root and TLD level are actually anycasted from multiple locations globally, so you connect to the closest instance. 3. Those instances are often clusters of physical servers. The big ones have fully redundant networking, so any router or switch failing doesn't take it down. Some run different DNS server software on each physical server, so even software bugs won't take down the whole system.

by west0n 3y ago

west0n 3y ago

What surprises me the most is that this project is developed in C#, while Dragonfly is developed in C++, and Redis is in C.

amir734jj 3y ago
If you look at the store code you see a lot of "unsafe" C# code (i.e pointer manipulation)
- zigzag312 3y ago
  Shows how flexible C# is. You can trade high-level expressiveness for low-level control where needed and you don't have to deal with any FFI to do it.
  - CharlieDigital 3y ago
    C# and .NET are both truly underrated in the wider community -- I think -- because of some early snafus. The late adoption of an OSS model, Windows-centric in the .NET Framework days, too dependent on Visual Studio for a very long time. Nowadays, I think it's probably the most natural language and platform for teams that need to move on from TypeScript rather than Go or Rust given the similar constructs and idioms.
    - mdaniel 3y ago
      For a while there I had high hopes that one could pull in Java libraries via their JVM interop on top of the Common Language Runtime, but it doesn't seem to have caught on and I'm not in that ecosystem enough to know why. But yes, for the many excellent reasons you cited it means the library ecosystem is nowhere near that of the JVM, and thus I haven't once considered .net for a new project regardless of how much I love C# the language. Well, that, and my experience with the observability and introspection components of the JVM are second to none
      - CharlieDigital 3y ago
        > the library ecosystem I don't really see many gaps. There tends to be fewer libraries, but the libraries available generally feel more complete and well thought out because users tend to cluster around the known libraries. Many of the first party libraries are really, really good. EF Core is a prime example of possibly one of the best ORMs on the market right now in terms of productivity, ergonomics, and performance.
        mdaniel 3y ago
        > I don't really see many gaps isn't that the "works on my machine" of this discussion? What's the Apache Tika for .net then? I don't mean, pdf parsing, I don't mean .docx parsing, I mean a framework for interacting with all their supported types <https://tika.apache.org/2.9.1/formats.html https://tika.apache.org/2.9.1/formats.html> with one surface area?
        CharlieDigital 3y ago
        https://github.com/KevM/tikaondotnet https://github.com/KevM/tikaondotnet ?
        mdaniel 3y ago
        > https://github.com/KevM/tikaondotnet/releases/tag/v1.17.1 https://github.com/KevM/tikaondotnet/releases/tag/v1.17.1 - Apr 3, 2018 You're right, how silly of me, I'll install some rando's 6 year old build of it right away. But in seriousness, that readme did remind me of the thing I was thinking of: http://www.ikvm.net/userguide/ikvmc.html http://www.ikvm.net/userguide/ikvmc.html
declaredapple 3y ago
I assume it's similar to the 1 billion row challenge with java - C# and Java have sufficiently advanced VM's that they can compete with C/CPP performance - assuming you go out of your way to optimize for it. https://github.com/gunnarmorling/1brc https://github.com/gunnarmorling/1brc
- neonsunset 3y ago
  Comparison between languages: https://hotforknowledge.com/2024/01/13/1brc-in-dotnet-among-fastest-on-linux-my-optimization-journey/#results https://hotforknowledge.com/2024/01/13/1brc-in-dotnet-among-...

whimsicalism 3y ago

surprised to see a garbage collected language project (C# for Garnet) beat redis/dragonfly

latchkey 3y ago
I think it is rare, but isn't impossible. I remember in the 2000's watching someone do a demo of Java/JVM beating the pants off a C++ application. If I remember correctly, it was something about how the JIT was doing an optimization that you would have had to write assembly in order to optimize it to the same level.
- nurettin 3y ago
  There were coding competition benchmarks at the time putting javac against the underdeveloped gcc 2.95 of the time. The trick they did with java programs was to allocate a region of memory only once and reuse that whenever they needed more, simulating a stack. Then the programs were benchmarked as hot start and cold start. Hot start timings were used to bench against C++ programs. If the algorithm they used was better, this sometimes resulted in head to head or better performance.
  - latchkey 3y ago
    I remember that one too, but this one was not just the hot memory trick, which seems relatively easy. It was entirely JIT based, where the JIT happened to pick a better path than gcc.
- thayne 3y ago
  There are at least two reasons I can think of why a JIT language with GC can outperform C/c++: 1. Memory management with a GC often has higher throughput. With the downside that you can have high latency when a garbage collection occurs. 2. The JIT compilation can potentially do a better job of optimizing, because it has information about how the code has been run so far. It is possible for c or c++ (or rust) to get get the first using alternative memory management strategies, and the second by hand optimizing, and/or using profile guided optimization.

hipadev23 3y ago

> Redis may require a significant performance optimization. Redis is single-threaded, it’s simple and effective. I’m not sure it needs optimization, and we have 3 alternatives here. Garnet however is the first alternative to actually outperform Redis at both low and high levels of concurrency, which is remarkable. I can’t wait to try it out.

gigatexal 3y ago
Redis for 99% of the intended use-cases and companies will be just fine. It has always been rock solid when we used it. It was the opposite of DNS: it was never a problem.
- moritonal 3y ago
  Could you expand on "It was the opposite of DNS: it was never a problem."? Feel like I'm missing some interesting history here.
  - klohto 3y ago
    isitdns.com
    - takeda 3y ago
      This is lame, it should say No, as you need DNS to load that page :P
  - akvadrako 3y ago
    It's weird since DNS is one of the most rock solid systems we have, with redundancy at every level. 1. Clients query a list of servers (IPs) and handle failover when you don't quickly get a reply. 2. Most of those servers at the root and TLD level are actually anycasted from multiple locations globally, so you connect to the closest instance. 3. Those instances are often clusters of physical servers. The big ones have fully redundant networking, so any router or switch failing doesn't take it down. Some run different DNS server software on each physical server, so even software bugs won't take down the whole system.