15 ms·

Garnet – A new remote cache-store from Microsoft Research

370 points by saganus 3y ago

lxe 3y ago
A drop-in redis replacement with rather impressive latency and throughput bench figures. Wonder what its like to operate in a non-azure stack in the real world.
- nurettin 3y ago
  Are you sure it is drop-in? I don't see any indication of xstream support.
  - StepWeiwu 3y ago
    The title does say that it can work with existing redis clients but it's unclear if they mean full compatibility.
    - vanhanenjjv 3y ago
      https://microsoft.github.io/garnet/docs/commands/api-compatibility https://microsoft.github.io/garnet/docs/commands/api-compati...
giancarlostoro 3y ago
Definitely impressive, Microsoft Research comes out with some impressive projects from time to time, must be fun getting paid to do R&D. I wish big companies did more R&D style projects that benefit the industry in general. I sure hope a good company takes over Hashicorp if they're on the market to be bought.
west0n 3y ago
From the benchmark performance charts(https://microsoft.github.io/garnet/docs/benchmarking/results-resp-bench https://microsoft.github.io/garnet/docs/benchmarking/results...), the throughput of the GET command exceeds that of Dragonfly by more than tenfold. While 50% latency is slightly higher than Dragonfly, the 99th percentile is slightly lower than Dragonfly. Both the throughput and latency of Garnet and Dragonfly are far better than Redis, indicating that Redis may require a significant performance optimization.
- west0n 3y ago
  What surprises me the most is that this project is developed in C#, while Dragonfly is developed in C++, and Redis is in C.
  - amir734jj 3y ago
    If you look at the store code you see a lot of "unsafe" C# code (i.e pointer manipulation)
    - zigzag312 3y ago
      Shows how flexible C# is. You can trade high-level expressiveness for low-level control where needed and you don't have to deal with any FFI to do it.
      - CharlieDigital 3y ago
        C# and .NET are both truly underrated in the wider community -- I think -- because of some early snafus. The late adoption of an OSS model, Windows-centric in the .NET Framework days, too dependent on Visual Studio for a very long time. Nowadays, I think it's probably the most natural language and platform for teams that need to move on from TypeScript rather than Go or Rust given the similar constructs and idioms.
        mdaniel 3y ago
        For a while there I had high hopes that one could pull in Java libraries via their JVM interop on top of the Common Language Runtime, but it doesn't seem to have caught on and I'm not in that ecosystem enough to know why. But yes, for the many excellent reasons you cited it means the library ecosystem is nowhere near that of the JVM, and thus I haven't once considered .net for a new project regardless of how much I love C# the language. Well, that, and my experience with the observability and introspection components of the JVM are second to none
wokwokwok 3y ago
After the aborted/abandoned attempt to port redis to windows (0) this feels like a second-try at the same thing, but first party. Of course, as a research project it doesn't have the same stability / support, etc. as redis, but I could easily imagine this rolling into a real product if it's popular. ...and as an MIT license, if nothing else, the code is a fun read. :) [0] - https://github.com/microsoftarchive/redis?tab=readme-ov-file#redis-on-windows https://github.com/microsoftarchive/redis?tab=readme-ov-file...
- mordae 3y ago
  MIT license, but with CLA. It's bizzare. What more rights could they possibly want on top of MIT to warrant CLA?
  - orthoxerox 3y ago
    It's probably a blanket requirement by the MSFT legal team so they can just release everything as "(c) Microsoft" instead of "(c) Microsoft and 999 more contributors, one of which has actually forgot to put their name in the AUTHORS.txt and thus we're technically violating the terms of the license"
    - withinboredom 3y ago
      Yep, the CLA only applies if you want to merge your changes into the project.
  - easton 3y ago
    IIRC, it’s to allow them to relicense contributions in the event they decide to move away from MIT?
jiggawatts 3y ago
I just wish they had something like this embedded into Azure App Service, so it wouldn't be necessarily to use a remote service for caching. For reference, something commonly used with IIS for ASP.NET apps was to have an out-of-process "session state" store, so that if the web app process restarted, users wouldn't lose their sessions and have to log in from scratch. Sure, you can put this somewhere central like SQL Server, but then every web page request sits there waiting for the session state to load before it does any processing at all. Session state is also typically locked in some way, which has all sorts of performance issues. The typical current solution is to use Redis for both caching and session state, and this works... okay-ish. Throughput is high, sure, but Redis is a separate resource in Azure and is stupidly expensive. I really don't want to pay Oracle DB prices for something this simple. It's also a bit of a hassle to wire up. In this article they talk about 300 microsecond response times, but that's irrelevant in any zone-redundant design because all Azure load balancers use random zone selection. So you'll have a web server picked in a random zone, then it'll contact a cache server in a random zone in turn. That server in turn may not have your key and have to contact yet another random zone to get your cache data! Your traffic ping-pongs between data centres. This introduces about 1-3ms of delays, up to 10x higher than the advertised numbers for Garnet. The ideal scenario would be something like what Microsoft Service Fabric does: it has a "reliable collections"[1] service that runs locally on each host node and replicates to two other nodes. A web app can always read its cached values from the same physical host. The latency can be single-digit microseconds in some cases, which is thousands of times faster than any naively load balanced external service, no matter how well optimised. I don't want 30% faster than Redis. I want 3,000x faster. [1] https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-reliable-services-reliable-collections https://learn.microsoft.com/en-us/azure/service-fabric/servi...
- withinboredom 3y ago
  This is (one reason why) I love k8s with cilium. I can set up a service to always go to the local service (or any other routing topology). It is great for any kind of dns or application cache.
  - zinclozenge 3y ago
legulere 3y ago
> Garnet’s storage layer, called Tsavorite, was forked from our prior open-source project FASTER Would be interesting to know why it was forked, why the changes can't be incorporated and wether FASTER continues to be developed
- compressedgas 3y ago
  I tried to compare the source trees. They are nearly identical except for replacing Faster with Tsavorite which makes a direct comparison much harder as they renamed directories and files.
- withinboredom 3y ago
  “Our” may be the key word here. In other words, it may be political; they wanted the freedom to make changes to their original project without having to deal with PRs to the current maintainers.
zokier 3y ago
So what's the catch, where this doesn't perform well? Would be neat to see benchmarks on smaller instance types too, 72vcpu is quite chunky boi
KyleSanderson 3y ago
Garnet’s storage layer, called Tsavorite, was forked from OSS FASTER, and includes strong database features such as thread scalability, tiered storage support (memory, SSD, and cloud storage), fast non-blocking checkpointing, recovery, operation logging for durability, multi-key transaction support, and better memory management and reuse. https://www.microsoft.com/en-us/research/blog/introducing-garnet-an-open-source-next-generation-faster-cache-store-for-accelerating-applications-and-services/ https://www.microsoft.com/en-us/research/blog/introducing-ga...
- wangyjx 3y ago
  Yes, another OSS project by Microsoft
- mtmk 3y ago
  I did follow FASTER implementation long ago. I thought it was very promising as a persistence library for a proof of concept project with high performance requirements I was working on then. BTW, it looks like both efforts are lead by the same person [1] [1]: https://github.com/badrishc https://github.com/badrishc
cpressland 3y ago
This looks really good. I hope ultimately this replaces the “Azure Cache for Redis” resource. It’s slow, it’s a fork of Redis made to run on Windows, and it takes nearly an hour to create an instance of it.
- robertlagrant 3y ago
  I don't know why they wouldn't just run Redis on Linux.
  - jiggawatts 3y ago
    If you have a Windows container for your app, you can only orchestrate that together with other Windows containers on one dev box. So if you need to spin up a cache as a sidecar, it has to be Windows.
- cawa_msft 3y ago
  Thanks for your interest in Azure Cache for Redis! The Enterprise SKUs (E-series) takes less provisioning time. Around 7 min. Hosted in Linux. Would that fit your requirements?
oaiey 3y ago
It is interesting to see how Microsoft and the .NET team are building some very impressive hack-your-own-infrastructure project. Yarp is a reverse proxy/API gateway/whatever you do. Now Garnet for memory caches. Seems they have tons of internal need and are willing to share.
Weryj 3y ago
Orleans will love being bundled with this
- jeremycarter 3y ago
  Each Orleans node could have a Garnet node, I can see the Aspire configuration now
  - deskamess 3y ago
    I am just happy there is a native Redis-like available on Windows. I believe there is another RavenDB replacement but this one is 'more' official! Is it possible to use Aspire locally or is it just a cloud only 'framework'?
throwaway38375 3y ago
Neat project, but I will be sticking with Redis. I trust Redis to not do something weird with their licensing or pricing in the future. Plus Redis has billions of production hours under its belt. It's easier to install and understand.
- oaiey 3y ago
  100% on that. I think Garnet (and similar YARP) are useful if you build your own tweaked version of that. Otherwise, off-the-shelf or PaaS thing are more useful. Also, this is Microsoft Research. This thing is code sharing not a product. The real interesting part is what Azure will do with it.
- sciurus 3y ago
  The licensing has been through a number of changes in the last few years, depending on which distribution and modules you use. https://redis.com/legal/licenses/ https://redis.com/legal/licenses/
- dmw_ng 3y ago
  Tech has all the attention span of a goldfish https://techcrunch.com/2019/02/21/redis-labs-changes-its-open-source-license-again/ https://techcrunch.com/2019/02/21/redis-labs-changes-its-ope...
- Someone 3y ago
  This is a project from Microsoft Research, so I would worry a lot less about licensing and pricing than about lack of updates (either features, maintenance, or security ones)
  - jeremycarter 3y ago
    I see this becoming an Azure service. But you are right, half of the Microsoft Research projects fizzle out within a year - unless used internally.
    - esafak 3y ago
DonnyV 3y ago
I'm looking forward to see where they are using this in production. "After thousands of unit tests and a couple of years working with first-party teams at Microsoft deploying Garnet in production (more on this in future blog posts!), we felt it was time to release it publicly" https://microsoft.github.io/garnet/blog https://microsoft.github.io/garnet/blog
caleblloyd 3y ago
> Garnet being multi-threaded, `MSET` is not atomic. For an atomic version of `MSET`, you would need to express it as a transaction (stored procedure). I am having trouble understanding this. Why wouldn't they wrap that in a transaction internally for you, and make the command atomic? What other atomicity "gotchas" are there.
- danbruc 3y ago
  Because it would mean a performance penalty for everyone who does not need MSET to be atomic without being able to opt out of that transaction. On the other hand, if you want to be a drop-in replacement for Redis, then this is an issue as Redis guarantees atomicity. Maybe you could have a configuration option that lets you select between compatibility and performance, at least if being a drop-in replacement is a design goal.
ksec 3y ago
Judging from comments here I guess no one uses memcached anymore ?
- dormando 3y ago
  :'(
  - 2oMg3YWV26eKIs 3y ago
    <3 Thanks for your continuing work on memcached! I'd be very curious how garnet's benchmarks compare with memcached.
  - ksec 3y ago
    <3 and Thank You :)
- jamesfinlayson 3y ago
  I remember last using it in 2016 or so and it didn't seem particularly easy to configure (we wanted an LRU cache and it had a bunch of options that weren't very well explained, and we couldn't work out why it kept evicting items). We swapped it out for Redis and it was easy to configure and worked how we expected.
mtmk 3y ago
This is excellent news for people who needs to run Redis (or compatible in this case) directly on Microsoft Windows Server, without relying on WSL2. Previously, there was a Redis port available [1] (which is now in archive status) that had memory usage issues (mainly because of memory-mapped files AFAIK) and, of course, is no longer supported. It's also quite intriguing for me to see it's written in C#, as that's my native tongue. I'd be keen on dedicating some time to delve into the code. [1]: https://github.com/microsoftarchive/redis https://github.com/microsoftarchive/redis

370 points by saganus 3y ago

lxe 3y ago

A drop-in redis replacement with rather impressive latency and throughput bench figures. Wonder what its like to operate in a non-azure stack in the real world.

nurettin 3y ago
Are you sure it is drop-in? I don't see any indication of xstream support.
- StepWeiwu 3y ago
  The title does say that it can work with existing redis clients but it's unclear if they mean full compatibility.
  - vanhanenjjv 3y ago
    https://microsoft.github.io/garnet/docs/commands/api-compatibility https://microsoft.github.io/garnet/docs/commands/api-compati...

giancarlostoro 3y ago

Definitely impressive, Microsoft Research comes out with some impressive projects from time to time, must be fun getting paid to do R&D. I wish big companies did more R&D style projects that benefit the industry in general. I sure hope a good company takes over Hashicorp if they're on the market to be bought.

west0n 3y ago

From the benchmark performance charts(https://microsoft.github.io/garnet/docs/benchmarking/results-resp-bench https://microsoft.github.io/garnet/docs/benchmarking/results...), the throughput of the GET command exceeds that of Dragonfly by more than tenfold. While 50% latency is slightly higher than Dragonfly, the 99th percentile is slightly lower than Dragonfly. Both the throughput and latency of Garnet and Dragonfly are far better than Redis, indicating that Redis may require a significant performance optimization.

west0n 3y ago
What surprises me the most is that this project is developed in C#, while Dragonfly is developed in C++, and Redis is in C.
- amir734jj 3y ago
  If you look at the store code you see a lot of "unsafe" C# code (i.e pointer manipulation)
  - zigzag312 3y ago
    Shows how flexible C# is. You can trade high-level expressiveness for low-level control where needed and you don't have to deal with any FFI to do it.
    - CharlieDigital 3y ago
      C# and .NET are both truly underrated in the wider community -- I think -- because of some early snafus. The late adoption of an OSS model, Windows-centric in the .NET Framework days, too dependent on Visual Studio for a very long time. Nowadays, I think it's probably the most natural language and platform for teams that need to move on from TypeScript rather than Go or Rust given the similar constructs and idioms.
      - mdaniel 3y ago
        For a while there I had high hopes that one could pull in Java libraries via their JVM interop on top of the Common Language Runtime, but it doesn't seem to have caught on and I'm not in that ecosystem enough to know why. But yes, for the many excellent reasons you cited it means the library ecosystem is nowhere near that of the JVM, and thus I haven't once considered .net for a new project regardless of how much I love C# the language. Well, that, and my experience with the observability and introspection components of the JVM are second to none

wokwokwok 3y ago

After the aborted/abandoned attempt to port redis to windows (0) this feels like a second-try at the same thing, but first party. Of course, as a research project it doesn't have the same stability / support, etc. as redis, but I could easily imagine this rolling into a real product if it's popular. ...and as an MIT license, if nothing else, the code is a fun read. :) [0] - https://github.com/microsoftarchive/redis?tab=readme-ov-file#redis-on-windows https://github.com/microsoftarchive/redis?tab=readme-ov-file...

mordae 3y ago
MIT license, but with CLA. It's bizzare. What more rights could they possibly want on top of MIT to warrant CLA?
- orthoxerox 3y ago
  It's probably a blanket requirement by the MSFT legal team so they can just release everything as "(c) Microsoft" instead of "(c) Microsoft and 999 more contributors, one of which has actually forgot to put their name in the AUTHORS.txt and thus we're technically violating the terms of the license"
  - withinboredom 3y ago
    Yep, the CLA only applies if you want to merge your changes into the project.
- easton 3y ago
  IIRC, it’s to allow them to relicense contributions in the event they decide to move away from MIT?

jiggawatts 3y ago

I just wish they had something like this embedded into Azure App Service, so it wouldn't be necessarily to use a remote service for caching. For reference, something commonly used with IIS for ASP.NET apps was to have an out-of-process "session state" store, so that if the web app process restarted, users wouldn't lose their sessions and have to log in from scratch. Sure, you can put this somewhere central like SQL Server, but then every web page request sits there waiting for the session state to load before it does any processing at all. Session state is also typically locked in some way, which has all sorts of performance issues. The typical current solution is to use Redis for both caching and session state, and this works... okay-ish. Throughput is high, sure, but Redis is a separate resource in Azure and is stupidly expensive. I really don't want to pay Oracle DB prices for something this simple. It's also a bit of a hassle to wire up. In this article they talk about 300 microsecond response times, but that's irrelevant in any zone-redundant design because all Azure load balancers use random zone selection. So you'll have a web server picked in a random zone, then it'll contact a cache server in a random zone in turn. That server in turn may not have your key and have to contact yet another random zone to get your cache data! Your traffic ping-pongs between data centres. This introduces about 1-3ms of delays, up to 10x higher than the advertised numbers for Garnet. The ideal scenario would be something like what Microsoft Service Fabric does: it has a "reliable collections"[1] service that runs locally on each host node and replicates to two other nodes. A web app can always read its cached values from the same physical host. The latency can be single-digit microseconds in some cases, which is thousands of times faster than any naively load balanced external service, no matter how well optimised. I don't want 30% faster than Redis. I want 3,000x faster. [1] https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-reliable-services-reliable-collections https://learn.microsoft.com/en-us/azure/service-fabric/servi...

withinboredom 3y ago
This is (one reason why) I love k8s with cilium. I can set up a service to always go to the local service (or any other routing topology). It is great for any kind of dns or application cache.
- zinclozenge 3y ago

legulere 3y ago

> Garnet’s storage layer, called Tsavorite, was forked from our prior open-source project FASTER Would be interesting to know why it was forked, why the changes can't be incorporated and wether FASTER continues to be developed

compressedgas 3y ago
I tried to compare the source trees. They are nearly identical except for replacing Faster with Tsavorite which makes a direct comparison much harder as they renamed directories and files.
withinboredom 3y ago
“Our” may be the key word here. In other words, it may be political; they wanted the freedom to make changes to their original project without having to deal with PRs to the current maintainers.

zokier 3y ago

So what's the catch, where this doesn't perform well? Would be neat to see benchmarks on smaller instance types too, 72vcpu is quite chunky boi

KyleSanderson 3y ago

Garnet’s storage layer, called Tsavorite, was forked from OSS FASTER, and includes strong database features such as thread scalability, tiered storage support (memory, SSD, and cloud storage), fast non-blocking checkpointing, recovery, operation logging for durability, multi-key transaction support, and better memory management and reuse. https://www.microsoft.com/en-us/research/blog/introducing-garnet-an-open-source-next-generation-faster-cache-store-for-accelerating-applications-and-services/ https://www.microsoft.com/en-us/research/blog/introducing-ga...

wangyjx 3y ago
Yes, another OSS project by Microsoft
mtmk 3y ago
I did follow FASTER implementation long ago. I thought it was very promising as a persistence library for a proof of concept project with high performance requirements I was working on then. BTW, it looks like both efforts are lead by the same person [1] [1]: https://github.com/badrishc https://github.com/badrishc

cpressland 3y ago

This looks really good. I hope ultimately this replaces the “Azure Cache for Redis” resource. It’s slow, it’s a fork of Redis made to run on Windows, and it takes nearly an hour to create an instance of it.

robertlagrant 3y ago
I don't know why they wouldn't just run Redis on Linux.
- jiggawatts 3y ago
  If you have a Windows container for your app, you can only orchestrate that together with other Windows containers on one dev box. So if you need to spin up a cache as a sidecar, it has to be Windows.
cawa_msft 3y ago
Thanks for your interest in Azure Cache for Redis! The Enterprise SKUs (E-series) takes less provisioning time. Around 7 min. Hosted in Linux. Would that fit your requirements?

oaiey 3y ago

It is interesting to see how Microsoft and the .NET team are building some very impressive hack-your-own-infrastructure project. Yarp is a reverse proxy/API gateway/whatever you do. Now Garnet for memory caches. Seems they have tons of internal need and are willing to share.

Weryj 3y ago

Orleans will love being bundled with this

jeremycarter 3y ago
Each Orleans node could have a Garnet node, I can see the Aspire configuration now
- deskamess 3y ago
  I am just happy there is a native Redis-like available on Windows. I believe there is another RavenDB replacement but this one is 'more' official! Is it possible to use Aspire locally or is it just a cloud only 'framework'?

throwaway38375 3y ago

Neat project, but I will be sticking with Redis. I trust Redis to not do something weird with their licensing or pricing in the future. Plus Redis has billions of production hours under its belt. It's easier to install and understand.

oaiey 3y ago
100% on that. I think Garnet (and similar YARP) are useful if you build your own tweaked version of that. Otherwise, off-the-shelf or PaaS thing are more useful. Also, this is Microsoft Research. This thing is code sharing not a product. The real interesting part is what Azure will do with it.
sciurus 3y ago
The licensing has been through a number of changes in the last few years, depending on which distribution and modules you use. https://redis.com/legal/licenses/ https://redis.com/legal/licenses/
dmw_ng 3y ago
Tech has all the attention span of a goldfish https://techcrunch.com/2019/02/21/redis-labs-changes-its-open-source-license-again/ https://techcrunch.com/2019/02/21/redis-labs-changes-its-ope...
Someone 3y ago
This is a project from Microsoft Research, so I would worry a lot less about licensing and pricing than about lack of updates (either features, maintenance, or security ones)
- jeremycarter 3y ago
  I see this becoming an Azure service. But you are right, half of the Microsoft Research projects fizzle out within a year - unless used internally.
  - esafak 3y ago

DonnyV 3y ago

I'm looking forward to see where they are using this in production. "After thousands of unit tests and a couple of years working with first-party teams at Microsoft deploying Garnet in production (more on this in future blog posts!), we felt it was time to release it publicly" https://microsoft.github.io/garnet/blog https://microsoft.github.io/garnet/blog

caleblloyd 3y ago

> Garnet being multi-threaded, `MSET` is not atomic. For an atomic version of `MSET`, you would need to express it as a transaction (stored procedure). I am having trouble understanding this. Why wouldn't they wrap that in a transaction internally for you, and make the command atomic? What other atomicity "gotchas" are there.

danbruc 3y ago
Because it would mean a performance penalty for everyone who does not need MSET to be atomic without being able to opt out of that transaction. On the other hand, if you want to be a drop-in replacement for Redis, then this is an issue as Redis guarantees atomicity. Maybe you could have a configuration option that lets you select between compatibility and performance, at least if being a drop-in replacement is a design goal.

ksec 3y ago

Judging from comments here I guess no one uses memcached anymore ?

dormando 3y ago
:'(
- 2oMg3YWV26eKIs 3y ago
  <3 Thanks for your continuing work on memcached! I'd be very curious how garnet's benchmarks compare with memcached.
- ksec 3y ago
  <3 and Thank You :)
jamesfinlayson 3y ago
I remember last using it in 2016 or so and it didn't seem particularly easy to configure (we wanted an LRU cache and it had a bunch of options that weren't very well explained, and we couldn't work out why it kept evicting items). We swapped it out for Redis and it was easy to configure and worked how we expected.

mtmk 3y ago

This is excellent news for people who needs to run Redis (or compatible in this case) directly on Microsoft Windows Server, without relying on WSL2. Previously, there was a Redis port available [1] (which is now in archive status) that had memory usage issues (mainly because of memory-mapped files AFAIK) and, of course, is no longer supported. It's also quite intriguing for me to see it's written in C#, as that's my native tongue. I'd be keen on dedicating some time to delve into the code. [1]: https://github.com/microsoftarchive/redis https://github.com/microsoftarchive/redis