4 ms·

I'm astounded at the number of developers in my own company that reach for Redis when they just need a cache. It's a huge waste of effort for far worse performa

I'm astounded at the number of developers in my own company that reach for Redis when they just need a cache. It's a huge waste of effort for far worse performance and less reliability than something like Java's Caffiene. Stateful servers run into huge problems when other people try to access them from non-static ip (usually sticky sessions are IP based). We, at least in apps I work on, stuff all the shared state into JWT tokens to avoid this. Everything else that needs to be cached goes into an in-memory in-app store. If you're using a "fast" language to start with, you won't need more than a dozen servers unless you're at Google scale so cache hit rates are good even though each instance maintains it's own. Again if you're using a "fast" language memory overhead from in app storage isn't much more than putting things in a dedicated KV store. There's really no point to it when the performance drop is usually 20x. We use DropWizard and Vert.X extensively, and most of our endpoints can handle many thousands of requests per second. The ones that use the cache instead of hitting the DB every time hit upwards of 1 million requests/sec. You're very lucky to get 50k with any remote KV store in the same app. Most load balancer strategies send many requests in a row from a client to the same backend server anyways for TCP performance reasons. The only reason for a KV cache is if you're running a slow language like Ruby, PHP, Python, where the memory overhead for objects is large and the speed of a remote cache is roughly the same as one in memory. If you find yourself needing a cache for good enough performance you shouldn't be using these languages in the first place. In memory caches don't work well when you need 100+ server instances to handle the load anyways

zemo 7y ago
> Stateful servers run into huge problems when other people try to access them from non-static ip (usually sticky sessions are IP based). We, at least in apps I work on, stuff all the shared state into JWT tokens to avoid this. They're not saying that an application server should never query another server, they're saying that the server that functions as a state container should host a domain-specific api for manipulating the state. See figure 2 b; the frontend and stateful application server layers are separate layers.
deleted 7y ago
[deleted]
alexchamberlain 7y ago
I run several Python servers backed by Redis; Redis was added when we profiled the apps and determined that loading data from several databases was taking too much time. Using a different language wouldn't have changed how much time various RDBMSs take to return an answer to your query. I think there's some citation needed on the performance of in memory vs out of process caches in interpreted languages.
- nullwasamistake 7y ago
  In my experience with Postgres at least, our queries average less than a millisecond in Java. Our slowest query averages 5. We're not running anything crazy, pretty generic CRUD app. We can hit about 50k queries per second on one server from our limited stress testing. Redis was around 150k. In memory cache is ~1.2 million. This is JSON requests using Vert.X on Java. I just don't see the advantage to an out of JVM cache when your service is so fast that you only need more than one instance for failover. In slower languages where you need a bunch of front end servers to handle a reasonable load, out of app caches look more appealing. The hit rate for in app caches becomes too low when you have a bunch of instances. But my point is that you get a lot more performance with a simpler setup if you use a language fast enough that you can run just a few instances and cache locally. Out of app caches are usually an anti pattern brought about by needing too many front end servers because your app is too slow for in-app cache to be useful(more instances means too low hit rate without shared cache). You're effectively forced to use slower out of app caching because of the number of instances you need to run
  - graycat 7y ago
    Sounds like someone should design a server farm architecture tool that will take in various parameters and report designs and what constraints are tight, that is, the solution is right at the constraint.
  - graycat 7y ago
    "We can hit about 50k queries per second on one serveR" WOW! For my 1.0 case, I was thinking maybe 10 a second!
twa927 7y ago
> The only reason for a KV cache is if you're running a slow language like Ruby, PHP, Python, where the memory overhead for objects is large and the speed of a remote cache is roughly the same as one in memory. If you find yourself needing a cache for good enough performance you shouldn't be using these languages in the first place. In memory caches don't work well when you need 100+ server instances to handle the load anyways You're saying reading a serialized Python object from Redis and deserializing it is comparable to accessing a ready in-memory object? No, it's an order of magnitude slower. I think the reason in-memory caches are not used much in Python/Ruby is a concurrency model, ie. in Python/Ruby you usually have to run multiple processes and in Java you run multiple threads, so the in-memory cache can be just a regular data structure normally accessed. Also, I think you are overestimating the Python's memory overhead compared to Java - Python's int is 24 bytes and Java's Integer is 16 bytes (and using PyPy is another story - it can use several times less memory compared to Java).
hliyan 7y ago
I have to agree. When I first moved from the C++ world to the Web world five years ago, I was surprised to see an entire generation of engineers for whom the idea of using application memory is alien. It's as if we're ignoring an entire layer of the memory hierarchy because we're afraid of managing the complexity of application state with load balancing / sticky sessions.
- huffmsa 7y ago
  But once you get to deployment, particularly if you're using shiny new containerized orchestration, in my experience, it is safer and easier to allocate the host node's memory to the redis container and keep it protected from application layer fires. Same as you would do with a database. It speeds up CI/CD and keeps things safe / reduces downtime in cases of roll back (assuming you're versioning your keys).

zemo 7y ago

> Stateful servers run into huge problems when other people try to access them from non-static ip (usually sticky sessions are IP based). We, at least in apps I work on, stuff all the shared state into JWT tokens to avoid this. They're not saying that an application server should never query another server, they're saying that the server that functions as a state container should host a domain-specific api for manipulating the state. See figure 2 b; the frontend and stateful application server layers are separate layers.

deleted 7y ago

[deleted]

alexchamberlain 7y ago

I run several Python servers backed by Redis; Redis was added when we profiled the apps and determined that loading data from several databases was taking too much time. Using a different language wouldn't have changed how much time various RDBMSs take to return an answer to your query. I think there's some citation needed on the performance of in memory vs out of process caches in interpreted languages.

nullwasamistake 7y ago
In my experience with Postgres at least, our queries average less than a millisecond in Java. Our slowest query averages 5. We're not running anything crazy, pretty generic CRUD app. We can hit about 50k queries per second on one server from our limited stress testing. Redis was around 150k. In memory cache is ~1.2 million. This is JSON requests using Vert.X on Java. I just don't see the advantage to an out of JVM cache when your service is so fast that you only need more than one instance for failover. In slower languages where you need a bunch of front end servers to handle a reasonable load, out of app caches look more appealing. The hit rate for in app caches becomes too low when you have a bunch of instances. But my point is that you get a lot more performance with a simpler setup if you use a language fast enough that you can run just a few instances and cache locally. Out of app caches are usually an anti pattern brought about by needing too many front end servers because your app is too slow for in-app cache to be useful(more instances means too low hit rate without shared cache). You're effectively forced to use slower out of app caching because of the number of instances you need to run
- graycat 7y ago
  Sounds like someone should design a server farm architecture tool that will take in various parameters and report designs and what constraints are tight, that is, the solution is right at the constraint.
- graycat 7y ago
  "We can hit about 50k queries per second on one serveR" WOW! For my 1.0 case, I was thinking maybe 10 a second!

twa927 7y ago

> The only reason for a KV cache is if you're running a slow language like Ruby, PHP, Python, where the memory overhead for objects is large and the speed of a remote cache is roughly the same as one in memory. If you find yourself needing a cache for good enough performance you shouldn't be using these languages in the first place. In memory caches don't work well when you need 100+ server instances to handle the load anyways You're saying reading a serialized Python object from Redis and deserializing it is comparable to accessing a ready in-memory object? No, it's an order of magnitude slower. I think the reason in-memory caches are not used much in Python/Ruby is a concurrency model, ie. in Python/Ruby you usually have to run multiple processes and in Java you run multiple threads, so the in-memory cache can be just a regular data structure normally accessed. Also, I think you are overestimating the Python's memory overhead compared to Java - Python's int is 24 bytes and Java's Integer is 16 bytes (and using PyPy is another story - it can use several times less memory compared to Java).

hliyan 7y ago

I have to agree. When I first moved from the C++ world to the Web world five years ago, I was surprised to see an entire generation of engineers for whom the idea of using application memory is alien. It's as if we're ignoring an entire layer of the memory hierarchy because we're afraid of managing the complexity of application state with load balancing / sticky sessions.

huffmsa 7y ago
But once you get to deployment, particularly if you're using shiny new containerized orchestration, in my experience, it is safer and easier to allocate the host node's memory to the redis container and keep it protected from application layer fires. Same as you would do with a database. It speeds up CI/CD and keeps things safe / reduces downtime in cases of roll back (assuming you're versioning your keys).