6 ms·

In my experience with Postgres at least, our queries average less than a millisecond in Java. Our slowest query averages 5. We're not running anything crazy, pr

In my experience with Postgres at least, our queries average less than a millisecond in Java. Our slowest query averages 5. We're not running anything crazy, pretty generic CRUD app. We can hit about 50k queries per second on one server from our limited stress testing. Redis was around 150k. In memory cache is ~1.2 million. This is JSON requests using Vert.X on Java. I just don't see the advantage to an out of JVM cache when your service is so fast that you only need more than one instance for failover. In slower languages where you need a bunch of front end servers to handle a reasonable load, out of app caches look more appealing. The hit rate for in app caches becomes too low when you have a bunch of instances. But my point is that you get a lot more performance with a simpler setup if you use a language fast enough that you can run just a few instances and cache locally. Out of app caches are usually an anti pattern brought about by needing too many front end servers because your app is too slow for in-app cache to be useful(more instances means too low hit rate without shared cache). You're effectively forced to use slower out of app caching because of the number of instances you need to run

graycat 7y ago
Sounds like someone should design a server farm architecture tool that will take in various parameters and report designs and what constraints are tight, that is, the solution is right at the constraint.
graycat 7y ago
"We can hit about 50k queries per second on one serveR" WOW! For my 1.0 case, I was thinking maybe 10 a second!
- nullwasamistake 7y ago
  It's really far beyond what we need. We never hit over 5% cpu unless we break an index or do something dumb. But it's great that we can run just a few instances and in-memory cache anything coming back from slow services. We've got a lot of dog slow apps buried in the backend, and running minimal instances makes our hit rate on in-app caches nice enough that we can avoid complexity and slowness of shared cache. Postgres is extremely fast when decently tuned, probably not much slower than Redis(both IO bound). For an ORM the same can be said of Hibernate. And Vert.X is just stupidly fast in general, just need to be careful not to block event loop or use Fibers with Quasar. There's a lot of tweaks for Jackson serialization speed and Hibernate + Postgres performance that are off by default as well. Do we need all this speed? Absolutely not. But caching is one of the most evil things in comp sci and having so much excess capacity allowed us to simplify a lot of things For instance, we cache virtually nothing coming from the DB, only slow backends and other third party services. Hibernate also caches some reads for us, and since we don't have many instances the hit rate is fine on that as well. We probably don't need to worry about performance ever for the future of our app. If the stack wasn't so fast we would need a lot of front end servers and a (slow) shared cache would be needed. Kinda a snowball effect of complexity when you try to retrofit something to be fast down the road
  - Tarq0n 7y ago
    What about cold starts when the cache is empty? Are those painful?
    - nullwasamistake 7y ago
      Not terrible because we do rolling restarts, but if all the servers went down at once it could be a problem.
speedplane 7y ago
> I just don't see the advantage to an out of JVM cache when your service is so fast that you only need more than one instance for failover I don't think anyone would advocate that you move to a remote cache. If you can fit your entire app on a single machine, that's great. It's a huge reduction of complexity, and you'll likely be able to get very fast performance like you indicate. You definitely do not need or want a remote cache. These tools are for those that have more data or traffic than a single machine can handle. Things get much more complicated when you move from one computer to two.

graycat 7y ago

Sounds like someone should design a server farm architecture tool that will take in various parameters and report designs and what constraints are tight, that is, the solution is right at the constraint.

graycat 7y ago

"We can hit about 50k queries per second on one serveR" WOW! For my 1.0 case, I was thinking maybe 10 a second!

nullwasamistake 7y ago
It's really far beyond what we need. We never hit over 5% cpu unless we break an index or do something dumb. But it's great that we can run just a few instances and in-memory cache anything coming back from slow services. We've got a lot of dog slow apps buried in the backend, and running minimal instances makes our hit rate on in-app caches nice enough that we can avoid complexity and slowness of shared cache. Postgres is extremely fast when decently tuned, probably not much slower than Redis(both IO bound). For an ORM the same can be said of Hibernate. And Vert.X is just stupidly fast in general, just need to be careful not to block event loop or use Fibers with Quasar. There's a lot of tweaks for Jackson serialization speed and Hibernate + Postgres performance that are off by default as well. Do we need all this speed? Absolutely not. But caching is one of the most evil things in comp sci and having so much excess capacity allowed us to simplify a lot of things For instance, we cache virtually nothing coming from the DB, only slow backends and other third party services. Hibernate also caches some reads for us, and since we don't have many instances the hit rate is fine on that as well. We probably don't need to worry about performance ever for the future of our app. If the stack wasn't so fast we would need a lot of front end servers and a (slow) shared cache would be needed. Kinda a snowball effect of complexity when you try to retrofit something to be fast down the road
- Tarq0n 7y ago
  What about cold starts when the cache is empty? Are those painful?
  - nullwasamistake 7y ago
    Not terrible because we do rolling restarts, but if all the servers went down at once it could be a problem.

speedplane 7y ago

> I just don't see the advantage to an out of JVM cache when your service is so fast that you only need more than one instance for failover I don't think anyone would advocate that you move to a remote cache. If you can fit your entire app on a single machine, that's great. It's a huge reduction of complexity, and you'll likely be able to get very fast performance like you indicate. You definitely do not need or want a remote cache. These tools are for those that have more data or traffic than a single machine can handle. Things get much more complicated when you move from one computer to two.