5 ms·

This. It's a lot easier to horizontally scale things with a lean towards consistently lower operational latency. You can keep raking in the benefits and crank

This. It's a lot easier to horizontally scale things with a lean towards consistently lower operational latency. You can keep raking in the benefits and cranking up throughput without a whole lot of thought. It's much more expensive and complex to take an erratic latency operation and bring it down by throwing on more resources. As far as I can tell, the normal design course is making sure all your major actions are either pure or idempotent allowing parallel (and redundant!) requests to be made... which is a large (worthy, but large) engineering effort, and then we're talking about scaling to 2x or more just so you can make that redundant-request thing your default behavior.

jsiepkes 10y ago
In case of Java there is also a JVM implementation that does pauseless GC (zing): https://www.azul.com/products/zing/pgc/ https://www.azul.com/products/zing/pgc/ . For OpemJDK/Hotspot there will be a <100ms GC in Java 9: http://openjdk.java.net/jeps/189 http://openjdk.java.net/jeps/189
- geodel 10y ago
  'Pauseless' is marketing term like those unlimited data plans from telecom/cable providers. Here is how Zing use flags: -XX:+UseGenPauselessGC - Generational Pauseless GC (GPGC) optimized for minimal pause times.
  - johncolanduoni 10y ago
    You're right, Zing pause times are "only" uncorrelated with heap size or usage patterns.
avar 10y ago
Another approach you can use in some cases with the JVM, which is often the simplest, is to set up the JVM so it doesn't GC (give it a lot of memory), then either just spawn a new JVM to take over, or take the machine about to run a GC out of your load-balanced pool before running a full GC, then put it back in again. Doing the manually triggered & staggered GC trick on a pool of machines you control can give you very low latency guarantees, since no production request will ever hit a GC-ing JVM.

jsiepkes 10y ago

In case of Java there is also a JVM implementation that does pauseless GC (zing): https://www.azul.com/products/zing/pgc/ https://www.azul.com/products/zing/pgc/ . For OpemJDK/Hotspot there will be a <100ms GC in Java 9: http://openjdk.java.net/jeps/189 http://openjdk.java.net/jeps/189

geodel 10y ago
'Pauseless' is marketing term like those unlimited data plans from telecom/cable providers. Here is how Zing use flags: -XX:+UseGenPauselessGC - Generational Pauseless GC (GPGC) optimized for minimal pause times.
- johncolanduoni 10y ago
  You're right, Zing pause times are "only" uncorrelated with heap size or usage patterns.

avar 10y ago

Another approach you can use in some cases with the JVM, which is often the simplest, is to set up the JVM so it doesn't GC (give it a lot of memory), then either just spawn a new JVM to take over, or take the machine about to run a GC out of your load-balanced pool before running a full GC, then put it back in again. Doing the manually triggered & staggered GC trick on a pool of machines you control can give you very low latency guarantees, since no production request will ever hit a GC-ing JVM.