4 ms·
Is this a bad thing? You can just increase the number of front-end servers and have high throughput in a service with reliable tail latency.
by inlined 10y ago
Is this a bad thing? You can just increase the number of front-end servers and have high throughput in a service with reliable tail latency.
- heavenlyhash 10y agoThis. It's a lot easier to horizontally scale things with a lean towards consistently lower operational latency. You can keep raking in the benefits and cranking up throughput without a whole lot of thought. It's much more expensive and complex to take an erratic latency operation and bring it down by throwing on more resources. As far as I can tell, the normal design course is making sure all your major actions are either pure or idempotent allowing parallel (and redundant!) requests to be made... which is a large (worthy, but large) engineering effort, and then we're talking about scaling to 2x or more just so you can make that redundant-request thing your default behavior.
- jsiepkes 10y agoIn case of Java there is also a JVM implementation that does pauseless GC (zing): https://www.azul.com/products/zing/pgc/ https://www.azul.com/products/zing/pgc/ . For OpemJDK/Hotspot there will be a <100ms GC in Java 9: http://openjdk.java.net/jeps/189 http://openjdk.java.net/jeps/189
- geodel 10y ago'Pauseless' is marketing term like those unlimited data plans from telecom/cable providers. Here is how Zing use flags: -XX:+UseGenPauselessGC - Generational Pauseless GC (GPGC) optimized for minimal pause times.
- johncolanduoni 10y agoYou're right, Zing pause times are "only" uncorrelated with heap size or usage patterns.
- avar 10y agoAnother approach you can use in some cases with the JVM, which is often the simplest, is to set up the JVM so it doesn't GC (give it a lot of memory), then either just spawn a new JVM to take over, or take the machine about to run a GC out of your load-balanced pool before running a full GC, then put it back in again. Doing the manually triggered & staggered GC trick on a pool of machines you control can give you very low latency guarantees, since no production request will ever hit a GC-ing JVM.
- mSparks 10y agoPersonally found Java's GC to be a little tricky but generally awesome. However, 100, even 200ms pauses get "lost" during network coms. But users tend to notice if a page takes 30 seconds to load rather than 1 second (differences i've seen optimising Javas GC, don't know at all how Go and Java compare). That kind of difference would be very expensive to solve using more servers.
- pjmlp 10y agoThe biggest issue is on single deployments like desktop or embedded, hence why these kind of improvements are so relevant.
- mSparks 10y agoThat's the key trade of these days in software imho. What kind of memory management you need. Sever side zero memory leaks absolute requirement and "real time" responses rarely a requirement. triggering gc during init and/or shutdown of a component often enough. Building a CNC machine - every tick is valuable, but as long as it can run a job for 2 days before crashing no one will notice if it leaks memory like a sieve when you run a calibration routine.
- kuschku 10y ago"just" You can also "just" remove servers that are currently in GC from your pool, and have high throughput in a service with reliable latency.