5 ms·

Oh, please. I once worked on a Java-based server at Google that had to answer requests with millisecond latency. In order to achieve this, the server had to bl

Oh, please. I once worked on a Java-based server at Google that had to answer requests with millisecond latency. In order to achieve this, the server had to block garbage collection most of the time. Periodically, each instance of the server would ask the load balancers to stop sending requests to it, so that it could then safely run the garbage collector, and then ask for traffic to return. We likely would not have used Java had we foreseen needing to do this. I believe some time after I left, it was all rewritten in C++. Sadly the garbage collection debate is full of people who want GC to be the answer to everything, and have chosen their arguments and beliefs through confirmation bias to support their desire. Many of these "GC is faster!" arguments come from that, but it just ain't true most of the time. Performant GC is incredibly complex and incredibly complex systems tend to fall over if you don't use them in exactly the right way. Worse, GC is extremely bad at managing resources other than local memory. In complex distributed systems or other software that manages external resources, GC languages tend to be ill-equipped for the job because they lack explicit resource-management tools like RAII. In Java, you sometimes see frameworks where everything has a `dispose()` method and complex systems are built to make sure that `dispose()` method gets called... this is a failure, it should be handled by destructors. GC is generally nice and convenient for application engineering, but a poor fit for systems engineering. The boundaries between the two are admittedly fuzzy.

luciusdomitius 4y ago
Speaking of this, have you of LMAX Disruptor[0]? You probably have. And yeah, having fine control over the memory allocation and garbage collection in such a high-level language (multi-platform vm!) is akin to building a ship in a bottle. However, in which stack creating multi-level distributed stock exchange with sub-milisecond response times is easy? 0. https://lmax-exchange.github.io/disruptor/ https://lmax-exchange.github.io/disruptor/
ransom1538 4y ago
Google should have picked a real language: php. When script is completed all resources are freed, let the OS figure it out. No manual GC drama.
- MaxBarraclough 4y ago
  That's a strategy called arena allocation or region-based memory-management, and it doesn't have to be tied to the lifetime of a process or interpreter. There's nothing special about PHP, which if I understand correctly uses reference-counting with conventional garbage collection to handle reference cycles. As I understand it this strategy is non-competitive in all performance metrics, compared against modern garbage collectors.
galdosdi 4y ago
> I once worked on a Java-based server at Google This is kind of a fair comment, but kind of not, because Java performance and GC internals have really advanced a lot in the last decade. It would really help if you qualified approximately when this was. > Sadly the garbage collection debate is full of people who want GC to be the answer to everything Good point. I think more recently, the Java world is very aware of this. Native and off heap memory has been getting more use in performance sensitive stuff for quite a while. You can totally just (essentially) malloc and free in Java, if you really need to for performance. That said, if you want to be able to make your code accessible to a wider audience, GC is a must. There are tons of junior and midlevel developers who don't really have experience working with non-GC application code (and in many cases are intimidated by it!), and you will be restricted from hiring any of them if you use a non-GC language. If that's ok, then that's ok, but with Java you can still do a little off heap stuff in the critical part you need to, encapsulate it behind a safe API, while letting the juniors run amok in the rest of the codebase.
- deleted 4y ago
  [deleted]
- kentonv 4y ago
  I agree with all this. Java is a great application programming language. But bullen was arguing that Java is the best for servers of all types, which is what I object to.
  - blub 4y ago
    If one considers applications as GUI programs running on consumer hardware like desktops, laptops or smartphones Java is in reality not at all great at this task. It’s better than Rust sure, but definitely worse than C++, C#, Swift or Objective-C which have access to the native UI frameworks of their respective platforms. I’ve had the opportunity to observe Java’s floundering in this area for a couple of decades now and the only decent Java application that I can tolerate is an IDE from Jetbrains. I’m not happy with the performance or UI, but the alternatives are often worse, because in an unexpected twist they’re written in JavaScript.
  - 4y ago
kelnos 4y ago
> I once worked on a Java-based server at Google that had to answer requests with millisecond latency. That's... not really normal, though, and sounds like the exception that proves the rule. For the vast majority of applications, Java will perform better, be easier to develop, and be safer to run, than an equivalent server written in C or C++. At my previous job we used to run realtime audio through a Java server (RTP streaming). I do remember in the Java 6 days that the GC would need tuning to ensure that GC pauses wouldn't delay audio to the degree that there would be dropouts or perceptual delays. But with Java 8 (and later releases), which came with better GC implementations, those problems just went away. Sure, realtime audio is usually fine with even up to 100ms pauses (or even 200ms, sometimes) -- so this is much more tolerant than your sub-ms example -- but we rarely saw anything even remotely that long with the more modern JVM GC implementations, without really having to tune behavior that much, or at all. Meanwhile, P99 stats for most JVM services were in the low to mid tens of milliseconds, and anything longer was always due to calling out to external services, like relational DBs. For the rare case like Google's, sure, it's absolutely expected and appropriate to need to use a non-GC'd language instead, at least for some things. For pretty much anyone else, the JVM is more than adequate, or can be made adequate with some reasonably simple GC tuning. > Many of these "GC is faster!" arguments come from that, but it just ain't true most of the time. I don't agree. I think it is true most of the time. But I think many people don't think about what they mean by "faster". Faster as in throughput? Sure, a modern, performance-oriented GC (like the JVM's) can very easily beat manual memory management there. Latency? Well, ok, that can be a bit harder, so you need to evaluate things on a case by case basis, and possibly do some GC tuning to get the latencies you need. But even then, you can usually do just as well (or better) on the latency axis as well. Just not always. But I don't subscribe to the "But sometimes..." school of objections. Yes, sometimes some technologies don't work for certain use cases. That's fine. Choose your technology wisely. But in Java's case, it really is just "sometimes". Not most of the time. Let me reiterate, though: Google is not the common case! By a long shot! It is an outlier, and it's expected that a company like Google will have to deviate from the mainstream to reach its performance targets sometimes. But also consider that (from what I understand) even Google has a ton of services written in Java, and they... work just fine, no? > In complex distributed systems or other software that manages external resources, GC languages tend to be ill-equipped for the job because they lack explicit resource-management tools like RAII. Eh. I initially thought of this as a problem, but in practice, I've rarely run into an issue with this sort of thing. Maybe it's because there's still vestiges of the C programmer in me that will always think about memory ownership and lifecycle, even when writing in a GC'd language, but I've rarely had my own issues (or seen issues written by others) where someone has forgotten a `.close()` or `.dispose()` on something. The "try with resources" syntax can help here too, even though IMO it can be kinda cumbersome. And as much as the Java docs tell you to essentially never override `finalize()`, it can be a useful "last ditch" tool to ensure that any "manual dispose" owned references get cleaned up, and you can also add logging there; I'll often do something like "Got to finalize() without disposing of Foo instances: BUG!". I also appreciate when third-party authors who write `dispose()` methods also do this. It's not perfect by any means, but IMO the convenience of relying on garbage collector rather than manual memory management far outweighs this downside. Lately I've been writing a lot of Rust, and I'm enjoying the sort of "middle ground" approach, where I don't have to think about memory management as much, but don't have to worry about GC performance, either. Certainly Rust doesn't eliminate these concerns; I still need to think about ownership and object lifetimes, but it's never "oh crap, I forgot to free() something and there's a memory leak", or "oh crap, I tried to use something after free()ing it and crashed", it's more like "ugh, rustc doesn't agree with me that this object lives long enough and refuses to compile it". Annoying, but I'd rather find this out at compile-time than runtime. But then I'll go back to writing Java or Scala after being in a Rust project for a while, and remember how nice it is to just not have to think about these things. > ...but a poor fit for systems engineering. Absolutely agreed. But I would not call writing distributed network servers "systems engineering", even though I do agree that the boundary between systems and applications engineering is indeed fuzzy.
discreteevent 4y ago
> requests with millisecond latency You're getting into hard real time territory there. That's a different universe and will require special considerations even in C. I think it's fair enough for people to discuss server performance and assume that the context is our normal programming universe.
- kentonv 4y ago
  Of course! There are plenty of use cases, including server use cases, where Java is a great choice. But I was reacting to bullen's assertion that "To use anything else on the server is madness.", and to the rather exaggerated performance claims.
chopin 4y ago
Afaik, a malloc is not guaranteed to return fast. With those requirements it's hard even in C.
- cozzyd 4y ago
  Right but you would typically use a preallocated arena.
  - chii 4y ago
    but if you're going to do that, the same could be done with java, with less pain for the remaining parts of the application that does not require such methods of memory allocation.
    - cozzyd 4y ago
      My Java is rusty, does Java provide things like placement new that make using arenas ergonomic?
      - chii 4y ago
        no, you cannot control allocation via regular means. What i meant is that you make pooling in java, and that effectively makes it work like an arena allocator (in that you end up setting a bit/boolean flag to allocate/deallocate an object). But the pooling can be constrained to just one portion of your app - the hot loop or the bit that needs the low latency. The remaining code - loading resources at startup for example - can just be regular old java that's easy to write.
        chopin 4y ago
        Pooled JDBC connections work this way in Java.

luciusdomitius 4y ago

Speaking of this, have you of LMAX Disruptor[0]? You probably have. And yeah, having fine control over the memory allocation and garbage collection in such a high-level language (multi-platform vm!) is akin to building a ship in a bottle. However, in which stack creating multi-level distributed stock exchange with sub-milisecond response times is easy? 0. https://lmax-exchange.github.io/disruptor/ https://lmax-exchange.github.io/disruptor/

ransom1538 4y ago

Google should have picked a real language: php. When script is completed all resources are freed, let the OS figure it out. No manual GC drama.

MaxBarraclough 4y ago
That's a strategy called arena allocation or region-based memory-management, and it doesn't have to be tied to the lifetime of a process or interpreter. There's nothing special about PHP, which if I understand correctly uses reference-counting with conventional garbage collection to handle reference cycles. As I understand it this strategy is non-competitive in all performance metrics, compared against modern garbage collectors.

galdosdi 4y ago

> I once worked on a Java-based server at Google This is kind of a fair comment, but kind of not, because Java performance and GC internals have really advanced a lot in the last decade. It would really help if you qualified approximately when this was. > Sadly the garbage collection debate is full of people who want GC to be the answer to everything Good point. I think more recently, the Java world is very aware of this. Native and off heap memory has been getting more use in performance sensitive stuff for quite a while. You can totally just (essentially) malloc and free in Java, if you really need to for performance. That said, if you want to be able to make your code accessible to a wider audience, GC is a must. There are tons of junior and midlevel developers who don't really have experience working with non-GC application code (and in many cases are intimidated by it!), and you will be restricted from hiring any of them if you use a non-GC language. If that's ok, then that's ok, but with Java you can still do a little off heap stuff in the critical part you need to, encapsulate it behind a safe API, while letting the juniors run amok in the rest of the codebase.

deleted 4y ago
[deleted]
kentonv 4y ago
I agree with all this. Java is a great application programming language. But bullen was arguing that Java is the best for servers of all types, which is what I object to.
- blub 4y ago
  If one considers applications as GUI programs running on consumer hardware like desktops, laptops or smartphones Java is in reality not at all great at this task. It’s better than Rust sure, but definitely worse than C++, C#, Swift or Objective-C which have access to the native UI frameworks of their respective platforms. I’ve had the opportunity to observe Java’s floundering in this area for a couple of decades now and the only decent Java application that I can tolerate is an IDE from Jetbrains. I’m not happy with the performance or UI, but the alternatives are often worse, because in an unexpected twist they’re written in JavaScript.
- 4y ago

kelnos 4y ago

> I once worked on a Java-based server at Google that had to answer requests with millisecond latency. That's... not really normal, though, and sounds like the exception that proves the rule. For the vast majority of applications, Java will perform better, be easier to develop, and be safer to run, than an equivalent server written in C or C++. At my previous job we used to run realtime audio through a Java server (RTP streaming). I do remember in the Java 6 days that the GC would need tuning to ensure that GC pauses wouldn't delay audio to the degree that there would be dropouts or perceptual delays. But with Java 8 (and later releases), which came with better GC implementations, those problems just went away. Sure, realtime audio is usually fine with even up to 100ms pauses (or even 200ms, sometimes) -- so this is much more tolerant than your sub-ms example -- but we rarely saw anything even remotely that long with the more modern JVM GC implementations, without really having to tune behavior that much, or at all. Meanwhile, P99 stats for most JVM services were in the low to mid tens of milliseconds, and anything longer was always due to calling out to external services, like relational DBs. For the rare case like Google's, sure, it's absolutely expected and appropriate to need to use a non-GC'd language instead, at least for some things. For pretty much anyone else, the JVM is more than adequate, or can be made adequate with some reasonably simple GC tuning. > Many of these "GC is faster!" arguments come from that, but it just ain't true most of the time. I don't agree. I think it is true most of the time. But I think many people don't think about what they mean by "faster". Faster as in throughput? Sure, a modern, performance-oriented GC (like the JVM's) can very easily beat manual memory management there. Latency? Well, ok, that can be a bit harder, so you need to evaluate things on a case by case basis, and possibly do some GC tuning to get the latencies you need. But even then, you can usually do just as well (or better) on the latency axis as well. Just not always. But I don't subscribe to the "But sometimes..." school of objections. Yes, sometimes some technologies don't work for certain use cases. That's fine. Choose your technology wisely. But in Java's case, it really is just "sometimes". Not most of the time. Let me reiterate, though: Google is not the common case! By a long shot! It is an outlier, and it's expected that a company like Google will have to deviate from the mainstream to reach its performance targets sometimes. But also consider that (from what I understand) even Google has a ton of services written in Java, and they... work just fine, no? > In complex distributed systems or other software that manages external resources, GC languages tend to be ill-equipped for the job because they lack explicit resource-management tools like RAII. Eh. I initially thought of this as a problem, but in practice, I've rarely run into an issue with this sort of thing. Maybe it's because there's still vestiges of the C programmer in me that will always think about memory ownership and lifecycle, even when writing in a GC'd language, but I've rarely had my own issues (or seen issues written by others) where someone has forgotten a `.close()` or `.dispose()` on something. The "try with resources" syntax can help here too, even though IMO it can be kinda cumbersome. And as much as the Java docs tell you to essentially never override `finalize()`, it can be a useful "last ditch" tool to ensure that any "manual dispose" owned references get cleaned up, and you can also add logging there; I'll often do something like "Got to finalize() without disposing of Foo instances: BUG!". I also appreciate when third-party authors who write `dispose()` methods also do this. It's not perfect by any means, but IMO the convenience of relying on garbage collector rather than manual memory management far outweighs this downside. Lately I've been writing a lot of Rust, and I'm enjoying the sort of "middle ground" approach, where I don't have to think about memory management as much, but don't have to worry about GC performance, either. Certainly Rust doesn't eliminate these concerns; I still need to think about ownership and object lifetimes, but it's never "oh crap, I forgot to free() something and there's a memory leak", or "oh crap, I tried to use something after free()ing it and crashed", it's more like "ugh, rustc doesn't agree with me that this object lives long enough and refuses to compile it". Annoying, but I'd rather find this out at compile-time than runtime. But then I'll go back to writing Java or Scala after being in a Rust project for a while, and remember how nice it is to just not have to think about these things. > ...but a poor fit for systems engineering. Absolutely agreed. But I would not call writing distributed network servers "systems engineering", even though I do agree that the boundary between systems and applications engineering is indeed fuzzy.

discreteevent 4y ago

> requests with millisecond latency You're getting into hard real time territory there. That's a different universe and will require special considerations even in C. I think it's fair enough for people to discuss server performance and assume that the context is our normal programming universe.

kentonv 4y ago
Of course! There are plenty of use cases, including server use cases, where Java is a great choice. But I was reacting to bullen's assertion that "To use anything else on the server is madness.", and to the rather exaggerated performance claims.

chopin 4y ago

Afaik, a malloc is not guaranteed to return fast. With those requirements it's hard even in C.

cozzyd 4y ago
Right but you would typically use a preallocated arena.
- chii 4y ago
  but if you're going to do that, the same could be done with java, with less pain for the remaining parts of the application that does not require such methods of memory allocation.
  - cozzyd 4y ago
    My Java is rusty, does Java provide things like placement new that make using arenas ergonomic?
    - chii 4y ago
      no, you cannot control allocation via regular means. What i meant is that you make pooling in java, and that effectively makes it work like an arena allocator (in that you end up setting a bit/boolean flag to allocate/deallocate an object). But the pooling can be constrained to just one portion of your app - the hot loop or the bit that needs the low latency. The remaining code - loading resources at startup for example - can just be regular old java that's easy to write.
      - chopin 4y ago
        Pooled JDBC connections work this way in Java.