10 ms·

If you're fighting with the JVM this much, why not just use C++? The problems were garbage collection, lack of control over memory layout, and bloated types. To

If you're fighting with the JVM this much, why not just use C++? The problems were garbage collection, lack of control over memory layout, and bloated types. Tool for the job, no?

gorset 15y ago
The JVM is very good if you are able to handle or avoid the GC pauses. Unless you are running Azul's Zing JVM, some tuning is usually required for non-trivial systems. We have tuned our system to be very fast with low latency most of the time, and handle timeouts by using a load balancer in front of a few servers. The chance that all servers are having a GC pause at the same time can be made sufficiently small.
- maximusprime 15y ago
  -Xincgc avoids pauses no?
  - mbell 15y ago
    In Java 6 -Xincgc simply translates to: -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:+CMSIncrementalMode causes more frequent, smaller pauses. It was designed for low CPU core count, small heap size applications where your willing to pay a CPU premium to gain better response times. It is specifically not recommended for 2+ CPUs and/or large heap sizes. With large heap sizes you'll be paying a large CPU penalty and if your are using / freeing a lot of objects it may not be able to "keep up" with its short pauses resulting in the heap filling up quickly.
  - gorset 15y ago
    There's usually a tradeoff between many small pauses and few big ones (but even with CMS there will be a point where the JVM needs to do a full GC). The usual approach for low latency systems are to spend the GC pauses incrementally over time. But for us, even though we require consistent low latency, big pauses are perfectly fine because we can run several replicas of the system behind a load balancer. Not all systems can be designed to do this, and it usually implies a idempotent architecture where you can run the same transaction multiple times without any problems. But when it's possible, you can start guaranteeing low latency with a very high probability. Worst case: simply run the transaction on all instances and use the result from the first one that answers.
angelbob 15y ago
Those are the problems. But often in C++ you've still got problems with mixed collection types, knowing types at runtime, stability (Java's not perfect, but it's better), ability to replace running code at runtime, et al. Getting Java's garbage collection tuned isn't trivial, but it's still easier than solving any of those problems in C++ at serious scale.
- rubashov 15y ago
  > problems with mixed collection types I don't understand. C++ has exactly the same model for collections of polymorphic types. But you can also pursue data oriented designs that aren't feasible in Java. > knowing types at runtime RTTI works fine. Querying object types at run time is probably a sign of horrible design. > stability By this do you mean unhandled runtime exceptions? If anything it's easier to write C++ free of fatal "null pointer" exceptions by avoiding pointers in favor of references and using RAII well.
  - angelbob 15y ago
    By this do you mean unhandled runtime exceptions? Partially. Mostly I mean that bad code can make your runtime unstable by corrupting memory. You could say "but I write code that never does that". Pretending for a moment that I believe you, you're still assuming nobody else does that, either. RAII is great, modulo the problem that you have to handle the whole "no exceptions in constructors" problem to avoid resource leaks. Querying object types at run time is probably a sign of horrible design Querying object types at run time is usually a sign of doing something that wasn't anticipated at design time. As systems get larger and older, and as they get pieces patched at runtime, doing unanticipated things becomes inevitable. Java isn't designed to be maximally clean and flexible for a tiny team. It's designed to build behemoth software for industry which is very forgiving of problems. C++ doesn't do that as well. I don't understand. C++ has exactly the same model for collections of polymorphic types. Yes and no. C++ has more restrictions in terms of using only RTTI-capable types in mixed collections. What types aren't RTTI-capable? Primitives and structs. Java has the same problem, patched by autoboxing in such a way that you can't actually screw it up thoroughly. C++ lets you get into unfixable problems pretty quickly. You're right - use only instances of classes with virtual functions, never a primitive or non-virtual class, and you can avoid it. But "with enough auditing, the problem goes away" assumes some pretty specific things about your final environment, including the assumption that errors either don't make it through or are very recoverable.
thwarted 15y ago
There does seem to be a line beyond which continuously manually tweaking the GC would be more work than manual memory management. GC allows you to get up and running faster, but the level of detail you need to know of the JVM to do manual tweaking of the GC implementation approaches the level of detail you need to be aware of in your app to do manual memory management. That is to say, optimizing the JVM's GC settings and effectively writing/debugging C and C++ memory management seem to be wizardly-level tasks of about equal standing. TANSTAFL applies.
- angelbob 15y ago
  Sure. If you have this problem in 10% of your code, you can save the wizarding for the other 90% where tuning the garbage collector isn't terribly important, and you wouldn't need to write memory management code for acceptable performance. 10% of your code being performance critical would be a lot, in my experience. The usual number where I've worked is probably closer to 1%.
- Yeroc 15y ago
  I really don't think you can equate tuning the JVM to writing/debugging C and C++ memory management. Tuning the JVM doesn't require wizardry. It requires some attention to detail and careful monitoring but most of what you need to know were shared in these slides. There are also a few whitepapers out there on the subject that are pretty readable. I hardly think changing a few command line arguments on the JVM, using jvisualvm to monitor or -verbosegc etc. are anywhere near as difficult as actually implementing your own memory management system.
barrkel 15y ago
Tuning GC, by and large, lets you treat the whole system as a single abstraction. You monitor allocation rates and patterns, and adjust algorithms and generation sizes to meet your goals. But once you go to C++, you've dived into a world of itty bitty details; all the memory allocation behaviour is internal to the system, and if you want to change how it works, you have to do relatively severe cross-cutting stuff with overloaded new operator, allocator template arguments, etc. It's very invasive and easy to tie different parts of the system together inadvertently. And when you go with C++ you've forsaken type safety (i.e. memory safety). That's a substantial thing to give up, IMO.
- saurik 15y ago
  """And when you go with C++ you've forsaken type safety (i.e. memory safety). That's a substantial thing to give up, IMO.""" <- This is true only if you have poorly abstracted type casts. If you design your C++ program correctly, you simply won't ever be type casting or manually handling any of your memory. In fact, with a well designed system, this also solves your issue regarding centralized memory planning. I mean, it is as if you are assuming that once you touch C++ you are going to have a million little calls to malloc() with casts all over the place (which shouldn't even be true in a well-designed C program).
  - barrkel 15y ago
    I don't think you understand what memory safety means. Please look it up. It's independent of how you write or design your program. It's a property of the language + runtime combination. (Folks out there downvoting me: you could do with some education too... Parent is a completely ignorant (in the best possible sense - easily fixed) comment.)
    - saurik 15y ago
      You cannot escape the world of safety without using an unsafe feature: an unchecked pointer arithmetic or cast operator. These are features I can statically determine you are using from your code, and which I've stated in a well designed system you have fully abstracted out of the actual program. Yes: in a couple places in your software you will have a cast, but it is about as useful to complain about that as it would be to claim that the JVM itself has a cast operator in it; if the program code isn't using it, then the program code can be proven to be as safe as the runtime, and C++ does not remove your ability to make statements like that. So no: I believe that your comment is "completely ignorant" (and rude, to boot).
ig1 15y ago
I used to work at an investment bank where we were able out-do our competitors at latency even though we were using Java and most of the industry was using C++. The ease of building lock-free algorithms and changing threading models in Java was probably a significant factor.
wheels 15y ago
This also doesn't have to be an either-or sort of scenario. It's pretty easy to mix C++ and Java using the Java Native Interface. It's not going to win a beauty pageant, but it's not rocket science either. My company's web services are exactly that: we do all of the heavy graph traversal stuff down in the C++ layer and do the web services and XML parsing and all of that stuff in Java. The Java types that correspond to our graph elements basically just are a reference to our persistent graph store (basically a bunch of mmaped disk structures) which means that to do graph traversal on the Java side we never have to actually load the graph into the JVM. And the glue code between the JVM and our C++ still weighs in at under 1k LOC.
strlen 15y ago
> If you're fighting with the JVM this much, why not just use C++? The problems were garbage collection, lack of control over memory layout, and bloated types. Tool for the job, no? Here's why. Compare these two: http://hg.openjdk.java.net/jdk7/hotspot/jdk/file/9b8c96f96a0f/src/share/classes/java/util/concurrent/ConcurrentLinkedQueue.java http://hg.openjdk.java.net/jdk7/hotspot/jdk/file/9b8c96f96a0... <-- Doug Lea's java.util.concurrent.ConcurrentLinkedQueue https://github.com/afeinberg/lockfree/blob/master/src/lockfree_queue.cc https://github.com/afeinberg/lockfree/blob/master/src/lockfr... <-- my port of above to C++0x https://github.com/afeinberg/lockfree/blob/master/src/hazard_ptr_rec.cc https://github.com/afeinberg/lockfree/blob/master/src/hazard... <-- essentially an implementation of garbage collection that's needed to to work around the ABA problem tl;dr Shared memory concurrency is surprisingly hard with manual memory management. Not impossible, not infeasible, not impractical. Just hard. C++ is still a valid choice for many products: JVM isolates you from the underlying OS, the VM subsystem, and there are cases where the cost of garbage collection is prohibitive. However, I'll argue that vast majority of a site like Twitter (or LinkedIn, another high-scale JVM powered property) is best served by runtime like the JVM or CLR. Erlang is another great option, but it's more of something you'll have _along with_ JVM/CLR and C/C++: Erlang's model is (highly efficient, well abstracted) concurrency with message passing -- which is awesome, but not a full substitute for shared memory concurrency -- i.e., it's a great tool for some jobs, but not others. C++ makes more sense for things like a B+Tree implementation: I've been using a pure Java B+Tree implementation -- BerkeleyDB Java Edition, and can certainly mention the negatives of that approach. On the other hand, look at something like the routing layer of Voldemort, multi-Paxos implementation in ZooKeeper, or (as an example of another high scale, Java based service) the modified Paxos implementation in Google's MegaStore, non-storage parts of Amazon's SimpleDB (written at Java at one point, in Erlang at another -- not sure what it's written in now). I'll also argue that I'd rather use a less verbose and more functional language than Java -- and honestly, in some cases C++ far outdoes Java in terms of expressiveness. Go, Scala, languages in the ML family and Erlang (especially with tools like dialyzer and quick check, to get around the dynamic typing) are the right way forward for building highly concurrent user-land "systems-y" software (think more databases or distributed middleware than an OS) -- the parts where memory layout is critical and which tend to produce a lot of garbage can always be implemented in C/C++. tl;dr Programming language choice involves trade offs, Java is too verbose, but garbage collected languages/runtimes have their role in building highly scalable applications or service

If you're fighting with the JVM this much, why not just use C++? The problems were garbage collection, lack of control over memory layout, and bloated types. Tool for the job, no?

gorset 15y ago

The JVM is very good if you are able to handle or avoid the GC pauses. Unless you are running Azul's Zing JVM, some tuning is usually required for non-trivial systems. We have tuned our system to be very fast with low latency most of the time, and handle timeouts by using a load balancer in front of a few servers. The chance that all servers are having a GC pause at the same time can be made sufficiently small.

maximusprime 15y ago
-Xincgc avoids pauses no?
- mbell 15y ago
  In Java 6 -Xincgc simply translates to: -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:+CMSIncrementalMode causes more frequent, smaller pauses. It was designed for low CPU core count, small heap size applications where your willing to pay a CPU premium to gain better response times. It is specifically not recommended for 2+ CPUs and/or large heap sizes. With large heap sizes you'll be paying a large CPU penalty and if your are using / freeing a lot of objects it may not be able to "keep up" with its short pauses resulting in the heap filling up quickly.
- gorset 15y ago
  There's usually a tradeoff between many small pauses and few big ones (but even with CMS there will be a point where the JVM needs to do a full GC). The usual approach for low latency systems are to spend the GC pauses incrementally over time. But for us, even though we require consistent low latency, big pauses are perfectly fine because we can run several replicas of the system behind a load balancer. Not all systems can be designed to do this, and it usually implies a idempotent architecture where you can run the same transaction multiple times without any problems. But when it's possible, you can start guaranteeing low latency with a very high probability. Worst case: simply run the transaction on all instances and use the result from the first one that answers.

angelbob 15y ago

Those are the problems. But often in C++ you've still got problems with mixed collection types, knowing types at runtime, stability (Java's not perfect, but it's better), ability to replace running code at runtime, et al. Getting Java's garbage collection tuned isn't trivial, but it's still easier than solving any of those problems in C++ at serious scale.

rubashov 15y ago
> problems with mixed collection types I don't understand. C++ has exactly the same model for collections of polymorphic types. But you can also pursue data oriented designs that aren't feasible in Java. > knowing types at runtime RTTI works fine. Querying object types at run time is probably a sign of horrible design. > stability By this do you mean unhandled runtime exceptions? If anything it's easier to write C++ free of fatal "null pointer" exceptions by avoiding pointers in favor of references and using RAII well.
- angelbob 15y ago
  By this do you mean unhandled runtime exceptions? Partially. Mostly I mean that bad code can make your runtime unstable by corrupting memory. You could say "but I write code that never does that". Pretending for a moment that I believe you, you're still assuming nobody else does that, either. RAII is great, modulo the problem that you have to handle the whole "no exceptions in constructors" problem to avoid resource leaks. Querying object types at run time is probably a sign of horrible design Querying object types at run time is usually a sign of doing something that wasn't anticipated at design time. As systems get larger and older, and as they get pieces patched at runtime, doing unanticipated things becomes inevitable. Java isn't designed to be maximally clean and flexible for a tiny team. It's designed to build behemoth software for industry which is very forgiving of problems. C++ doesn't do that as well. I don't understand. C++ has exactly the same model for collections of polymorphic types. Yes and no. C++ has more restrictions in terms of using only RTTI-capable types in mixed collections. What types aren't RTTI-capable? Primitives and structs. Java has the same problem, patched by autoboxing in such a way that you can't actually screw it up thoroughly. C++ lets you get into unfixable problems pretty quickly. You're right - use only instances of classes with virtual functions, never a primitive or non-virtual class, and you can avoid it. But "with enough auditing, the problem goes away" assumes some pretty specific things about your final environment, including the assumption that errors either don't make it through or are very recoverable.

thwarted 15y ago

There does seem to be a line beyond which continuously manually tweaking the GC would be more work than manual memory management. GC allows you to get up and running faster, but the level of detail you need to know of the JVM to do manual tweaking of the GC implementation approaches the level of detail you need to be aware of in your app to do manual memory management. That is to say, optimizing the JVM's GC settings and effectively writing/debugging C and C++ memory management seem to be wizardly-level tasks of about equal standing. TANSTAFL applies.

angelbob 15y ago
Sure. If you have this problem in 10% of your code, you can save the wizarding for the other 90% where tuning the garbage collector isn't terribly important, and you wouldn't need to write memory management code for acceptable performance. 10% of your code being performance critical would be a lot, in my experience. The usual number where I've worked is probably closer to 1%.
Yeroc 15y ago
I really don't think you can equate tuning the JVM to writing/debugging C and C++ memory management. Tuning the JVM doesn't require wizardry. It requires some attention to detail and careful monitoring but most of what you need to know were shared in these slides. There are also a few whitepapers out there on the subject that are pretty readable. I hardly think changing a few command line arguments on the JVM, using jvisualvm to monitor or -verbosegc etc. are anywhere near as difficult as actually implementing your own memory management system.

barrkel 15y ago

Tuning GC, by and large, lets you treat the whole system as a single abstraction. You monitor allocation rates and patterns, and adjust algorithms and generation sizes to meet your goals. But once you go to C++, you've dived into a world of itty bitty details; all the memory allocation behaviour is internal to the system, and if you want to change how it works, you have to do relatively severe cross-cutting stuff with overloaded new operator, allocator template arguments, etc. It's very invasive and easy to tie different parts of the system together inadvertently. And when you go with C++ you've forsaken type safety (i.e. memory safety). That's a substantial thing to give up, IMO.

saurik 15y ago
"""And when you go with C++ you've forsaken type safety (i.e. memory safety). That's a substantial thing to give up, IMO.""" <- This is true only if you have poorly abstracted type casts. If you design your C++ program correctly, you simply won't ever be type casting or manually handling any of your memory. In fact, with a well designed system, this also solves your issue regarding centralized memory planning. I mean, it is as if you are assuming that once you touch C++ you are going to have a million little calls to malloc() with casts all over the place (which shouldn't even be true in a well-designed C program).
- barrkel 15y ago
  I don't think you understand what memory safety means. Please look it up. It's independent of how you write or design your program. It's a property of the language + runtime combination. (Folks out there downvoting me: you could do with some education too... Parent is a completely ignorant (in the best possible sense - easily fixed) comment.)
  - saurik 15y ago
    You cannot escape the world of safety without using an unsafe feature: an unchecked pointer arithmetic or cast operator. These are features I can statically determine you are using from your code, and which I've stated in a well designed system you have fully abstracted out of the actual program. Yes: in a couple places in your software you will have a cast, but it is about as useful to complain about that as it would be to claim that the JVM itself has a cast operator in it; if the program code isn't using it, then the program code can be proven to be as safe as the runtime, and C++ does not remove your ability to make statements like that. So no: I believe that your comment is "completely ignorant" (and rude, to boot).

ig1 15y ago

I used to work at an investment bank where we were able out-do our competitors at latency even though we were using Java and most of the industry was using C++. The ease of building lock-free algorithms and changing threading models in Java was probably a significant factor.

wheels 15y ago

This also doesn't have to be an either-or sort of scenario. It's pretty easy to mix C++ and Java using the Java Native Interface. It's not going to win a beauty pageant, but it's not rocket science either. My company's web services are exactly that: we do all of the heavy graph traversal stuff down in the C++ layer and do the web services and XML parsing and all of that stuff in Java. The Java types that correspond to our graph elements basically just are a reference to our persistent graph store (basically a bunch of mmaped disk structures) which means that to do graph traversal on the Java side we never have to actually load the graph into the JVM. And the glue code between the JVM and our C++ still weighs in at under 1k LOC.

strlen 15y ago

> If you're fighting with the JVM this much, why not just use C++? The problems were garbage collection, lack of control over memory layout, and bloated types. Tool for the job, no? Here's why. Compare these two: http://hg.openjdk.java.net/jdk7/hotspot/jdk/file/9b8c96f96a0f/src/share/classes/java/util/concurrent/ConcurrentLinkedQueue.java http://hg.openjdk.java.net/jdk7/hotspot/jdk/file/9b8c96f96a0... <-- Doug Lea's java.util.concurrent.ConcurrentLinkedQueue https://github.com/afeinberg/lockfree/blob/master/src/lockfree_queue.cc https://github.com/afeinberg/lockfree/blob/master/src/lockfr... <-- my port of above to C++0x https://github.com/afeinberg/lockfree/blob/master/src/hazard_ptr_rec.cc https://github.com/afeinberg/lockfree/blob/master/src/hazard... <-- essentially an implementation of garbage collection that's needed to to work around the ABA problem tl;dr Shared memory concurrency is surprisingly hard with manual memory management. Not impossible, not infeasible, not impractical. Just hard. C++ is still a valid choice for many products: JVM isolates you from the underlying OS, the VM subsystem, and there are cases where the cost of garbage collection is prohibitive. However, I'll argue that vast majority of a site like Twitter (or LinkedIn, another high-scale JVM powered property) is best served by runtime like the JVM or CLR. Erlang is another great option, but it's more of something you'll have _along with_ JVM/CLR and C/C++: Erlang's model is (highly efficient, well abstracted) concurrency with message passing -- which is awesome, but not a full substitute for shared memory concurrency -- i.e., it's a great tool for some jobs, but not others. C++ makes more sense for things like a B+Tree implementation: I've been using a pure Java B+Tree implementation -- BerkeleyDB Java Edition, and can certainly mention the negatives of that approach. On the other hand, look at something like the routing layer of Voldemort, multi-Paxos implementation in ZooKeeper, or (as an example of another high scale, Java based service) the modified Paxos implementation in Google's MegaStore, non-storage parts of Amazon's SimpleDB (written at Java at one point, in Erlang at another -- not sure what it's written in now). I'll also argue that I'd rather use a less verbose and more functional language than Java -- and honestly, in some cases C++ far outdoes Java in terms of expressiveness. Go, Scala, languages in the ML family and Erlang (especially with tools like dialyzer and quick check, to get around the dynamic typing) are the right way forward for building highly concurrent user-land "systems-y" software (think more databases or distributed middleware than an OS) -- the parts where memory layout is critical and which tend to produce a lot of garbage can always be implemented in C/C++. tl;dr Programming language choice involves trade offs, Java is too verbose, but garbage collected languages/runtimes have their role in building highly scalable applications or service