5 ms·

For .NET you should really read about structs, which allocates on the stack instead of the heap, and thus avoids the GC most of the time. Oracle is working on a

by Skinney 12y ago

For .NET you should really read about structs, which allocates on the stack instead of the heap, and thus avoids the GC most of the time. Oracle is working on a struct implementation for Java 9 or Java 10. Using an arena/buffer/cache in Java and .NET is not unheard of, and is a good way of avoiding allocations, and thus postponing collections. An arena will, however, increase your live memory which means that when a collection finally does happen, it will take longer. The arena will probably be promoted to the oldgen quickly though, so the collector shouldn't scan it to often. "if nothing points into this entire arena dump it in one chunk" -> is more or less how a GC works, so you get that for free. The way the GC does it, is that it scans everything that is live and considers everything not scanned as garbage, which is then collected. The beautiful part of this, is that even if you have allocated 1Gb of memory, you only pay for what is live. If you have 200mb of live memory, collection will be constant regardless of how much garbage you have, whether it is 1Gb or 2Gb of garbage. Increasing the size of memory of course makes collections happens less often. Your point regarding JIT is irrelevant in this particular case, as this case is exclusively related to the GC. We would be talking about the same potential speed improvements here with or without a JIT. That being said, you are also wrong. A JIT can produce faster code than any pre-compiler, due to there being more information available, and because a JIT can easily "undo" compiled code, allowing it to experiment with code that can fail in certain rare cases, but most of the time is vastly faster. Mono, the open source .NET implementation, can be pre-compiled using LLVM (originally a C/C++ compiler), but using the JIT produces faster code. The reason Java and .NET is "slow" for idiomatic (not higly optimized, which most code isn't) code, is due to the GC, which causes the application to freeze at certain intervals. This problem increases the larger live memory you have. A GC is probably faster overall compared to manual memory management though, the problem with GCs today is that they de-allocate (garbage collect) all at once, instead of de-allocating memory once said memory becomes garbage, which evens things out a bit. That being said, there are of course downsides to a JIT. It takes time before the JIT is "warm", which is to say it takes time before most of the code in the application is compiled. If you have applications that run for a short amount of time (I think Java's JIT compiles functions that are run 32 times or more), like shell scripts, they get no performance benefit of using a JIT. Another thing is memory use. Since a JIT compiles code as it goes, and nothing is pre-compiled, the JIT has to store compiled code in memory. So a JIT will naturally use more memory than a pre-compiled program, and a GC normally ads to this.

MichaelGG 12y ago
Except since everything is sharing the nursery (or everything on that core or what have you), you've got no control when you need to allocate slightly longer term objects. So your GC run ends up not being great anyways. Despite the great speed of the GCs, it still adds unnecessary work and as you note, can cause considerable slowdowns. But if there was a simple way to stack alloc objects when safe, that could mean a lot, and is certainly faster, and is hassle free- barely worth calling manual management. In fact, there is a way to do this, by using unsafe code and setting up your own object headers; except the GC won't scan your object, so you gotta be extra careful. The line about the JIT emitting better code is often repeated. I used to repeat it myself. But diving into the emitted code, I'm not overly impressed. So, sure, in theory, JIT codes going to be all amazing, taking advantage of my cache sizes, common flows, etc. In real life, this doesn't happen. As I mentioned, simply inlining critical code by hand results in improvements, meaning the JIT is doing a poor job in the first place. Structs even used to prevent optimizations in the CLR codegen!
- Skinney 12y ago
  "Except since everything is sharing the nursery (or everything on that core or what have you), you've got no control when you need to allocate slightly longer term objects." Of course you don't have control whether to place an object in the nursery or the oldgen, that is an implementation detail, but that has nothing to do with the nursery being shared across all threads. Erlang is, as far as I know, the only VM that doesn't share GC space between threads (actually, processes) but has to rely on string-copying, which has it's own issues. Even here, you can't decide where to place an object. "So your GC run ends up not being great anyways." When I worked on a C/C++ project a couple of years ago, me and a colleague spent six weeks tracking down a memory leak. I've considered GC runs to be the most amazing thing in software ever since. "Despite the great speed of the GCs, it still adds unnecessary work and as you note, can cause considerable slowdowns." The work a GC does is strictly necessary, and the collection time isn't really bad (depends on implementation of course), the only places you might have a problem is in realtime apps like games. In this case however, the GC related problem seems to be quite easily remedied by rewriting some code, and thousands are using a several year old, and crappy, GC in unity and mono 2.x. For most applications these days, the GC really isn't the issue, the code is. "But if there was a simple way to stack alloc objects when safe" Value types, Coming Soon to Java, already available in .NET. Of course, using these in the wrong places can prevent or work against later optimizations. Go gives you greater control over this, but that is Go. "But diving into the emitted code, I'm not overly impressed." There are certain things to consider here, like tiered compilation. Another thing to point out is also that the exact code you were looking at was difficult for the VM to optimise, or that it didn't run so often that the VM found it necessary to optimise as much as other functions. We had some developer tool we ported over from C++ to Java, more or less a straight copy of code, with changes to match the different APIs, the Java version was consistently faster (given enough runtime). Sure, you could probably make the C++ version run faster given enough skill and time, but then it wouldn't be idiomatic code anymore, and there the Java version excelled. "As I mentioned, simply inlining critical code by hand results in improvements, meaning the JIT is doing a poor job in the first place." There are several reasons why this might be the case. HotSpot has (I believe) an 'inline-budget' which means it only inlines functions less than a certain bytecode size. Similarly, there are other limitations on what code the JVM will optimise. Just because manual inlining worked in your particular example, does not mean you will get the same benefit every time. "Structs even used to prevent optimizations in the CLR codegen!" Who cares about 'used to'? It's normal that quirks are worked out over time, and that things improve. By the same logic, it's normal that Java produces faster code than .NET, and it's normal that GCC/LLVM produces faster native code today than ten years ago.

by Skinney 12y ago

MichaelGG 12y ago

Except since everything is sharing the nursery (or everything on that core or what have you), you've got no control when you need to allocate slightly longer term objects. So your GC run ends up not being great anyways. Despite the great speed of the GCs, it still adds unnecessary work and as you note, can cause considerable slowdowns. But if there was a simple way to stack alloc objects when safe, that could mean a lot, and is certainly faster, and is hassle free- barely worth calling manual management. In fact, there is a way to do this, by using unsafe code and setting up your own object headers; except the GC won't scan your object, so you gotta be extra careful. The line about the JIT emitting better code is often repeated. I used to repeat it myself. But diving into the emitted code, I'm not overly impressed. So, sure, in theory, JIT codes going to be all amazing, taking advantage of my cache sizes, common flows, etc. In real life, this doesn't happen. As I mentioned, simply inlining critical code by hand results in improvements, meaning the JIT is doing a poor job in the first place. Structs even used to prevent optimizations in the CLR codegen!

Skinney 12y ago
"Except since everything is sharing the nursery (or everything on that core or what have you), you've got no control when you need to allocate slightly longer term objects." Of course you don't have control whether to place an object in the nursery or the oldgen, that is an implementation detail, but that has nothing to do with the nursery being shared across all threads. Erlang is, as far as I know, the only VM that doesn't share GC space between threads (actually, processes) but has to rely on string-copying, which has it's own issues. Even here, you can't decide where to place an object. "So your GC run ends up not being great anyways." When I worked on a C/C++ project a couple of years ago, me and a colleague spent six weeks tracking down a memory leak. I've considered GC runs to be the most amazing thing in software ever since. "Despite the great speed of the GCs, it still adds unnecessary work and as you note, can cause considerable slowdowns." The work a GC does is strictly necessary, and the collection time isn't really bad (depends on implementation of course), the only places you might have a problem is in realtime apps like games. In this case however, the GC related problem seems to be quite easily remedied by rewriting some code, and thousands are using a several year old, and crappy, GC in unity and mono 2.x. For most applications these days, the GC really isn't the issue, the code is. "But if there was a simple way to stack alloc objects when safe" Value types, Coming Soon to Java, already available in .NET. Of course, using these in the wrong places can prevent or work against later optimizations. Go gives you greater control over this, but that is Go. "But diving into the emitted code, I'm not overly impressed." There are certain things to consider here, like tiered compilation. Another thing to point out is also that the exact code you were looking at was difficult for the VM to optimise, or that it didn't run so often that the VM found it necessary to optimise as much as other functions. We had some developer tool we ported over from C++ to Java, more or less a straight copy of code, with changes to match the different APIs, the Java version was consistently faster (given enough runtime). Sure, you could probably make the C++ version run faster given enough skill and time, but then it wouldn't be idiomatic code anymore, and there the Java version excelled. "As I mentioned, simply inlining critical code by hand results in improvements, meaning the JIT is doing a poor job in the first place." There are several reasons why this might be the case. HotSpot has (I believe) an 'inline-budget' which means it only inlines functions less than a certain bytecode size. Similarly, there are other limitations on what code the JVM will optimise. Just because manual inlining worked in your particular example, does not mean you will get the same benefit every time. "Structs even used to prevent optimizations in the CLR codegen!" Who cares about 'used to'? It's normal that quirks are worked out over time, and that things improve. By the same logic, it's normal that Java produces faster code than .NET, and it's normal that GCC/LLVM produces faster native code today than ten years ago.