12 ms·

I'm considering building a Java compatible VM that only has static memory allocation to avoid GC. Why hasn't this been done by anyone yet? Also confused why t

by bullen 4y ago

I'm considering building a Java compatible VM that only has static memory allocation to avoid GC. Why hasn't this been done by anyone yet? Also confused why this does not have a binary windows release yet: https://github.com/bytecodealliance/wasm-micro-runtime https://github.com/bytecodealliance/wasm-micro-runtime Edit: Epsilon is not the answer here you can stop mentioning that.

beardyw 4y ago
Not sure how that would play out. Just not running the GC is an unsatisfactory answer. Otherwise what?
- bullen 4y ago
  Basically you enforce static heap allocation. Meaning you cannot type new in a method only in the class definition. I don't even know if this can be enforced from the bytecode VM... just playing with it in my head.
  - ako 4y ago
    So the only place you really define what objects get created is in the class containing the main method? Seems fairly limited that you’d have to know upfront what your memory needs are? An app would have to allocate a large block of memory in the main class, and then handle memory management itself from this piece of memory?
    - bullen 4y ago
      Yes, you would have to overallocate and reuse. This is the way I code C now allready.
      - ako 4y ago
        That would remove a lot of the benefits of using Java, might as well use something like c. Garbage collection has been highly optimized in the jvm, and most would consider it a benefit of a jvm compared to managing memory yourself.
        the_only_law 4y ago
        I’ve never understood it either. I’ve heard people doing similar things with Java in the low-latency trading area. I mean Java isn’t the worst tool out there, but even C# outclasses it as a language. The biggest plus for Java is the massive, mature ecosystem which I imagine mostly evaporates when you’re only using a self restrictive subset of the language itself.
        kaba0 4y ago
        It doesn’t evaporate, more often than not only a small subset of these trading programs need that strict “no heap allocation” policy. The rest of the program is free to take advantage of the huge ecosystem.
      - invalidname 4y ago
        I have a bit of a different idea along somewhat different lines. Tweeted at you.
  - pron 4y ago
    That would violate the Java specification, i.e. it's not Java. On the other hand, what you could do is only allow allocations to succeed in class initialisation (Java's "static") and throw an error if done outside it. Note, however, that even adding an element to a Map outside of initialisers would fail when using OpenJDK's standard library even if both key and value have been preallocated in initialisers, as it may allocate an internal node and/or a new hash array. So you may want to change some of the standard library to make it more useful with your restrictions. But you may want to ask yourself why you want to do that. OpenJDK's GCs have become really, really good in both throughput and latency, and the main thing you pay in exchange is memory footprint.
karussell 4y ago
Isn't this already possible with -XX:+UseEpsilonGC? (disables the GC) And then you manage memory manually in big (primitive) arrays.
- bzzzt 4y ago
  Beat me to this answer ;) - I presume you can just allocate objects but you have to keep those allocations in check to prevent the JVM from terminating when going out of memory. Maybe object pooling (which helped performance in old JVM's in the 90s) will make a comeback? ;)
  - bullen 4y ago
    I'm going to try and prevent heap allocation in runtime. But since I'm going to use javac it's going to be ugly. Basically only static atomic arrays (int/float) in classes will be allowed, for cache and parallelism, and AoS up to 64 bytes encouraged to avoid parallel cache invalidation. And I'm even considering dropping float, and only have integer fixed point... but then I'll need to convert those in shaders as GPUs are hardcoded to float.
    - garblegarble 4y ago
      Not to be too negative here but... why? You'd be creating something syntax-compatible with Java, but where you wouldn't be able to use any existing Java code & unable to use most of the interesting features of Java (even string concatenation is handled by instantiating a new StringBuilder on your behalf by javac). Aren't you just reinventing a worse C at that point?
      - bullen 4y ago
        See my other 2 responses. On mobile here would reference on PC.
        usrusr 4y ago
        What would be the point of that things relationship with java though? Is it somehow important that the bytecode run by a tiny vm that is apparently distributed alongside could also run on a proper JVM?
bzzzt 4y ago
Because there's no need? There the 'epsilon' no-op GC in OpenJDK: https://openjdk.org/jeps/318 https://openjdk.org/jeps/318 If you keep your object allocation in check you can use this to guarantee no GC pauses.
- bullen 4y ago
  Yes, but then you are downloading and running executables that contain the GC.
  - bzzzt 4y ago
    The Java "GC" also contains the memory allocator and handling for out of memory conditions, so if you want to use Java there needs to be 'something' there to call.
lmm 4y ago
> Why hasn't this been done by anyone yet? What would be the point? Java programs assume GC (or at least, they assume they can allocate memory and not worry about when it will be freed). If you don't have GC then you're not going to be compatible with extant Java programs, so what's the point in trying to be "Java compatible" at all?
- singularity2001 4y ago
  > Java programs assume GC only users of long running programs assume GC, the program itself doesnt care.
  - marginalia_nu 4y ago
    The programmer most likely assumes short-lived objects are "free" (as they effectively are in Java), and can be allocated in loops and so on without filling up memory and without incurring any real penalty to performance (via TLAB or possibly elision).
- bullen 4y ago
  Because 1) you don't want GC pauses 2) you don't want GC code to bloat the VM 3) You don't want memory leaks 4) You don't want people to not know what they are allocating 5) Static allocation is enough to make anything. 6) int arrays do not have cache misses (but you might need to pad them to avoid cache invalidation) 7) parallel atomic multicore works on int arrays out of the box!
  - sorokod 4y ago
    I assume that by "you" you mean "I" From my perspective I am happy with the tradeoff of hanving gc pauses and not needing to manage memory manually.
  - trimbo 4y ago
    Why wouldn't you use C++?
    - 4y ago
kasperni 4y ago
From another comment > Meaning you cannot type new in a method only in the class definition. How would you handle the ~ 1000 java.x classes that even a minimal HelloWorld programs uses. There are plenty of random allocation.
- bullen 4y ago
  I'm not writing thus for old code, I'm writing it for new code. Specifically 3D action MMO scripting.
pron 4y ago
> Why hasn't this been done by anyone yet? That depends what you mean by "this." The Java spec requires you to support `new A()` by returning a fresh object or throwing a VMError (like OutOfMemoryError). If by "this" you mean that allocations would succeed until some fixed amount of memory is exhausted, then, as others have pointed out, this has been done even in OpenJDK. If by "this" you mean that the allocation of some particular set of objects -- say, only those allocated during class initialisation -- would succeed regardless of memory consumption and all others would fail with an OutOfMemoryError, then I guess that it hasn't been done in that particular way because people haven't found it particularly useful as most Java programs would fail, but you can give it a try. RTSJ, the specification for hard-realtime Java [1][2], actually goes further than that and supports both "static" memory (ImmortalMemory) and arenas (ScopedMemory). So if that's what you mean, then it has been done. [1]: https://www.rtsj.org/specjavadoc/book_index.html https://www.rtsj.org/specjavadoc/book_index.html [2]: https://www.aicas.com/download/rtsj/rtsj_76.pdf https://www.aicas.com/download/rtsj/rtsj_76.pdf
imhoguy 4y ago
You can check Java Card to see how GC-less low memory programing was sorted out there. https://en.m.wikipedia.org/wiki/Java_Card https://en.m.wikipedia.org/wiki/Java_Card
marginalia_nu 4y ago
Why do you want to avoid GC?
- bullen 4y ago
  See above.
tryfinally 4y ago
Not many people are motivated to take optimization this far in the Java world. On the .NET side, the language is more suitable because you can define your own value types (coming soon in Java AFAIK), and there is a lot of people using C# for game dev which is a big use case for GC optimization. Unity has been integrating unmanaged allocators in their engine, which lets developers skip the GC much more easily (without having to manually preallocate and reuse memory, which isn’t anyone’s favorite workflow, and doesn’t save you much compared to a fast native allocator). I’ve also seen a couple of roughly equivalent projects, eg. Smmalloc-CSharp, github.com/alaisi/nalloc.
grashalm 4y ago
Nobody did that because java is a safe language by design. Manual allocation would make it an unsafe language. For the vast majority of applications correctness wins over performance. Short lived objects are extremely cheap on the JVM with the right GC. Almost stack allocation cheap. So if you write code that avoids too many long lived objects there is almost no overhead to a GC. For long lived objects GC based compaction can give you even some performance advantage over manual allocation due to better memory locality. But that heavily depends on the application of course. In my experience people blame the GC way too early. Most often the application is just poorly written and some small tweaks can fix gc spikes.
- deleted 4y ago
  [deleted]
wtetzner 4y ago
At that point, why even use Java?
- bullen 4y ago
  See my other answers...
jffhn 4y ago
>Why hasn't this been done by anyone yet? I used to do Java apps which didn't allocate memory in the long run, and so did some banks for high frequency trading, by reusing objects on the spot or using pools for objects with a life cycle. It imposes some programming style/patterns, in particular for APIs design, but it's perfectly doable, so I guess people prefer to just do that. In what context that wouldn't be enough? For proof?

by bullen 4y ago

beardyw 4y ago

Not sure how that would play out. Just not running the GC is an unsatisfactory answer. Otherwise what?

bullen 4y ago
Basically you enforce static heap allocation. Meaning you cannot type new in a method only in the class definition. I don't even know if this can be enforced from the bytecode VM... just playing with it in my head.
- ako 4y ago
  So the only place you really define what objects get created is in the class containing the main method? Seems fairly limited that you’d have to know upfront what your memory needs are? An app would have to allocate a large block of memory in the main class, and then handle memory management itself from this piece of memory?
  - bullen 4y ago
    Yes, you would have to overallocate and reuse. This is the way I code C now allready.
    - ako 4y ago
      That would remove a lot of the benefits of using Java, might as well use something like c. Garbage collection has been highly optimized in the jvm, and most would consider it a benefit of a jvm compared to managing memory yourself.
      - the_only_law 4y ago
        I’ve never understood it either. I’ve heard people doing similar things with Java in the low-latency trading area. I mean Java isn’t the worst tool out there, but even C# outclasses it as a language. The biggest plus for Java is the massive, mature ecosystem which I imagine mostly evaporates when you’re only using a self restrictive subset of the language itself.
        kaba0 4y ago
        It doesn’t evaporate, more often than not only a small subset of these trading programs need that strict “no heap allocation” policy. The rest of the program is free to take advantage of the huge ecosystem.
    - invalidname 4y ago
      I have a bit of a different idea along somewhat different lines. Tweeted at you.
- pron 4y ago
  That would violate the Java specification, i.e. it's not Java. On the other hand, what you could do is only allow allocations to succeed in class initialisation (Java's "static") and throw an error if done outside it. Note, however, that even adding an element to a Map outside of initialisers would fail when using OpenJDK's standard library even if both key and value have been preallocated in initialisers, as it may allocate an internal node and/or a new hash array. So you may want to change some of the standard library to make it more useful with your restrictions. But you may want to ask yourself why you want to do that. OpenJDK's GCs have become really, really good in both throughput and latency, and the main thing you pay in exchange is memory footprint.

karussell 4y ago

Isn't this already possible with -XX:+UseEpsilonGC? (disables the GC) And then you manage memory manually in big (primitive) arrays.

bzzzt 4y ago
Beat me to this answer ;) - I presume you can just allocate objects but you have to keep those allocations in check to prevent the JVM from terminating when going out of memory. Maybe object pooling (which helped performance in old JVM's in the 90s) will make a comeback? ;)
- bullen 4y ago
  I'm going to try and prevent heap allocation in runtime. But since I'm going to use javac it's going to be ugly. Basically only static atomic arrays (int/float) in classes will be allowed, for cache and parallelism, and AoS up to 64 bytes encouraged to avoid parallel cache invalidation. And I'm even considering dropping float, and only have integer fixed point... but then I'll need to convert those in shaders as GPUs are hardcoded to float.
  - garblegarble 4y ago
    Not to be too negative here but... why? You'd be creating something syntax-compatible with Java, but where you wouldn't be able to use any existing Java code & unable to use most of the interesting features of Java (even string concatenation is handled by instantiating a new StringBuilder on your behalf by javac). Aren't you just reinventing a worse C at that point?
    - bullen 4y ago
      See my other 2 responses. On mobile here would reference on PC.
      - usrusr 4y ago
        What would be the point of that things relationship with java though? Is it somehow important that the bytecode run by a tiny vm that is apparently distributed alongside could also run on a proper JVM?

bzzzt 4y ago

Because there's no need? There the 'epsilon' no-op GC in OpenJDK: https://openjdk.org/jeps/318 https://openjdk.org/jeps/318 If you keep your object allocation in check you can use this to guarantee no GC pauses.

bullen 4y ago
Yes, but then you are downloading and running executables that contain the GC.
- bzzzt 4y ago
  The Java "GC" also contains the memory allocator and handling for out of memory conditions, so if you want to use Java there needs to be 'something' there to call.

lmm 4y ago

> Why hasn't this been done by anyone yet? What would be the point? Java programs assume GC (or at least, they assume they can allocate memory and not worry about when it will be freed). If you don't have GC then you're not going to be compatible with extant Java programs, so what's the point in trying to be "Java compatible" at all?

singularity2001 4y ago
> Java programs assume GC only users of long running programs assume GC, the program itself doesnt care.
- marginalia_nu 4y ago
  The programmer most likely assumes short-lived objects are "free" (as they effectively are in Java), and can be allocated in loops and so on without filling up memory and without incurring any real penalty to performance (via TLAB or possibly elision).
bullen 4y ago
Because 1) you don't want GC pauses 2) you don't want GC code to bloat the VM 3) You don't want memory leaks 4) You don't want people to not know what they are allocating 5) Static allocation is enough to make anything. 6) int arrays do not have cache misses (but you might need to pad them to avoid cache invalidation) 7) parallel atomic multicore works on int arrays out of the box!
- sorokod 4y ago
  I assume that by "you" you mean "I" From my perspective I am happy with the tradeoff of hanving gc pauses and not needing to manage memory manually.
- trimbo 4y ago
  Why wouldn't you use C++?
  - 4y ago

kasperni 4y ago

From another comment > Meaning you cannot type new in a method only in the class definition. How would you handle the ~ 1000 java.x classes that even a minimal HelloWorld programs uses. There are plenty of random allocation.

bullen 4y ago
I'm not writing thus for old code, I'm writing it for new code. Specifically 3D action MMO scripting.

pron 4y ago

> Why hasn't this been done by anyone yet? That depends what you mean by "this." The Java spec requires you to support `new A()` by returning a fresh object or throwing a VMError (like OutOfMemoryError). If by "this" you mean that allocations would succeed until some fixed amount of memory is exhausted, then, as others have pointed out, this has been done even in OpenJDK. If by "this" you mean that the allocation of some particular set of objects -- say, only those allocated during class initialisation -- would succeed regardless of memory consumption and all others would fail with an OutOfMemoryError, then I guess that it hasn't been done in that particular way because people haven't found it particularly useful as most Java programs would fail, but you can give it a try. RTSJ, the specification for hard-realtime Java [1][2], actually goes further than that and supports both "static" memory (ImmortalMemory) and arenas (ScopedMemory). So if that's what you mean, then it has been done. [1]: https://www.rtsj.org/specjavadoc/book_index.html https://www.rtsj.org/specjavadoc/book_index.html [2]: https://www.aicas.com/download/rtsj/rtsj_76.pdf https://www.aicas.com/download/rtsj/rtsj_76.pdf

imhoguy 4y ago

You can check Java Card to see how GC-less low memory programing was sorted out there. https://en.m.wikipedia.org/wiki/Java_Card https://en.m.wikipedia.org/wiki/Java_Card

marginalia_nu 4y ago

Why do you want to avoid GC?

bullen 4y ago
See above.

tryfinally 4y ago

Not many people are motivated to take optimization this far in the Java world. On the .NET side, the language is more suitable because you can define your own value types (coming soon in Java AFAIK), and there is a lot of people using C# for game dev which is a big use case for GC optimization. Unity has been integrating unmanaged allocators in their engine, which lets developers skip the GC much more easily (without having to manually preallocate and reuse memory, which isn’t anyone’s favorite workflow, and doesn’t save you much compared to a fast native allocator). I’ve also seen a couple of roughly equivalent projects, eg. Smmalloc-CSharp, github.com/alaisi/nalloc.

grashalm 4y ago

Nobody did that because java is a safe language by design. Manual allocation would make it an unsafe language. For the vast majority of applications correctness wins over performance. Short lived objects are extremely cheap on the JVM with the right GC. Almost stack allocation cheap. So if you write code that avoids too many long lived objects there is almost no overhead to a GC. For long lived objects GC based compaction can give you even some performance advantage over manual allocation due to better memory locality. But that heavily depends on the application of course. In my experience people blame the GC way too early. Most often the application is just poorly written and some small tweaks can fix gc spikes.

deleted 4y ago
[deleted]

wtetzner 4y ago

At that point, why even use Java?

bullen 4y ago
See my other answers...

jffhn 4y ago

>Why hasn't this been done by anyone yet? I used to do Java apps which didn't allocate memory in the long run, and so did some banks for high frequency trading, by reusing objects on the spot or using pools for objects with a life cycle. It imposes some programming style/patterns, in particular for APIs design, but it's perfectly doable, so I guess people prefer to just do that. In what context that wouldn't be enough? For proof?