8 ms·
Maybe we will soon see the Java Virtual Machine implemented wholly in (a subset of) Java...
by pulse7 4y ago
Maybe we will soon see the Java Virtual Machine implemented wholly in (a subset of) Java...
- Thaxll 4y agoIt's a good point, after so long why the JVM is not written in Java actually? For example the Go runtime and I think C# as well are written in their respective language. https://github.com/golang/go/tree/master/src/runtime https://github.com/golang/go/tree/master/src/runtime https://github.com/dotnet/runtime https://github.com/dotnet/runtime
- simias 4y agoMaybe it's my ignorance of the Java ecosystem or a lack of imagination but how could a language that requires a heavy runtime be written in itself? Wouldn't you need a runtime for the runtime, and then a runtime for the runtime for the runtime and then...? I can't quite imagine how you'd bootstrap something like that.
- chrisseaton 4y agoYou implement the runtime itself in Java, and you use that runtime to run your runtime.
- stingraycharles 4y agoExactly, implementing the runtime is the bootstrapping (apart from the compilation of course). There is no implementing the runtime that implements the runtime that implements the runtime.
- throwaway894345 4y agoThe missing detail is that you have to AOT compile the outermost runtime.
- acedTrex 4y agoGraal supports AOT compilation, so you can write the runtime in java and AOT compile it to a given arch. thats what the thread is talking about
- jlouis 4y agoNo. See Scheme48 or Go. There are some ways around that bootstrap problem.
- throwaway894345 4y agoNot sure about Scheme48, but Go's solution is AOT compiling the runtime, which is essentially what the parent is asking about.
- crustycoder 4y agoIt's no different to writing any other compiler for Language X in Language X, and that's a really common thing to do. The Java compiler emits files containing Java Bytecode which are subsequently run by a different invocation of the JVM.
- Taywee 4y agoIt's different in that the JVM is not a compiler. It's a bit like suggesting writing CPython in Python. It's possible only if the interpreter's Python source can be compiled ahead of time, like PyPy.
- chrisseaton 4y ago> It's possible only if the interpreter's Python source can be compiled ahead of time, like PyPy. That's what it does.
- Taywee 4y agoOh, I know, I'm just pointing out that a self-hosted compiler is a different ballgame than a runtime being written in the language that runs on it. Without some degree of AOT compilation (usually of a subset or dialect of the language compiled in a different fashion), it's not really possible. The GP seemed to miss that the discussion was about the JVM, not javac. With Graal, it's possible to write a JVM in Java, but the JVM doesn't depend on another JVM to run, and the way it runs bytecode isn't the way it was compiled in the first place. It's not really self-hosted in the same way that a compiler can be.
- stonemetal12 4y agoYou can write a C compiler in Java. The output of the C compiler doesn't depend on the JVM, it would be a regular old program written in C. Writing a Java runtime would be similar, the output would be an executable that doesn't depend on the JVM.
- Macha 4y agoA runtime is not a compiler. The process of bootstrapping a compiler is understood, but if the runtime itself requires a runtime at runtime, that is much more of a turtles all the way down problem than with a runtime less language. It's like how PyPy is Python but with the asterisk that it's bootstrapped with RPython which is an almost-subset of python so that it doesn't require a runtime. You could define a statically compilable Java subset (like that which gcj used to accept) and build a runtime in that which would mean omitting features such as reflection but a lot of defacto standard java tooling like Spring Framework would not be compatible.
- chrisseaton 4y ago> You could define a statically compilable Java subset (like that which gcj used to accept) and build a runtime in that which would mean omitting features such as reflection but a lot of defacto standard java tooling like Spring Framework would not be compatible. But the JVM you build using your statically compilable Java subset can then run the Spring Framework or whatever.
- Macha 4y agoSure, but that subset is only "kind of Java" just like RPython or CPython code is only "kind of Python". It's like calling a C compiler a C++ or Objective-C compiler because C is a subset of those languages.
- chrisseaton 4y ago> Sure, but that subset is only "kind of Java" How restrictive do you think the subset is? It only doesn't support some features you probably never wanted to use anyway, and arbitrary reflection. I maintain 125k lines of Java that conforms to the subset rules, and to be honest I never even think twice about the fact that it's a subset.
- twic 4y agoSome people at IBM did it in the '90s and wrote a paper about it: https://dl.acm.org/doi/10.1145/320385.320418 https://dl.acm.org/doi/10.1145/320385.320418
- pjmlp 4y agoBy using an existing implementation, and having intrisics. There are a couple of Java implementations written in Java, Jikes RVM being one of the first ones, almost 15 years ago.
- DeathArrow 4y agoJust imagine a Python interpreter written in Python. Then the interpreter could interpret its own code while interpreting its own code.
- sitkack 4y agohttps://www.pypy.org/ https://www.pypy.org/
- maxloh 4y agoJava isn't as performant as C++. BTW, how do you implement a garbage collector with a garbage-collected language?
- chrisseaton 4y ago> BTW, how do you implement a garbage collector with a garbage-collected language? You write it carefully so the garbage collector itself doesn't also need to allocate objects. Here's a GC for Java written in Java https://github.com/oracle/graal/tree/master/substratevm/src/com.oracle.svm.core.genscavenge/src/com/oracle/svm/core/genscavenge https://github.com/oracle/graal/tree/master/substratevm/src/....
- quelltext 4y agoDigging through that code might be a bit challenging. Do you happen to have a link to a paper/documentation on this? Or maybe some rough explanation how it works (that goes beyond the short summary you provided) Java doesn't provide a primitive to deallocate memory. So while I can see how for instance allocation a huge chunk / big array could be allocated and you represent objects in there don't you end up with a situation where your process will always occupy a fixed amount memory? Might not need to be fixed. You might also be able to extend more but how would you free that again?
- chrisseaton 4y agoNot being facetious - but it works exactly like any other GC. There's nothing magic about writing code in Java instead of C that makes a huge difference. But you might find this interesting as a specific example - this is where it actually obtains memory from the OS. https://github.com/oracle/graal/blob/44e68777b130c8ee781c72b0e24dfec922712d85/substratevm/src/com.oracle.svm.core.posix/src/com/oracle/svm/core/posix/PosixVirtualMemoryProvider.java#L126 https://github.com/oracle/graal/blob/44e68777b130c8ee781c72b... Note the @Uninterruptible annotation - that's saying that this code is safe to use within the GC itself. Notice how the file doesn't contain even a single 'new! (Outside of PosixVirtualMemoryProviderFeature, which is something else.)
- mike_hearn 4y agoParts of it are written in Java. Over time, more of the JVM has been rewritten in Java. For example recently parts of the reflection subsystem were rewritten in Java. Of course many libraries have been ported over time too. By the way, the .NET CLR is AFAIK written in C++, like with Hotspot. It's not fully self-hosting. It's a tricky process because HotSpot is highly performance sensitive code. People won't accept regressions just to convenience the JDK maintainers. Java meanwhile is deliberately a simple language to make it accessible for people, so you lose some low level techniques that are useful for performance. Nonetheless, GraalVM native image has proven that you can achieve HotSpot like performance with a JVM written in Java. However it requires a big change in the compilation model that isn't always appropriate.
- pjmlp 4y agoIn each CLR release there are little pieces that transition from C++ to C#, as C# currently is getting quite a few low level features for systems programing.
- vips7L 4y agoThe core CLR is written in C++ just like HotSpot.
- pjmlp 4y agoMostly, since the .NET Core got introduced, the C++ surface area gets reduced with each release, there are occasionally references to it when a new release comes out on MSDN blogs.
- pron 4y agoThe Java runtime contains ~8MLOC; of them, about 1.5M are C++ (and Assembly generated by a C++ DSL), and more and more of the runtime is being written in Java (e.g. the virtual thread scheduler is written in Java). The two main pieces that aren't are the bytecode interpreter, the JITs, and the GCs (although the Graal JIT is written in Java). Interestingly, the more latency-critical stuff is written in Java (the JITs and GCs work mostly in the background nowadays, and the interpreter is used only at startup and on deoptimisation slow-paths). Some of the reason is historical, and some has to do with warmup. Project Leyden and Graal's Native Image will help compile more Java AOT, allowing even more of the runtime to gradually be written in Java. It will take some time as it's not a top priority: it won't immediately deliver user-facing functionality, and most of the work on the JDK is already done in Java code anyway.
- pjmlp 4y agoThere are a couple of Java implementations written in Java, Jikes RVM being one of the first ones, almost 15 years ago. GraalVM is the evolution of MaximeVM, originally developed at SunLabs, also about 15 years ago.
- treffer 4y agoThere was a java implementation of a JVM, jikes rvm. I haven't looked at it for ages and looking at Wikipedia it seems dead, so curious if there is any newer attempts....
- chrisseaton 4y ago> so curious if there is any newer attempts.... You're commenting on an article about a newer attempt?
- agilob 4y agoThis is what it is. GraalVM is implemented in Java, there is a Truffle framework that compiles Java, Ruby, JS, Python to Java Truffle, which then runs on GraalVM JIT compiler.
- e12e 4y agoUnfortunately from tfa: > Oracle plans to contribute the most applicable portions of the GraalVM just-in-time (JIT) compiler and Native Image. Oracle does not currently intend to contribute the polyglot technologies supporting other languages such as Python, Ruby, R, and JavaScript. Additional details will follow in the coming months as we move forward through this process. It would appear this is about making native image artifacts / distribution a first class citizen across all of Java - making it an alternative to uberjars + jvm for running/distribution. Ie native desktop apps and native binaries for servers?
- pencilguin 4y agoWait, GraalVM is implemented in Java? Does that mean to run it you need another JVM to run it on top of? That sounds stupid... Maybe you need another VM just to run it on once, so it can translate itself to native code on the target?
- dragonwriter 4y ago> Does that mean to run it you need another JVM to run it on top of? GraalVM is the regular HotspotVM integrated with the GraalVM compiler (which is normally AOT compiled as a native library, but can be run as a JAR); it also supports tooling with the ability to compile code targeting the JVM to a native executable, as well as leveraging AOT compilation itself. https://www.graalvm.org/22.3/reference-manual/native-image/ https://www.graalvm.org/22.3/reference-manual/native-image/
- kaba0 4y agohttps://news.ycombinator.com/item?id=33341760 https://news.ycombinator.com/item?id=33341760 Graal reuses all the insanely good GC implementations, observability tools etc already present in OpenJDK (as throwing all that away would be stupid).
- dig1 4y agoCheck for Maxine VM [1][2] project. [1] https://en.wikipedia.org/wiki/Maxine_Virtual_Machine https://en.wikipedia.org/wiki/Maxine_Virtual_Machine [2] https://maxine-vm.readthedocs.io/en/latest/ https://maxine-vm.readthedocs.io/en/latest/
- mike_hearn 4y agoA component of the GraalVM project is called native-image. The native-image tool compiles a Java app to machine code ahead of time and combines it with a small JVM called SubstrateVM. That JVM is written in Java, just like the rest of your app. As pointed out by other comments, this concept of a "meta-circular VM" isn't new. It's been done before by two other projects, Maxine and Jikes. What's different about SubstrateVM is that this is a production tool rather than a research project, and it's not just a JVM, it's also a way to pre-initialize the app. Therefore programs compiled with native-image can start as fast as programs written in C. Actually, slightly faster in some cases. You may wonder how that's possible given that Java apps normally start slowly, but it's because there's no JIT compilation and the state of the heap is snapshotted, with classes pre-initialized including the JVM itself. So the program can literally just start executing at main() in machine code with no VM startup overhead, because it's done already. The downside is that snapshotting and AOT consume a lot of disk space. There are some questions below asking how this works. It sounds initially "impossible", like a lot of stuff GraalVM/Truffle does, but it's quite easy to understand really. You start with a bytecode compiler written in Java. This is a normal program written in the normal way, because a compiler is ultimately just a function that converts one stream of bytes to another. Then you write the runtime and GC in Java too, and compile that as well. This code is a bit special. It's still syntactically Java, but, some classes and methods are given special meanings and some extra rules apply. They aren't compiled in the same way as normal Java code. For example you can write code like this: UnsignedWord value = Pointer.readUnsignedWord(address) This doesn't allocate an object or call a static method. Instead it will be compiled down to a single mov instruction. Likewise for writing to memory - there are magic methods that are taken to mean "emit this assembly" instead of doing normal method calls. Several other tricks are required. GC code can't allocate because it would mess up the heap it's working with, so you can use annotations to mark methods as "never access the heap". But then, GC code is written in Java and Java must allocate for almost anything non trivial, so how does that work? The answer is, the GC code is initialized at build time and all the objects it needs are snapshotted into the default heap that's mapped into memory at startup. There are lots of other tricks, mostly annotations that control the compiler so that e.g. methods are guaranteed to be inlined and removed, objects are guaranteed to be stack allocated. This isn't available to normal Java but when you control the compiler it's not a problem. The advantage of this Java-superset (or subset) is that you can use all the normal tools that understand source code, like IntelliJ, JavaDoc etc.