16 ms·
Jacobin: A more than minimal JVM written in Go
- lopatin 3y agoI wonder if it uses Go routines as the virtual threads implementation.
- jerf 3y agoIf I am reading the code correctly, and as a parachute code reviewer I may very well not be, I am not sure this supports threading at all, or at least, not user-level threading. I can find references to a thread module and it clearly creates a "main" thread, but I actually can't find anywhere where it creates anything other than a main thread, nor do I see any code for task switching manually/cooperatively, or anything else like that. There's a global thread table but I can only find test code manipulating it. And the main core VM loop doesn't seem to have any concurrency in it to me. Corrections welcome; posted fully in the spirit of Cunningham's Law as I'm curious too.
- deleted 3y ago[deleted]
- layer8 3y agoThey’ll have to implement java.lang.Thread somehow, or not a lot will run on that JVM.
- samus 3y agoCorrect, implementing the core libraries is actually a massive effort. You can't just take the Hotspot one because it is a massive cumbersome codebase and it would force your JVM to work like Hotspot internally. (Let's ignore for a moment the question whether that would even be legal)
- andrewbinstock 3y agoLead dev here. Yup, that's right. There are a few classes that we will be forced to implement in Go. We're fixated on keeping this number as low as possible, but Thread is one that is inescapable. Class is another inescapable one--if we're to support reflection, etc.
- twic 3y agoI concur. Native methods have hardcoded implementations, wired up here, and there is nothing for java.lang.Thread: https://github.com/platypusguy/jacobin/blob/18c541820cb73bdaab24ea4b52a4221273dd84ac/src/classloader/mTable.go#L75 https://github.com/platypusguy/jacobin/blob/18c541820cb73bda... I couldn't figure out what happens if you invoke a native method that has not been set up like this.
- andrewbinstock 3y agoAuthor here. No threading currently. Once we have all the bytecodes executing, we will perforce be obliged to add threading. Starting with the heavier threading that Java pre-Loom uses.
- returningfory2 3y agoCan you just use green threads (goroutines) from the get-go? Or, by Hyrum's law, is the fact that legacy Java threads are really OS threads part of Java's de facto public API?
- andrewbinstock 3y agoThat's a good question. I don't know enough about the new green threads to give you a definitive answer, but it's certainly something we'll examine closely.
- jerf 3y agoYou have to distinguish between the implementation and the interface from the Java point of view. In Go, you'd use goroutines. Even if you want to use an OS thread you use a goroutine and then pin it to an OS thread, and there's no reason to do that in this case. However, inside Java it may look like the original Threading API. In that case you'd have a JVM running the Thread abstraction in Java on top of green threads in Go, while not supporting Java green threads, and there's no contradiction or even anything weird going on. Perfectly normal.
- returningfory2 3y agoRight right, but I wonder if there is some weird edge case where some Java program relies on threads being real OS threads...like, for sure Java threads being OS threads is an observable feature of a running Java program (you can see it in htop!). And maybe someone has built a Java based system somewhere that uses that...
- tracker1 3y agoI recall seeing some really hacky Java implementations that used thread references for state management... Mostly, in that the same devs expected the same experience in the C# ASP.Net lifecycle and it lead to a lot of wierd race bug conditions in practice.
- suumcuique 3y agoProject Loom is not included in JDK17.
- bullen 3y agoSo Go GC + Java GC?
- karmakaze 3y agoThe Go GC could be the Java GC.
- perbu 3y agoThen it would be hard to write this in Go. Can't see how you'd be able to hook into the Go GC in an application written in Go.
- nightowl_games 3y agoFrom TFA: > An important factor in reducing the size of the codebase and executable is that Jacobin relies on Go’s built-in memory management to perform garbage collection, and so it contains no GC code. I don't see how you can't see it.
- RetroTechie 3y agoI can see how using Go's GC reduces the codebase. But I don't see how that reduces binary size. Compiled Go programs include a Go runtime in their binary, correct? (statically linked). So if Go program uses Go's builtin GC, that GC wouldn't be part of the program code, but would be part of the runtime that it's packed with in the binary, right? In essence replacing [GC in Go program part of the binary] with [GC in Go runtime part of the binary]. I mean, I can see the advantage(s) there. But reducing binary size doesn't seem to be one of them. Unless you count [Go program using its own GC + the one in Go runtime] vs. [only the one in Go runtime]. It's still a (read: at least 1) GC included in the binary.
- skitter 3y agoYou're right, but that's what's meant - using the Go GC vs also implementing a custom one.
- deleted 3y ago[deleted]
- richieartoul 3y agoThis is so cool. Can’t wait to go digging around through the code this weekend. Good luck with the project!
- richieartoul 3y agoI know the link stresses that correctness and code clarity are the primary goals of this project, but I’m curious if performance is a goal as well? Do you hope that people will run “real” workloads with this or use it to embed other software in their Go applications?
- layer8 3y agoSince they emphasize cohesiveness and clear code, the goal seems to be more on the educational side. It doesn’t look like they’d want to implement JIT bytecode compilation.
- andrewbinstock 3y agoAuthor here: Right now we're entirely focused on getting parity of functionality so that anything that runs on the Hotspot JVM runs similarly on Jacobin. There's still a lot of work to do to get there. However, once we do get there, we'll start working on performance.
- riku_iki 3y agowhat is the reason for this project to be started? What is the end goal? Why current JVM is not good enough?..
- andrewbinstock 3y agoThanks for asking. I've always thought of the JVM as magical technlogy--I'm certainly not alone in that view. But in trying to learn more about it, I was greatly frustrated by the difficulty of reading the code base. As you likely know, the Hotspot JVM is open source. But reading the code is very difficult, in part because it grew organically and in part because of its unusual design, in which many actions are buried deep at the end of a long series of function calls involving unexpected classes and unusual methods, etc. This led me to thinking there would be value in a JVM written as a single cohesive codebase. And given that there is a 300+ page JVM specification and a reference implementation, I thought to myself, how long could this take? Two years later, and with help from two major contributors, we're still finding out! ;-) Eventually, we hope, it will be a fun/interesting experience for users to pop open their Go IDE and watch a Java program execute--which is why we're intent on making sure it's written in 100% go. In a larger context, Jacobin might eventually be useful as an embeddable JVM.
- caeril 3y ago[flagged]
- tombert 3y agoI would be very curious to side by side performance benchmarks between this, GraalVM, and vanilla JDK. My gut tells me (with no data to back this us) that the vanilla JVM will inch ahead once it's paid the cost of starting up but I would be interested to see how wrong I am.
- twic 3y agoThis JVM has no JIT, so OpenJDK and Graal will rather more than inch past it. It's not even a given that it will start up faster, since i don't believe it has been optimised nearly as hard.
- pg_1234 3y agoYou'd probably get better results writing a Go compiler in Java ...
- cfiggers 3y agoWrote a Go compiler in Java, used it to compile this JVM implementation written in Go
- seanw444 3y agoOuroboros
- KRAKRISMOTT 3y agoLambda the ultimate
- rizky05 3y ago[dead]
- paulddraper 3y agoGreat, now just use this JVM to run your compiler :)
- baryphonic 3y agoI had no idea classes were allowed in a Jacobin system
- joeatwork 3y agoslow clap
- deleted 3y ago[deleted]
- paulddraper 3y agoGood job sir, now please leave
- baryphonic 3y agoTo the guillotine? Talk about mark-and-sweep garbage collection.
- yaqubroli 3y agoInitially I was very confused; why would a socialist magazine have an article about the JVM?
- lambda_garden 3y agoThey use Haskell in any case [0] [0] https://jacobin.com/2022/05/jacobin-is-looking-for-a-programming-fellow/ https://jacobin.com/2022/05/jacobin-is-looking-for-a-program...
- andrewbinstock 3y agoAuthor here: Long ago, when I got the domain, the project was going to be JAva COmpiler to BINary = Jacobin. And as the Jacobins were revolutionaries, it sort of fit the intended project. I don't recall whether Jacobin Magazine was in publication then or not TBH.
- paulddraper 3y agoReally? Wouldn't your initial thought be that 18th century revolutionaries created a JVM?
- philipov 3y agoI heard the project repository suffers from a detached head.
- toastercat 3y agoheh
- sbdaman 3y agoInteresting name choice
- mdaniel 3y agoWhile they seem to be "git commit && exit", there are sibling repos that are interesting, too: https://github.com/platypusguy/jacobin-dart https://github.com/platypusguy/jacobin-dart (A JVM written in Dart) https://github.com/platypusguy/jacobin-swift https://github.com/platypusguy/jacobin-swift (Jacobin JVM written in Swift)
- andrewbinstock 3y agoAuthor here: Yeah, those were first attempts in other languages.
- gabereiser 3y agoUmmm, excuse me, but where the f&$k has this been hiding? I’ve been looking for ways to extend my go applications with scripting support. I started with Lua (worked ) then Python (worked but hacky) then javascript using otto [1]. However it lacks ES6 support so having pretty OOP js code is a non-starter. I would love to have Java as a runtime that can be executed from goroutines. [1] https://github.com/robertkrimen/otto https://github.com/robertkrimen/otto
- rainingmonkey 3y agoWhat was your experience with Lua if it worked? What were the disadvantages that kept you looking for other extension languages?
- gabereiser 3y agoAdvantages, Lua is old and simple so implementing it was a breeze. Disadvantage, I kept running into scenarios where oop made more sense and extending an interface. I could have done that through convention, but it’s not for me. I need to “enforce” the interface more than just a try catch. That led me to Python. Which also worked and suited that need for interface contracts but it introduced new problems. How can I enforce memory guards and sandboxing and execute parallel Python contexts? I ran into road blocks and gave Otto a try as it’s written in go, I would have more luck making the necessary changes if I needed a feature.
- donio 3y agoNot the parent but there are several high quality native (meaning no CGO) Lua implementations for Go and it's a great choice if you want an embedded scripting language: https://github.com/yuin/gopher-lua https://github.com/yuin/gopher-lua https://github.com/Shopify/go-lua https://github.com/Shopify/go-lua Unless you specifically need a JVM either of these will be a much more practical and mature choice for embedded scripting. Alternatively if you prefer JS then Otto is a good choice: https://github.com/robertkrimen/otto https://github.com/robertkrimen/otto
- 3y ago
- butterisgood 3y agoI can't help but wonder if this means Java might work on Plan 9/9front with this. But I'm also not sure that's something I'll ever need.
- nazgulsenpai 3y ago"I have spent the last eight months researching the JVM--reading the docs and articles and doing exploratory coding in various languages with which to write the Jacobin JVM."[0] I wish I had the discipline to decide I wanted to do create a long-term pet project, spend months researching what kind of project and which programming language to implement it in, and to still be motivated enough to keep actively updating it 2 years later. Also, the code comments are educational, and the write-ups are inspiring. Great stuff. [0] from http://binstock.blogspot.com/2021/08/a-whole-new-project-jvm.html http://binstock.blogspot.com/2021/08/a-whole-new-project-jvm... linked from article
- smokel 3y agoDiscipline gets a lot of attention around here. My suggestion is to find something that piques your interest for a long enough time. Thought around the "Flow" concept [1] suggests to pick something that is just within reach of your capabilities. And perhaps try meditation to exercise building up discipline. It also helps to ask yourself several times per day: "What would be the smartest thing to do now?", until it becomes a habit. You will then accidentally think that phrase, which gives some freedom to override other (possibly) bad habits. [1] https://en.m.wikipedia.org/wiki/Flow_(psychology) https://en.m.wikipedia.org/wiki/Flow_(psychology)
- agumonkey 3y agoAll of these have one problem, they're fickle. Intrinsic motivation, flow.. things most (myself included) seek and love, are not guaranteed. The experience of adjusting difficulty and investment (you feel tired, you have emergencies.. do less, but do something, have rest days), going slow and steady versus inspiration driven is key IMO. What is hard is accepting the near invisible small steps that don't seem to bring you closer to your goal.. that's where motivation dies.
- slashdev 3y agoMotivation is fickle, discipline is about doing the action when you’re not motivated and don’t want to. Motivation comes and goes. Discipline shows up for work every day, rain or shine. I’ve worked on long term projects, and it’s the discipline that lets you power through those tasks you don’t want to do. It’s required to finish anything of substance. One thing that’s really helped me build discipline is the gym. It’s hard, I usually don’t want to go. But even on the days I don’t feel it, I go and get a workout in, even if it’s not a great workout. Even if I just go for a walk in the park or on a treadmill some days. Once you do something consistently enough, it becomes habit, and develops its own inertia, to the point where it can become harder not to do it. I think habit and consistency are important in maintaining discipline.
- steno132 3y agoI once said that "just as there's no founder like Elon, there's no Java expert like Andrew (project author)" Andrew's Java skills are at a different level. I only briefly met him once, but he used to be a core contributor to Oracle's Java magazine. 80% of the "Unix greybeard" type advanced Java knowledge I have is due to his articles. Happy to see he's still building. Thanks Andrew!
- usrnm 3y agoSo, where is Andrew's x/twitter?
- deleted 3y ago[deleted]
- andrewbinstock 3y agoThank you for such kind words!
- AtlasBarfed 3y agoOk why are you doing this? To see if some end around to another language GC is faster?
- cylinder714 3y agoI can see it being used to bootstrap OpenJDK, particularly on platforms other than Windows, macOS or Linux.
- whalesalad 3y agoWill it run clojure?
- crickey 3y agoSo the garbage collected language is written in a garbage collected language? How does that effect the system?
- ternaryoperator 3y agoLead dev here. A significant portion of the JDK and the JVM is written in Java. There is much that happens in the JVM were gc is a good thing. Classes are constantly being instantiated and then discarded. Having those discarded classes GC'd is a benefit indeed.
- rbanffy 3y agoFor a second I wondered why the Jacobin would publish a story about a JVM…
- deleted 3y ago[deleted]
- meisel 3y agoJust curious, what's the purpose of this? Are there any production use cases it's targeting, or is it just for fun?
- andrewbinstock 3y agoLead dev here. I gave a detailed reply earlier. Grep for "I've always thought of the JVM as magical technlogy" on this page and you'll find it.
- meisel 3y agoI grepped and did not find
- ternaryoperator 3y agoHere's the text I believe that you're looking for: > I've always thought of the JVM as magical technlogy--I'm certainly not alone in that view. But in trying to learn more about it, I was greatly frustrated by the difficulty of reading the code base. As you likely know, the Hotspot JVM is open source. But reading the code is very difficult, in part because it grew organically and in part because of its unusual design, in which many actions are buried deep at the end of a long series of function calls involving unexpected classes and unusual methods, etc. > This led me to thinking there would be value in a JVM written as a single cohesive codebase. And given that there is a 300+ page JVM specification and a reference implementation, I thought to myself, how long could this take? Two years later, and with help from two major contributors, we're still finding out! ;-) > Eventually, we hope, it will be a fun/interesting experience for users to pop open their Go IDE and watch a Java program execute--which is why we're intent on making sure it's written in 100% go. > In a larger context, Jacobin might eventually be useful as an embeddable JVM.
- meisel 3y agoThanks!
- kernal 3y ago>The goal is to provide a more-than-minimal implementation of the JVM that can run most class files and JARs As with everything, the final 10% is 90% of the work.
- andrewbinstock 3y agoLead dev here. It's been an amazing amount of work. I'd change the above adage to: Every 10% is 90% of the work!
- ninkendo 3y ago> An important factor in reducing the size of the codebase and executable is that Jacobin relies on Go’s built-in memory management to perform garbage collection, and so it contains no GC code. This breaks my brain thinking about it. A lot of what the JVM does is interpreting/JITing bytecode and ensuring it links/executes correctly, and writing that logic itself in Go is one thing. But how does Go's GC help you garbage collect objects in the JVM you're implementing? For example, you have objects in the JVM heap, tracked by the code you're writing in Go. You need to do a GC run. How does the Go GC know about the objects you're managing in the JVM? Do you just... write wrapper objects in Go around each of them, and code up a destructor so that freeing the Go object frees the tracked JVM object? How do you inform the Go VM about which pointers exist to your JVM objects? I realize I'm in way out of my depth here, and only have a "user"'s understanding of what GC's do, having never implemented one myself, but it seems crazy to me that Go's GC can Just Work with a VM you're writing in Go itself.
- jsd1982 3y agoSince the VM controls allocation of Java objects, just implement the VM to allocate the Java objects into Go's heap using Go's native allocator thereby allowing the native Go GC to clean those up when they become unreferenced.
- ninkendo 3y agoHow would one allocate Java objects using Go’s allocator, as a program written in Go? Does go provide such primitives? Naively, something like: // Called when the hosted JVM code wants to allocate func makeJVMObject() (jObj, error) { var obj = new(jObj) // on go’s heap // do stuff return obj, nil } would make sense, except how do we keep track of who’s referencing it? JVM objects have fields which tell the GC how to crawl the object tree in the mark phase (and so do Go objects), but how do we make the Go GC aware of the fields the JVM knows about? A map maybe? Hmm, I guess a map could work… the jObj struct could have a map of fields it knows about, keys being the field name and values being where they point to… Now that I think of it this probably must be how all GC’s work, they can’t rely on static information to know the fields of each type they’ve compiled, it’s gotta be something like a map somewhere. I guess I may have answered my own question here.
- peter_d_sherman 3y agoVery cool! Hey, you know a feature which I would love to see (and maybe it's already in there) -- would be the ability to "orchestrate" Java code. In other words, to be able to add external event hooks from functions/procedures/methods -- at runtime... These event hooks, when encountered, would be able to do such things as: * Print/pipe a debug message to an external program or log or log viewer; * The debug printer should contain the function/procedure/method name, parameter names, and parameter values automatically (there should be functionality to have them in both the invocation and after the function/procedure/method's code is complete, prior to returning to the caller, so two possible places per function/procedure/method...); * The ability to selectively turn on/off such events at runtime; * The ability to add additional code which could be evaluated for every such event (also at runtime), and make the determination if the event should be processed or skipped; * The ability to do all of the above programmatically, via API... * Some sort of GUI which automatically imports all function/procedure/method names of a running Java program at runtime, then gives the user the ability to track/log whichever ones they want, by simply selecting them to a secondary listbox of tracked function/procedure/methods... Now, maybe some or all of that -- is already baked in there. I didn't look at the codebase long enough to know... But if it is in there, that's awesome! And if one or more of those features are missing, then maybe a future version maintainer or forked version maintainer might be persuaded to add them in there... (Side note: It would be nice to have the above functionality for all programming languages!) Anyway, as said previously, looks very cool!
- andrewbinstock 3y agoLead dev here. Some very cool ideas in there. Will copy/paste this into our task tracker on YouTrack for further consideration. Much appreciated.
- mike_hearn 3y agoThat sounds like what the Java world calls aspect-oriented programming. Also, GraalVM/Truffle has that sort of ability today, supported for any Truffle language: https://www.graalvm.org/latest/tools/graalvm-insight/ https://www.graalvm.org/latest/tools/graalvm-insight/
- jsd1982 3y agoDoes "capable of running Java 17 classes" imply Java >= 17 or Java <= 17? I tried to run a Java 11 jar on my M1 Mac with $JAVA_HOME pointed at a temurin-11.0.20 JVM but no such luck: $ ./jacobin -jar my.jar Class Format Error: Class has two package names: apple/security and com/sun/crypto/provider detected by file: cpParser.go, line: 241 ParseAndPostClass: error parsing classes/module-info.class. Exiting. Class Format Error: Invalid access flags of MethodParameters attribute #1 in main detected by file: methodParser.go, line: 333 Class Format Error: detected by file: methodParser.go, line: 109 ParseAndPostClass: error parsing my.Main. Exiting.
- andrewbinstock 3y agoThanks for your note. The package notes on Jacobin say that we strongly discourage folks from running it in its present form. There are enough features still to be implemented, that for anything but trivial classes, you won't have a good experience. TBH, we're about a year out (we think) from having a version we can solicit users to test. Nonetheless, if you'd be kind enough to post the above error and the class you used into the GitHub Issues tracker [0], we'll definitely include it in our test suite and make sure whatever the problem is, it'll be corrected. [0] https://github.com/platypusguy/jacobin/issues https://github.com/platypusguy/jacobin/issues
- djha-skin 3y agoThe real question is, can it run clojure JARS and does that improve clojure start time?
- mohitgangrade 3y agoI don't know much about this project, or about JVMs in general. But could this run Minecraft? :)