13 ms·
Building a Minimalistic Virtual Machine
- jart 4y ago> or that I should base my system on an existing processor architecture and work on something like Justine Tunney’s blink instead. If she did then she'd be very welcome as a Blink developer.
- maxime_cb 4y agoYou seem to be doing just fine without me :) Blink is a very impressive project. Mad props.
- fdaryfdyfgd 4y agoi use qemu for that. bypass all speed and virtualization improvements via flags and you have a rather isolated system
- donnowhy 4y agoI have not ever studied POSIX academically (curricularly) but: > a small set of minimalistic APIs that remain stable over time but from what I've gathered (over years, by 'osmosis'). Isn't this the essential idea behind POSIX?
- maxime_cb 4y agoYes, but POSIX doesn't provide APIs for graphics or audio, for example, even though these things are necessary to build a lot of end-user software.
- euclaise 4y agoReminds me of the Dis VM and Inferno OS
- narag 4y agoVery interesting for embedded or kiosk. Or in general, if the VM is run as a server inside a host: encryption in a box, instead of a library. Curious about what kind of multitask will be added.
- maxime_cb 4y agoI could actually use some feedback when it comes to the design of the parallelism model for UVM. I have a few ideas but it's not my area of expertise, so I would welcome feedback and suggestions.
- narag 4y agoI have experience as a user, implementation is way over my head. Depending on the minimum host supported, you could only use lightweight threads or write a scheduler. IIRC, JVM started using its own thread implementation and later used the one provided by the host OS, when available. Best of luck!
- maxime_cb 4y agoAt the moment I have an event-driven system where you can set up callbacks and timers. It's not green threads but it makes it easy to have multiple different update events running at different rates, for example: https://github.com/maximecb/uvm/blob/main/ncc/examples/attackers.c#L326 https://github.com/maximecb/uvm/blob/main/ncc/examples/attac...
- chc4 4y agoFor parallelism I'd expect something like Cilk (or Rayon, which was inspired by it), where there's some syscalls for spawning and then joining on tasks, but the VM handles starting and managing the thread pool. For concurrency it's a lot harder to make a good interface
- maxime_cb 4y ago> For concurrency it's a lot harder to make a good interface Can you elaborate?
- efficax 4y agothe jvm already exists and you can run ancient compiled class files with it.
- maxime_cb 4y agoThe level of cynicism on HN is sometimes really depressing. > the jvm already exists and you can run [some] ancient compiled class files with it [but many won't work correctly]. FTFY.
- UncleEntity 4y agoI’ve been toying with the idea of taking an ancient bootstrapping JVM implementation, forget which now dead project it was written for, dusting it off and doing something similar to your project. None of these new millennium fancified features — compute like it’s 1996. Though I’m mostly interested in compiler tech, as an amateur, and don’t really want to delve too deeply into VM theory. Time is the main factor holding me back though…sigh.
- maxime_cb 4y agoIf you want to compute like it's 1986, I think it would be fun to build a BASIC interpreter for UVM. Bonus points if you do your own text rendering on a blue background and you add simple primitives for 2D graphics. https://github.com/maximecb/uvm/issues/7 https://github.com/maximecb/uvm/issues/7
- packetlost 4y agook, but not everything wants the baggage that comes with the JVM.
- eatonphil 4y agoNice idea! Some of the goals remind me of CHIP-8 [0] ("the hello world of emulators" [1]) but presumably not constrained to 8 bits and with a bit more than just the ISA and graphics support? [0] http://www.cs.columbia.edu/~sedwards/classes/2016/4840-spring/designs/Chip8.pdf http://www.cs.columbia.edu/~sedwards/classes/2016/4840-sprin... [1] https://www.reddit.com/r/EmuDev/comments/9ezrvt/easiest_emulator_project_to_learn_emulator/ https://www.reddit.com/r/EmuDev/comments/9ezrvt/easiest_emul...
- carapace 4y agoSee also Turing Drawings http://maximecb.github.io/Turing-Drawings/ http://maximecb.github.io/Turing-Drawings/ (One of my favorite things on the Internet!)
- syntheweave 4y agoI don't think it's a bad idea, but having looked into this sort of thing pretty deeply, I do think it's a "now you have two problems" idea when started by centering an ideal VM. Foundational software is very slippery stuff - it's analogous to having a yeast culture. Before the software was there we had machine language and the specific hardware, but gradually we built a stack of compiler tech, common protocols, etc. And from there we went and started targeting the end of further abstracting it, making it perfectly portable and so forth, but that's like trying to engineer a perfect yeast: while some might get a better benchmark on some metric, the overall metric is a pass/fail: "Can I bake bread with this?" Of course, a lot of people give up along the way and buy something off the shelf, because they want bread now. And that's fine - they're eating off it, and the world keeps turning. But if you do try, you can "do more with less" by carefully limiting the kinds of computation and I/O you're doing and describing that as a protocol, then developing software around the protocol. At that point, the software's specification has an insulation barrier from its dependencies, and you don't necessarily care about the whole of the dependency because you've selected a subset whose specific features and codepaths can be vetted in more depth. And the odd thing about this is that it's the kind of thing that, at its outset, doesn't lead towards complexity, because you intentionally started with a system that does less than its dependent parts: but once you have that, you can knock out and replace the dependencies to "do one thing well," ship of Theseus style. At that point it can become larger again, and evolve. And that's roughly how we ended up with the giant soup of stuff that is "Web tech". At every point along the way, it did bake bread. But when it was first specified, it hardly did anything at all: it bundled "download remote files, present files as documents".
- carapace 4y agoThis is fantastic! Just what the doctor ordered!
- imtringued 4y agoI think this is a dead end. The world doesn't need beautiful soultions, it needs ugly solutions that work tomorrow.
- shard_rocker6 4y ago[dead]
- slay_hearts7 4y ago[dead]
- lion_ankles6 4y ago[dead]
- peez_weez9 4y ago[dead]
- steel_rebel6 4y ago[dead]
- heart_stealer1 4y ago[dead]
- shadow_lord3 4y ago[dead]
- wolf_gang7 4y ago[dead]
- crow_srows9 4y ago[dead]
- SasukeUchiha1 4y ago[dead]
- l_theanine 4y agoArguably more interesting is UXN: https://100r.co/site/uxn.html https://100r.co/site/uxn.html A small personal computing stack, with a plethora of examples ready to go. Built by two they/them hackers who live on a boat and basically have bootstrapped everything about their vessel, their computing, their engineering, etc. It's very, VERY old-school hacker-y. They would've made excellent phreaks back in the good ol' days.
- throwaway290 4y agoI was about to mention uxn. It's almost as if people with enough motivation to build and use this are also people who will never agree to use and contribute to one another's work. Not to say any of this work is a waste of course
- entaloneralie 4y agoCreator of Uxn here, Maxime is a dear friend of mine and we've jammed extensively on the design of UVM together.
- deckard1 4y agowelcome to the world of Forth/Lisp It is somewhat sadly ironic that to preserve software one has to actually create software that other people care about preserving. And, as we've seen with emulators, if you have software people care about then there are no roadblocks that will stand in the way of someone figuring out how to get your ancient code to run. I take the whole 100R thing as more an interesting art project than a serious goal. More of a "what if". With that said, I feel most devs are the same. How often do you come across a lead or senior dev that becomes passive aggressive over people modifying "their" code? Quite often, in my experience. Any code that I didn't write sucks and any code I have to touch that I didn't write is the worst code ever written. Code that I didn't write is old obsolete dog turds and code I wrote is "modern". Every dev slaps "modern" on their open source project today.
- maxime_cb 4y agoAuthor here. The creator of UXN is a friend of mine and we chat semi-regularly about our VMs. I have a lot of respect for uxn and credit it as an inspiration, but the goals of each project are different. UXN is a 16-bit system with 64KB of RAM accessible. It will also probably always remain interpreted. These design restrictions are seen as tools to foster creativity. UVM is a 32/64-bit VM. It's currently interpreted, but I've designed the instruction set with JIT compilation in mind. I have a PhD in compiler design and I'm fairly confident that I can make a fast JIT for UVM in a relatively short amount of time, when I feel the design is mature/stable enough. At the moment, UVM is relatively immature, but I want it to be a small/minimalistic VM that you can still build "real" or modern software in that takes good advantage of the capabilities and performance of your machine. Another difference is that IMO, UVM is more approachable. UXN's assembly language is fairly esoteric IMO. It doesn't look like any other assembly language I've ever seen. That doesn't make it bad, but it does potentially make it harder to learn and harder to leverage an existing base of programming skills. UVM's assembly is designed to not be surprising if you've ever programmed in assembly and know the basic ideas about how a stack machine works. I also have a WIP C compiler that's already usable to write simple programs. See my little snake game for a fun toy example: https://github.com/maximecb/uvm/blob/main/ncc/examples/snake.c https://github.com/maximecb/uvm/blob/main/ncc/examples/snake... Assembly syntax example: https://github.com/maximecb/uvm/blob/main/vm/examples/factorial.asm https://github.com/maximecb/uvm/blob/main/vm/examples/factor... I'll point to the fact that there is almost no boilerplate necessary to start drawing some pixels on a 2D canvas, which IMO makes it a fun platform to develop for. Like I said, it's immature, but I'll iron out all the bugs I can find and keep making it better.
- rhn_mk1 4y agoI wonder how this compares to LLVM. That project had building a directly interpretable bytecode and a virtual machine as an initial goal.
- maxime_cb 4y agoLLVM is more heavyweight. Has a lot of analysis and optimization passes for static compilation. UVM is currently very lightweight, will be JIT compiled. Crucially UVM will provide graphics, audio and networking primitives.
- rhn_mk1 4y agoThat's what LLVM is today, but IIRC their goals were the same. It might be worth it to look at what made it turn away from the minimal idea and into the heavyweight that it is now.
- maxime_cb 4y agoIf I had to guess, I would say it probably comes down to wanting to go with static, ahead of time compilation rather than JIT. There are real and valid advantages to AOT compilation. There's downsides with the LLVM approach too. For example, LLVM makes it explicit that they provide zero guarantees when it comes to the stability of their bitcode format. That somewhat limits what you can do with it.
- int_19h 4y agoLLVM also has a strong emphasis on performance and interoperability with existing code and is willing to introduce arch- and platform-specific features for that purpose.
- samsquire 4y agoThis is awesome. Thank you for sharing. My ancient ruby code and nodejs code all broke because I didn't pin dependencies. As a result I've got software that is unrunnable. More software shall be unrunnable as time goes on, I don't know many trends that prevent software from being unbuildable and unrunnable due to change except maybe repeatable builds and hermetic builds. Given platform toolchains complexity and libc versions and complexity of static Vs dynamic linking, I suspect preserving software is very difficult. It seems doing + - ×÷ on numbers is not the difficult part of computers but arranging information into the right places in order to do it. Logistics and package management are difficult to get right. I think Java got something right. Bytecode is longlasting. Can rewrite the JVM for new platforms and architectures. I am writing my own language and it is implemented as an assembly interpreter and a compiler for that interpreter. This lets me get development speedier. What the hard thing I think is more interesting than bytecode or virtual machines is INTEROP. The Amd64 SysV Binary interface of registers for C calling interface and the System call Interface of Linux. Mozilla abandoned XPCOM extensions, part of the reason was performance of the interop between JavaScript and C++. If I could run a virtual machine and interop with modern code that would mean the software was useable for longer.
- whartung 4y agoThe JVM is an interesting use case. Indeed, at a compiled level, byte code is mostly long lasting. I haven’t fired up a 10 year jar file recently but it would not surprise me if it Just Worked. The success of that is twofold. First is simply that whatever changes are being made to the JVM, they’re mostly forward looking and don’t deprecate running code. The other is that the conventional packaging mechanic is, essentially, a static binary. A “fat jar” is the term of art, with all of the dependencies bundled in. But there’s still potential problems. They’ve been removing large subsystems from the JDK as of late. XML, web services, Java FX are poster examples. So legacy binaries depending on those will fail outright. These can be added back to the Java runtime, but still “one more thing”. Of course from the source code side, Java suffers dependency hell and code rot along with the best of them. Network based dependencies up and vanish. Long standing projects may not publish 10 year old jar files any more. Also, Java has had other clods dropped into its churn. Oracle shutting down the java.net website was a huge sudden black hole in the community consciousness of Java. Overnight thousands of articles, blog posts, forum entries, and other artifacts vanished like Keyser Soze. Leaving behind a debris field of dead links across the internet. So, to be fair, the JVM is a boon. I really like Java, and it’s still going strong. The VM architecture and the comprehensive nature of the Java runtime has made the moving of running code across systems much easier. As someone with an enterprise Java background, used to deploying WAR and EAR files, I got to mostly avoid the entire Docker and other such family of infrastructure. Install a JDK, install an App Server, all fairly trivial to isolate, and the system is ready to go. But in the large, it takes more than a VM to get things accomplished. There’s always an eco-system at play. And one primary characteristic of eco-systems is they evolve. Time marches on, and waits for no one.
- kaba0 4y agoHi! I can’t help but see a big similarity to the JVM, both in terms of design goals and the semantics of the byte codes - only “not having dynamic linking” being an exception. Could you expand on what you don’t find sufficient in JVM byte code, when it is arguably one of the best platform for backwards compatibility, has easy support for bringing up a canvas and start painting, etc.
- maxime_cb 4y agoFor one the JVM is a huge piece of software. Large enough that only a large corporation could realistically reimplemented or maintain it. It also exposes many APIs with a large surface area. Then there's the issue of Oracle and how you feel about them as a company. UVM has obviously nowhere near the ecosystem, but you can draw pixels to a frame buffer with two function calls, and your UI will be guaranteed to look the same everywhere.
- kaba0 4y agoJVM is a specification which can be implemented (and has been plenty of times, completely independently of each other) by a single developer in like half a years tops. It is a simple stack-machine with 100+ basic instructions, a simple exception mechanism and a heap. A GC is not even necessary if we are talking about minimalistic approaches, but a basic tracing GC is also not hard. It is only a (huge) plus that it can be run everywhere with top of the line performance thanks to OpenJDK (which is mostly developed by Oracle, but is big enough that an insane amount of companies critically depend on it and several one could single-handedly finance the future of the platform if anything were to happen, which won’t because it has the same license as Linux).
- maxime_cb 4y agoImplementing a good GC is incredibly hard. The JVM may have just "100+ basic instructions", but it also has classes, objects, arrays, and a whole set of APIs it provides. Your JVM is kind of useless if it doesn't ship with all of the user interface primitives (and other APIs/classes) people expect, for instance. Otherwise what you have is not what people expect to find in a JVM. I'm also under the impression that building a good JIT for a JVM would be a massive undertaking. It literally took over a decade for the Sun/Oracle JVM's JIT to become mature enough. I've designed UVM in a way that I believe it will be possible to design a good JIT with relatively little effort