15 ms·
Building a unikernel that runs WebAssembly – part 1
- marcocastignoli 3y agoMolto interessante, complimenti
- deleted 3y ago[deleted]
- pzmarzly 3y agoAs "promised" years ago in Birth & Death of Javascript [0], at some point we shall get a unikernel running a safe GC-collected runtime in kernel-space, at which point we could drop virtual memory mapping support from CPUs, making them faster. While in 2014 the author predicted this will be JS with asm.js, now WASM seems like the way to go. Can't wait (haha)! [0]: https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript https://www.destroyallsoftware.com/talks/the-birth-and-death...
- tralarpa 3y agoJavascript/WASM evolution: designed for applications running in a browser -> writing desktop and server applications -> writing an OS or kernel I can't put my finger on it, but somehow this looks familiar (hint: it starts with a "J", too)
- timschmidt 3y agoWASM manages the trick with a vastly simpler specification and runtime. Not much more than a compile target for other languages.
- meindnoch 3y agoExcept it just got a garbage collector.
- titzer 3y agoWe're working to make sure that there will be an officially blessed subset (called a "profile") that will not require GC.
- AndrewDucker 3y agoI'd be interested in knowing more about that. Is there a summary of current progress anywhere?
- titzer 3y agoThis is the repo for the "profiles" feature by which we will define standard subsets: https://github.com/webassembly/profiles https://github.com/webassembly/profiles
- swsieber 3y agoIt did? I thought that proposal was basically stuck; I haven't checked in a while, but I haven't heard of it moving forward either.
- johncolanduoni 3y agoChrome will be shipping it in the next version: https://chromestatus.com/feature/6062715726462976 https://chromestatus.com/feature/6062715726462976
- davexunit 3y agoWhich makes it an even better compilation target!
- starlevel003 3y agoCan't wait for the Wazelle processor extensions to drop.
- pjmlp 3y ago> The JavaStation was a Network Computer (NC) developed by Sun Microsystems between 1996 and 2000, intended to run only Java applications. https://en.wikipedia.org/wiki/JavaStation https://en.wikipedia.org/wiki/JavaStation
- bheadmaster 3y agoThe earliest implementation (that I know of) of that idea was in 1997. with Inferno OS [0]. One more recent effort that also implements the same idea is the Phantom OS [1]. [0] https://www.vitanuova.com/inferno/ https://www.vitanuova.com/inferno/ [1] http://phantomos.org/ http://phantomos.org/
- jacobgorm 3y agoSee also Singularity from Microsoft Research, using the .net CLR. https://en.wikipedia.org/wiki/Singularity_(operating_system) https://en.wikipedia.org/wiki/Singularity_(operating_system) . In reality, I think there is always going to be a hypervisor to separate the various workloads, and the hypervisor is likely to keep using paging, to support dynamic memory partitioning -- though perhaps with a larger page size, so as to not create too much pressure on the TLB.
- fulafel 3y agoAlso MirageOS, probably the most real-world used unikernel (OCaml based).
- pjmlp 3y agoThe earliest implementation was Burroughs B5500, in 1961, a bytecode OS written in safe systems language (ESPOL shortly thereafter replaced with NEWP), where all hardware operations are exposed via intrinsics, and is one of the first recorded use of explicit unsafe code blocks. The CPUs were microcoded, so the bytecode was for all practical purposes Assembly. https://en.m.wikipedia.org/wiki/Burroughs_Large_Systems https://en.m.wikipedia.org/wiki/Burroughs_Large_Systems
- rcarmo 3y agoWithout an MMU, swapping to disk becomes a sizeable challenge. I don't think WASM (or Java, or any other kind of VM) should assume it has infinite physical resources of any kind, but am not surprised that JS folk are so far away from hardware they will sometimes forget how computers actually work...
- mananaysiempre 3y ago> Without an MMU, swapping to disk becomes a sizeable challenge. Swapping object graphs out to disk (and substituting entry points by swap-in proxies) was a thing in Smalltalk systems, and I expect Lisp machines must have had their own solutions. For that matter, 16-bit Windows could (with great difficulty) swap on an 8086, and other DOS “overlay managers” existed. Not that I like the idea, necessarily, but this one problem is not unsolvable.
- rcarmo 3y agoWell, let's add efficiency to the mix then (I used Smalltalk and LISP machines, and neither managed RAM effectively enough, to the point where emacs was... fast! at the time).
- pjmlp 3y agoAnd in all cases they made use of MMUs to make it perform at an usable speed. I still remember using overlays on Turbo Pascal, Turbo Basic and Clipper. Amiga also didn't had a MMU, and we all "enjoyed" our Guru Meditation momments.
- gmueckl 3y agoSeeing that flashing red rectangle was quite a common sight, I might add.
- crest 3y agoIf you have a GC tracking accesses (not just writes) it could also be used to move seldom referenced objects from memory to disk.
- 3y ago
- dathinab 3y ago> drop virtual memory mapping support the more I think about it the less it makes sense - js engine rely on vmm, and wasm does so, too (in many ways) - close to every non embedding, non trivial program I have seen is in subtle ways based on the assumption of vmm - some vm technology, especially around micro vms uses vmm, too. And Unikernels only really make sense as VMs
- H8crilA 3y agoAlso how would software memory protection (like seen in JVM, JavaScript, Python, ...) be faster than hardware MMU? Hardware simply adds more transistors that run the translation concurrently. Faults are either bugs (segfaults) or features you'd have to reimplement anyways.
- pzmarzly 3y agoThe thing is, now we use and pay the price for both - memory is managed in software, and yet CPU MMU and caches have to sacrifice space on the die for complex memory mappings. Instead we could get extra transistors for better performance (or, like in Apple CPUs, dedicated instructions for GC languages).
- SkiFire13 3y ago> like in Apple CPUs, dedicated instructions for GC languages Could you expand on this?
- pzmarzly 3y agoI was trying to refer to this https://news.ycombinator.com/item?id=25233554 https://news.ycombinator.com/item?id=25233554 (https://threadreaderapp.com/thread/1331735383193903104.html https://threadreaderapp.com/thread/1331735383193903104.html) but I didn't have time to look it up, sorry.
- masklinn 3y ago
- gpderetta 3y agoSingle Address Space OSs have been around forever. Turns out that memory protection is useful even if you are running memory safe code. Also spectre.
- crest 3y agoIf the hardware is designed to support single address space OSs it doesn't have be a security problem. It can help avoid spectre like problems because it can lower the expected overhead of permission checks so far that there is no advantage of speculating on them instead of performing them.
- gpderetta 3y agoI think you are confusing meltdown (a speculation attack on hardware permission checks which was patched in later revisions of intel silicon and never affected other vendors) with Spectre, a general family of attacks on speculative execution, which are generally unsolved. You could of course add dedicated hardware to lower the overhead specifically of memory access permission checks. In fact most CPUs already do, it is called an MMU.
- Aerbil313 3y agoSafe-language OSes like Theseus don't have this class of problems, by their very design. I think it's a superior architecture to current conventional OSes which rely on hardware for protection.
- gpderetta 3y agoHow does Theseus prevents speculation attacks? This page [1] mentions them, but has nothing on how the software prevents them. [1] https://www.theseus-os.com/Theseus/book/design/idea.html https://www.theseus-os.com/Theseus/book/design/idea.html
- Aerbil313 3y agoMy understanding is that conventional OSes rely on hardware to provide kernel and userspace data isolation, while Theseus relies on Rust compiler, as in safe Rust you can't access arbitrary memory locations. Maybe watch the project founder's talk? https://youtu.be/n7r8zO7SodE?si=nswWcFrkTj7K1GpZ https://youtu.be/n7r8zO7SodE?si=nswWcFrkTj7K1GpZ
- crest 3y agoI would like to see a single address space kernel with hardware for permissions and remapping split. This would enable virtually tagged and indexed caches all the way down to the last level cache without risking aliasing. There could be special cases for a handful permissions checks using (base,size) for things like the current stack, largest few code blocks etc. relieving the pressure on the page based permission check hardware which could also run in parallel with cache accesses (just pretty please don't leave observable uarch state changes behind on denied accesses). To support efficient fork() the hardware could differentiate between local and global addresses by xor or add/sub the process identifier into the address if a tag bit is present in the upper address bits. This should move a lot of expensive steps off the critical path to memory without breaking anything userspace software has to do. Add a form of efficient delegation of permissions (e.g. hardware protected capabilities) and you have the building blocks to allow very fast IPC even for large messages.
- hiimkeks 3y ago> drop virtual memory mapping support from CPUs, making them faster. In the video, his argument was that the browsers are single-process anyway, and if everything runs in that process, we don't need that separation. However, since then, we've learned that single-process browsers are a security nightmare, so these days browsers are actually not single-process anymore to provide proper sandboxing. But I love how close to correct that video is, and it's interesting to see in what ways it turned out to be wrong.
- conradev 3y agoDefense-in-depth is always best practice in security. The more layers the attacker has to break and the harder each layer is, the better. All layers can and will be broken. Apple has spent a long time hardening the JavaScriptCore web sandbox to run untrusted code. We’ve come a long way since JailbreakMe’s web-based jailbreak, but ultimately memory safety requires participation from all parts of the stack and JavaScriptCore and V8 are still both written in C++. You can trigger memory-safety vulnerabilities in the host VM using guest code. wasmtime is supposedly a hardened WebAssembly runtime written in Rust, but it’s also a JIT, and I have no idea if anyone has put it through its paces security-wise yet. The idea is that WebAssembly can have JIT-like performance without JIT-like security concerns thanks to a simpler translation layer and minimal runtime. I could see an argument for dropping some layers if the VM isolation become stronger
- fensgrim 3y ago> The more layers the attacker has to break and the harder each layer is, the better. No its not, when it comes to end-user app performance, experience or privacy. Sure, by adding security we can have another reason to let developers end up with golang app compiled to wasm running within electron sandboxed through API redirection (OS + antimalware/antivirus/BPF based EDR) and use it for, like, listening music in a very secure way.. With all these layers happily streaming all kinds of telemetry to knows where, with owning nothing but a bunch of numbers behind a ton of DRM layers, and with no ability to change things to the point where we can't have an app's theme matching system colors because crossplatform compatibility/security reasons. Case 1, firefox: > dom.security.unexpected_system_load_telemetry_enabled > security.app_menu.recordEventTelemetry > security.protectionspopup.recordEventTelemetry > security.certerrors.recordEventTelemetry I don't want to accept developer's assumption that these have to be enabled by default. Case 2, Windows: can't even do a build of a trusted codebase under IntelliJ without antimalware adding up, like, +150% to build time. While IntelliJ (or some of its extensions or plugins that creep up during development) is happily reporting that performance issue back to its masters. Ugly.
- alexvitkov 3y agoYou'd still very much need virtual memory to isolate WASM linear memories of different processes, unless you want to range check every memory access. If we're dropping linear memory and using the new age GC WASM stuff, sure. An exploit to the runtime in such a system obviously would of course be a disaster of upmost proportions, and to have any chance of a decent performance you'd need a very complex (read exploitable) runtime.
- cmrdporcupine 3y agoI suspect the underlying assumption here is that each WASM module/program would/could likely exist in its own unikernel on the hypervisor. Which is something I guess you could do since boot and startup times could be pretty minimal. How you would share state between the two, I'm unclear on, though. The question is.. if you have full isolation and separation of the processes etc... why are you bothering with the WASM now?
- walterbell 3y ago> if you have full isolation and separation of the processes etc... why are you bothering with the WASM now? WASM can help with portability. Any sandbox layer can help with anomaly/exploit/bug detection, accelerating fixes to untrusted code, or a neighboring sandbox layer. "Phrack: Twenty years of Escaping the Java Sandbox" (2018), https://www.exploit-db.com/papers/45517 https://www.exploit-db.com/papers/45517
- cmrdporcupine 3y agoThen we must go deeper! Put some WASM in a JVM in the WASM. In an OS. In a hypervisor.
- walterbell 3y agohaha, today's shiny network effect attractor is tomorrow's legacy quicksand to be abstracted, emulated or deprecated. The addition and deletion of turtles will continue. > Put some WASM in a JVM in the WASM. In an OS. In a hypervisor. Intel TDX comes to mind.
- cmrdporcupine 3y agoVirtual memory and paging isn't just about protection/security/process isolation. It's also about making the most effective use of physical memory -- process virtual usage can exceed process RSS and not just because of swapping -- and providing a set of abstractions for managing memory generally. The OS and the allocator are working together, with the OS having a lot of smarts on machine usage in order to make that Fairly Smart in the general case. So I don't think there's an automatic win in terms of performance by ridding yourself of it. Especially if you're running through the (pretty slow) WASM VM layer anyways. For some applications (e.g. databases), running unikernel or closer to kernel and having direct access to the MMU could be a big win (e.g. see https://github.com/tuhhosg/exmap https://github.com/tuhhosg/exmap & https://github.com/viktorleis/vmcache https://github.com/viktorleis/vmcache & https://www.cs.cit.tum.de/fileadmin/w00cfj/dis/_my_direct_uploads/vmcache.pdf https://www.cs.cit.tum.de/fileadmin/w00cfj/dis/_my_direct_up...). For general applications esp those written to a POSIX standard or making assumptions that the machine they're running on looks like a typical modern day computer? Dubious. You'd end up writing a bunch of what the VMM layer does in user code.
- PaulDavisThe1st 3y agoSome folks in this subthread would benefit with re-acquanting themselves with some old OS research. I am specifically thinking of Opal [0] which differentiates the various roles virtual memory management plays. In Opal, all tasks (processes) share a single 64 bit address space (so you can just share pointers) but hardware provides page-level protection. [0] https://homes.cs.washington.edu/~levy/opal/opal.html https://homes.cs.washington.edu/~levy/opal/opal.html
- deleted 3y ago[deleted]
- colesantiago 3y agoWhat are the use cases of unikernels and WASM?
- quickthrower2 3y agoProbably to expand where you WASM can go: in the browser, in a docker container, and now in a lightweight OS that could go on an embedded device.
- pjmlp 3y agoOne day it can even run on SIM cards and Visa/Mastercard chips!
- H8crilA 3y agoAll the different ways to reinvent the JVM.
- quickthrower2 3y agoTrue but can’t Web Assembly also be non-GC’d?
- pjmlp 3y agoOnly because they are already five years late adding GC support, and even then WASM isn't the first bytecode format supporting C and C++, there are already a couple since 1980's.
- JonChesterfield 3y agoSay a non-OS hacker wants a unikernel. What's the sanest way to go about getting to that? Options that come to mind are: - build your application as a linux kernel module, load it into a normal kernel, and generally ignore the userspace that runs anyway - take Linux and hack it down pretty aggressively plus splice your code into it - find some github unikernel effort and go from there (which I think the OP does) - take some other OS - freebsd? - and similarly hack out parts Other? I like the idea of a x64 machine running a VM connected to a network card as a generic compute resource that does whatever tasks are assigned by sending it data over the network. It's not been worth the hassle relative to a userspace daemon, but one day I may find the time and would be interested in the HN perspective on where best to start the OS level hackery.
- ralls_ebfe 3y agoThere is a framework for OCaml for this: https://mirage.io/ https://mirage.io/ So if you are interested in learning OCaml and want a unikernel, this would be a possible path to take.
- JonChesterfield 3y agoOCaml is a good language but perhaps unikernel does not mean what I thought it did: > fully-standalone, specialised unikernel that runs under a Xen or KVM hypervisor. Or maybe xen / kvm are no longer called operating systems? I'm interested in having my code be responsible for thread scheduling and page tables - no OS layer to syscall into - but am not as keen on DIYing the device drivers to get it talking to the rest of the world.
- walterbell 3y agoMirageOS unikernels run directly on Xen, e.g. http://roscidus.com/blog/blog/2016/01/01/a-unikernel-firewall-for-qubesos/ http://roscidus.com/blog/blog/2016/01/01/a-unikernel-firewal... > I replace the [QubesOS] Linux firewall VM with a MirageOS unikernel. The resulting VM uses safe (bounds-checked, type-checked) OCaml code to process network traffic, uses less than a tenth of the memory of the default FirewallVM, boots several times faster, and should be much simpler to audit or extend. NanoVMs has OSS tools for golang unikernels on multiple hypervisors and cloud platforms, https://nanovms.com/dev/tutorials/running-go-unikernels https://nanovms.com/dev/tutorials/running-go-unikernels
- tetsuhamu 3y ago[flagged]
- crest 3y agoDoes anyone else immediately jump to: https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript https://www.destroyallsoftware.com/talks/the-birth-and-death... ?
- withinboredom 3y agoI literally came here to bring this video up.
- skavi 3y agoYes, and also this old now archived project which had a similar aim: https://github.com/nebulet/nebulet https://github.com/nebulet/nebulet
- milansuk 3y agoNice project! I love WASM. It's designed to be sandboxed and portable from day one. I wish WASM was invented instead of Javascript in the 90s. WASM will eat the world. What I hope most is endurance. There are many programs that we are not able to run anymore. The best examples are probably older games. I hope WASM will change that, although I'm a little bit nervous about adding new features, because simple specs have a higher chance of surviving, but the future of binaries looks exciting.
- galangalalgol 3y agoI naively hope the web bifurcates into sandboxed wasm apps and document content that doesn't even need js, much less wasm. I'm not sure what a middle ground would look like or why I'd want it. But the realist in me knows wasm will eat the document content too, meaning adblockers and reader view are doomed...
- josephg 3y ago> meaning adblockers and reader view are doomed Maybe. As inconvenient as accessibility is, with any luck the need to make web content legible to screen readers will also keep adblockers working. Even with wasm, I don’t think the DOM is going anywhere any time soon. I haven’t seen any proposal to replace it.
- galangalalgol 3y agoYou are probably right. Raster frameworks that talk straight to a gl context are out there, eframe/egui is one I've used. And yeah, accessibility is bad. Pair that with encrypted websockets and webTPM which if it isn't a thing, will be, you won't have any control over the chain between the screen and the server.
- cmrdporcupine 3y agoBelieve it or not, back in the 90s we thought (on the whole) that web browsers were for browsing hypertext documents. Not for replacing the operating system. There's a reason JS started out limited to basic scripting functionality for wiring up e.g. on-click handlers and form validation. That it grew into something else is not indicative of any design fault in JS (tho it has plenty), but with the use it was shoehorned into. The browser as delivery mechanism for the types of things you're talking about is... not what Tim Berners Lee or even Marc Andreesen had in mind? Back then "the network is the computer" people ended up shipping thin X clients: https://en.wikipedia.org/wiki/Network_Computer https://en.wikipedia.org/wiki/Network_Computer in order to do richer applications. I have very mixed feelings about WASM. There is a large... hype-and-novelty screen held up in front of it right now. There are many Bad Things about treating the web browser as nothing more than a viewport for whatever UI designer and SWE language-of-the-wek fantasy is going around. Especially when we get into things like accessibility, screen readers, etc. As for the people treating WASM as the universal VM system outside the browser... Yeah, been down that road 30 years ago, that's what the JVM was supposed to be? But I understand that's not "cool" now, so... Sigh.
- VikingCoder 3y agoI absolutely love this. I also hadn't seen several of the linked technologies before, so I'm bookmarking all of them, too. Next up, I want to configure the hypervisor with a WireGuard connection (possibly through something like Tailscale to establish connections?)... So I have WebAssembly over here on this machine, talking directly to this WebAssembly over there. Based on configuration and capabilities being passed in. Rather than based on the process opening TCP connections to random locations.
- eyberg 3y agoWe did this with wireguard: https://nanovms.com/dev/tutorials/running-nanos-wireguard-vpn-gateway https://nanovms.com/dev/tutorials/running-nanos-wireguard-vp... .
- brundolf 3y agoHow long till we see dedicated WASM hardware?
- ynx 3y agoPedantically - never, because in the strictest sense it is not specified enough for that. But perhaps someone could make a "wasm-but-it's-actually-RISCV-underneath" kinda thing.
- brundolf 3y agoFair, may have been a dumb question :)
- eru 3y agoI am fairly sure someone will make some. Just like we had Lisp machines and even specific JVM CPUs. But my prediction is that those will always stay niche, because running WASM on conventional stock hardware will always be faster in general. Mostly because WASM was designed to run fast on stock hardware, and the economics of scale for conventional general purpose processors are much better. Compare also how the 'International Conference on Functional Programming' started out as the 'Functional Programming and Computer Architecture' conference, but then people figured out how to compile lazy functional programming languages like Haskell to run efficiently on conventional hardware. Similar also for the Lisp and Java machines: one reason see we don't see things like them anymore is because compiler technology has caught up.
- RantyDave 3y agoI'm late to the party but... Has anyone contemplated running Zephyr as a unikernel? https://docs.zephyrproject.org/latest/boards/x86/acrn/doc/index.html https://docs.zephyrproject.org/latest/boards/x86/acrn/doc/in...