6 ms·
I put alot of time and effort into unikernels and the technology is very appealing to me. It has been something of a puzzle as to why they have gotten no tract
by andrewstuart 8y ago
I put alot of time and effort into unikernels and the technology is very appealing to me.
It has been something of a puzzle as to why they have gotten no traction in any significant way.
My conclusion is that there are other technologies (i.e. containers/Docker) that are not as good as Unikernels, but are good enough to solve the core industry demand for lightweight nested virtualization. And indeed containers solve that problem better than unikernels in two important ways which is boot time (and I'm talking here about booting useful unikernel based applications as opposed to theoretical unikernel OS boot times which can be milliseconds), and also containers are better because they don't need nested hardware virtualization switched on in the CPU - that's important obviously for cloud computing where nested virtualization has now arrived but is not widespread.
And good enough is all that is needed.
It's hard for me to imagine a time when unikernels gain more acceptance than they already have, which is basically little more than none, and the staunch backing of operating systems researchers and a few true believers who really hang on to the theoretical benefits.
- rwmj 8y agoKVM runs guests as host processes, so KVM + unikernel = ordinary Unix process.
- cdoxsey 8y agoUnikernels can be difficult to debug. All the tools that come with your operating system are missing. A big chunk of managing distributed systems at scale is trying to figure out what's wrong with them when they don't behave. Most operators will gladly trade some performance overhead for access to a suite of tools they're accustomed too.
- andrewstuart 8y ago>>Unikernels can be difficult to debug. This is an irrelevant objection that seems to get massive traction. Just like some politicians have scathing criticisms of other politicians, which despite being not true, are easily and widely believed. It's not even a relevant point. Nothing I ever did with Unikernels hit this as even an issue on the horizon. If you want to knock down the technology, this is the most effective way to do it because it very effectively frightens off people considering the technology. The origin of the "Unikernels can be difficult to debug." is from someone who works at a company that bet everything on containers. Nuff said.
- kjeetgill 8y agoHonest question: Do you still have the ability to ls, find,grep, netstat, lsof stuff? I don't know if unikernels can be difficult to debug but if you told me I didn't have those I would be nervous at a glance at least. Maybe we've been mislead.
- lkurusa 8y agoMost unikernels are a kernel specifically packaged for a certain application, so the answer would be no.
- pjmlp 8y agoNo, but they again you don't really need them. Remote debugging and cluster monitoring tools like JEE servers provide would already be quite good. Debugging UNIX way is not the only path.
- noselasd 8y agoPeople need to debug their application. If you need to convince people that this is a myth, you need to tell people how to debug and inspect their application and system. If anything your own response here is like the typical politician- avoiding the issue, providing anecdotes that promotes your view, going on the offense and changing the subject.
- andrewstuart 8y ago>> People need to debug their application. I'm not saying applications don't need to be debugged, that would be silly. What I am saying is that this "Unikernels are hard to debug" is a hand wavey way of knocking down the technology, and it has really worked - witness this subthread. The unikernel is the final build, the package, the output of application development and debugging which happens upstream, before you put it into the package. There's always ways you can expose the inner workings of a unikernel to gain insight into whatever might be going wrong for whatever reason - if you really need to do it at the stage that it is a running unikernel - but in most cases its just not necessary because application development and debugging happens, as I say, upstream of the final building of the thing into a unikernel.
- wmf 8y agoYeah, production-quality unikernels would need to have built-in observability, tracing, and debugging support (probably in the libOS) but AFAIK no one has started building such tooling. I suspect that production-quality unikernels would end up re-introducing a lot of the "bloat" that unikernels were supposed to eliminate.
- convolvatron 8y agoyes. idk how bloaty it would be. an x86-64 gdb stub weighs in at < 1000 lines. if one were starting from scratch, a uniform measurement/management interface along the lines of osquery, sysctl, or /proc would be alot more straightforward on both the back and front end than the current weird mix of command line tools that linux presents. I think the most interesting question, as others have mentioned in the thread, is providing decent debugging and introspection in the distributed context. the development of those kinds of tools seems pretty orthogonal to the size of your kernel and whether or not it runs systemd.
- himom 8y agoYup. GNU/Linux has just been reinvented. Another flavor of OS to support equals $$$$.
- jacobush 8y agoIBM OS 360 has just been reinvented. It's turtles all the way down to the 1960s. This is just like nature. We are witnessing something like the evolution of one celled organisms. Also, multicelled organisms have started to appear. That would be cloud services (mats of algae and bacteria) and machine learning (true multicelled organisms).
- perbu 8y agoI've heard this quite a few times. There is really nothing fundamental in keeping you from having the same or better observability in a Unikernel as opposed to a traditional POSIX kernel. A couple of examples. The Gnu Debugger (gdb) supports connecting to qemu-backed virtual machines, so you have the ability to everything that gdb can on a unikernel as well, given the unikernel is implemented in something gdb groks. In addition we've created a trace layer, similar to strace that allows you to run strace inside the vm itself. Since the POSIX libs are just libs wrapping them inside something like strace is rather trivial. Also the unikernels have a huge advantage of having everything in the same memory space, meaning that you can observe the application and operating system at the same time. High level languages like Python or Node.js can typically have debuggers loaded dynamically and as such would be able to offer full debuggability without having to do tricks with gdb. All this doesn't have to induce bloat. The socalled bloat in POSIX systems is mostly caused by the security barriers raised by a design that forces strict separation between unprivileged and privileged access to the system. In addition a POSIX kernel will typically be built to support "everything" that a user might need whereas a unikernel can take a build-time decision on what to include into the OS.
- baybal2 8y ago>Unikernels can be difficult to debug. All the tools that come with your operating system are missing. Do you even need to? If something so simple breaks, why not to use, say, JTAG?
- wmf 8y agoDid Elastic Cloud JTAG come out while I wasn't looking?
- baybal2 8y agoi think giving JTAG access on cloud machines may not be that a bad idea. Connect as low-end sbb to server's JTAG output, and have it connected to the network. JTAG is beautiful tool for debugging because it requires no modification of software being debugged.
- comex 8y agoThat would give the user full control over the physical CPU, though, whereas most "cloud" services only give you access to a virtual machine. On the other hand, it should be possible for the hypervisor to provide roughly the same capabilities in software, without having to modify the software inside the virtual machine. Indeed, most hypervisors already provide a GDB stub for this purpose, including, e.g., Xen [1]; it's just that public clouds don't typically give customers access to it. It would be nice if that changed. [1] https://github.com/xen-project/xen/tree/master/tools/debugger/gdbsx https://github.com/xen-project/xen/tree/master/tools/debugge...
- baybal2 8y agoSomebody's cloud is engineered the wrong way if somebody can wreck the entirety of it when only having access to a single server
- Nullabillity 8y agoUsually the same physical server has multiple tenants, which do not trust each other.
- lbotos 8y agoIf someone builds a really easy way to slap a django app/rails app/node app "into" a unikernel, and an easy way to deploy that with perf gains over VM/Docker, you might get some ears. I think the hardest part is like you say, docker "works well enough" and the pressure isn't on to deliver the unikernel solution.
- andrewstuart 8y ago>> If someone builds a really easy way to slap a django app/rails app/node app "into" a unikernel I solved this in my research, but it's still not enough to get unikernels any real traction because containers have won the battle.
- ingenieroariel 8y agoIf you change django/rails/node with golang/rust statically linked binary would you still get enough ears?
- lbotos 8y agoI don't think so. I think you of have to catch the people who haven't adopted docker yet, and the people who have "aging" codebases ("last gen"). Convince a few of them to convert, see good gains and "easier maintenance", get a groundswell, and go.
- derefr 8y ago
- kjeetgill 8y agoWhat's the use for unikernels you'd like to see considered more? I suspect the lack of traction is the lack of a mission statement: When would I want a unikernel over xyz? What am I missing out on if you go with xyz instead? (e.g. performance, security, ease of deployment, etc.)
- yjftsjthsd-h 8y agoI would expect them to have better security than containers while having a lower resource footprint than normal VMs.
- tlals 8y agoIn a broad stroke, I think we are seeing a progression of the code and the environment merging together and becoming more compact as a result. Unikernels seem to me to be the natural next step after containers. I'm not sold on this, but perhaps the next natural step in the progression after unikernels is serverless architectures. If that is the case, we may never see the widespread use of unikernels. Containers are good enough. The full promise of unikernels has not fully matured. And serverless architectures seem to be gaining some traction. In the end, maybe unikernels just get passed over. That would be shame for me personally. When I read my first blog post on unikernels, some vague thoughts on the boundary between code and environment snapped into place. Unikernels seemed like the only game in town that made perfect sense thinking from first principles. I have been hankering for an opportunity to fit unikernels into a serious project ever since.
- sethev 8y agoI think one of the biggest problems is having to start (almost) from scratch. You can run pretty much any software you want in a container - even if you just have the binary. With a unikernel, you'd have to write the software specifically for the unikernel.
- nwmcsween 8y agoBecause unikernels have many downsides vs an exokernel
- bluejekyll 8y agoI generally have the same thought, and on top of that, things that you take for granted become hard, like now you need some custom thread library if you want threading. I love this project, but I wonder if getting behind something like feL4 might have more interesting payoffs. A really small lightweight kernel, with only the drivers and libraries you need. Running side card debuggers become easier I would think, and make them more debuggable. Redox probably fits in this category as a potential, if less stress tested, option. Then there is nebulet which by using wasm, would remove the system call expense. All of these are exciting, it makes me happy to see people continuing to play around in this area.
- im_down_w_otp 8y agoDon't worry, feL4 and unikernels aren't mutually exclusive.
- perbu 8y agoWhy are unikernels always compared to containers? Unikernels are obviously not meant to solve generic compute needs. They are per definition niche systems that do single, simple tasks. There is a lot of stuff that isn't done in containers or at least isn't done well. Network functions, CPU-based IoT, serverless. Packaging up a complex application in a container and moving it around seems to work quite well. You get to abstract away the core operating system and that is all well and good.
- omginternets 8y agoI'm really not an expert, but it seems to me that unikernels are missing some of the peripheral (yet extremely useful) features that e.g. Docker provides. Namely: - layered filesystem - a full-fledged OS, which is useful for debugging In addition, I haven't seen something comparable to Docker Hub (yet).
- chatmasta 8y agoUnikernels are perfect for stateless applications, especially in networking. Think switches, routers etc. Because any business running those nodes is very bandwidth intensive, it’s unlikely to host in the cloud where transit price gouging makes such high bandwidth models impossible. So we don’t see any tooling for unikernels in the cloud, which inhibits adoption from more mainstream devs.