7 ms·
Project started to run Erlang on bare metal
- gruseom 14y agoIt's an exciting idea. Why should server software run on OS's designed to let you play Tetris or compose emails at the same time? It's not like you get all that for free. In theory, specialized apps have a lot to gain from running on more specialized platforms. There ought to be more experimentation in this area. The name "Kerlnel" is going to be a problem, because it's so awkward to say out loud. You don't want your name to be difficult for people to say; you want them saying it. "Kerlnel" is a clever pun but that doesn't last beyond the first time you read it.
- halvsjur 14y ago> specialized apps have a lot to gain from running on more specialized platforms Guess there could be some gains, but a lot? Enough to bother with? What about hardware support?
- wmf 14y agoIn many such projects the hardware support is handled by a hypervisor. But that's more like bare virtual metal.
- gruseom 14y agoWhat about hardware support? Quite so, hardware is the holy grail. (Alan Kay: "People who are really serious about software should make their own hardware.") I dream of a golden age of experimentation in vertical stacks: specialized hardware designed for specialized classes of application with only so much OS as is needed to support them. Perhaps if the cost of developing hardware falls the way the cost of developing software did, we might see something. Why not an Erlang machine? A Lua machine? A spreadsheet machine? Enough to bother with? Order of magnitude is table stakes for interesting, wouldn't you say? Radical experiments demand radical gains. Surely there is room for an order of magnitude if one is willing to sacrifice general-purpose computing.
- wmf 14y agoOrder of magnitude performance improvement isn't going to be possible. That basically requires that over 90% of your cycles are currently being wasted by the OS somehow. Maybe this project could get 20% improvement.
- doktrin 14y ago> Order of magnitude performance improvement isn't going to be possible. I think the term 'order of magnitude' has started taking on a connotation of essentially meaning 'a lot'. It's a fair observation, but I hear it bandied about so often that I rarely actually think the parties are in fact using it literally.
- wmf 14y agoIn this case I don't even think 2x is possible.
- cpleppert 14y agoIt depends on how efficient/inefficient the OS's network stack and data transfer to user space is. For managed runtimes in a VM taking advantage of zero copy APIs is a challenge. I don't think 'order of magnitude' is possible but clearly there are a lot of cases where if implemented correctly this idea could dramatically improve performance.
- gruseom 14y agoPretty sure you may assume that people here know what "order of magnitude" means.
- gruseom 14y agoYou're talking about the OP's project and I was not – at least not when I brought up orders of magnitude. The confusion is my fault. I implicitly changed the subject to my own fantasy tangent. My point is that if one is going to build a narrow vertical stack up from specialized hardware, there had better be a 10x advantage over running the application the ordinary way or the experiment becomes a why-bother. Also, the application had better be valuable enough to justify the effort. This vision of systems design has been alive in the Forth community for a long time – maybe not the "iterating on hardware as part of application development" part, but certainly the specialized vertical stack idea, just in a very austere form. They make the tradeoff of dramatically reducing what the software will do in order to make it feasible to develop that way. That's a tradeoff most of us aren't willing to make. But I have a feeling there are more options if one is talking strictly about servers.
- newobj 14y agoGlad I'm not the only one who immediately balked at the name. I just spent like the last two minutes repeating it in a trance to see if it would become any easier to say. Nope, sounds like I'm trying to say "kernel" while balancing a grape on the back of my tongue. It's a physically uncomfortable word to say, which is an impressive linguistic construction!
- andrewflnr 14y agoSeems kernerl would have been better.
- dllthomas 14y agoI'd say "curl-null", more or less. Not too bad I think.
- TheAmazingIdiot 14y agoIt's better than Kernelang. You could get frowny faces from HR with comments like "Can you start the server Kernelang, Gus?" Yeah. That was bad. I'll go back to lurking.
- tzs 14y ago> The name "Kerlnel" is going to be a problem, because it's so awkward to say out loud It sounds pretty good out loud if you say it while doing a Stephen Hawking impression.
- pi18n 14y agoHow about Kerlang? Pronounced by adding a k sound to the front of Erlang.
- mrmekon 14y agoAs far as I can tell, it's a vague concept, and two "authors" who haven't authored anything asking people to help without really explaining any of the goals? Help with what, exactly? I fully support making more languages embeddable, but this project doesn't seem to be real. I hope there's more to it than what's listed on their site right now!
- abraininavat 14y agoWould what's proposed be considered an "exokernel"? EDIT: To add some context, would it be like http://www.openmirage.org/ http://www.openmirage.org/ ?
- wmf 14y agoIt has been suggested that running an app directly in a VM with no OS uses the hypervisor like an exokernel.
- andrewflnr 14y agoI was just going to post the Wikipedia link (http://en.wikipedia.org/wiki/Exokernel http://en.wikipedia.org/wiki/Exokernel), but Mirage looks pretty cool. I've liked the idea of an exokernel since I heard of them. Thanks.
- thirsteh 14y agoRelated: http://erlangonxen.org/ http://erlangonxen.org/
- tapiwa 14y agoI have been following this project, and it makes so much more sense to me. As another poster mentioned, going all the way down to the bare metal does not make sense to me. Or maybe I have lived through too many hardware upfrades to not want to worry about whether the APP will run if we upgrade the RAID controller or something.
- dioptre 14y agoWhen Darren and I were looking at baremetal OS - we liked it because we could quantify what Xen and other kernels were missing - generic drivers - and we saw we could improve everything by making a system more specialized and what we wanted to use (think about what you actually put a commercial system on, there's not too much choice - especially for HPC). An example of this would be an athlete like a runner, and a shot-putter. Think of their body shapes - they are very different! Yet we all treat operating systems as if they are the same (for compatibility). Being more specialist is obviously not for everyone! Linux (or any other OS) is... Regarding compatibility - my personal long term aspiration is to abstract the HW through LLVM which will eventually allow us to target Xen or other exokernel systems or run more close to the metal by running/developing a true erlang kernel through a mixed ERTS/BMOS (Baremetal OS) code-base. As mentioned in earlier comments, erlang already has a pretty mean scheduler, and memory management system. Personally as an academic exercise - I'd like to see how this operates at a kernel level and in the future, I'll be spending more time on this. All other existing operating systems I personally believe are just getting in the way of best possible performance. Finally erlangonxen which you mentioned above is not open source. That does not make sense to me, and it does not give me what I want... Doing this is a very selfish exercise, as it's what I want. Hopefully other people will want (and make sense of) this too, but atm I'm not fussed. Cheers Andrew
- quasque 14y agoIt would be interesting to see benchmarks to indicate if such a project would give a useful gain in performance, compared to say, just running Erlang on its own, via of the many available OS kernels.
- jff 14y agoDon't announce your project until you at least have something going. The empty repo at https://github.com/kerlnel/kerlnel https://github.com/kerlnel/kerlnel ? That should at least have enough code in it to boot something, even if it's just a "hello world!" that doesn't have anything to do with Erlang. Getting a build system in place and developing some sort of contribution guidelines are also important. There are major advantages to keeping your grand projects in stealth mode for a while. For one, it's good to wait until you have instructions for even the dumbest ADD teenager to compile and run the thing, or else you'll have to deal with a flood of endless terrible questions. For another, it doesn't exactly inspire confidence to come out and say, "Hey, we want to run Erlang on bare metal, we have README.md, please contribute!" I fully acknowledge that the creators intended to wait longer and that bandris may have just come across an interesting looking Github repo, but even that proves a point: Your repo should be private until you have something to show. When Linus announced Linux, he already had code running in userspace, with bash and gcc ported! Edit: I say this having been involved in projects that announced too soon. It's important to establish something concrete first.
- SparrowOS 14y agoIt's not God's official temple. God says... uncultivated acute unintelligible commit justice saving volumes nilled sometime substantial balneum thirdly accomplish ready piece ceased pronounced done humility ---- The Internet archive shows mine was 64-bit before bare metal existed. Plus God will speak. Don't fuck with me. God says... forsooth smack_some_sense_into_you Homer's excessive stays thunder thunderest None slavish clingeth depraved garden ---- I'm in a CIA prison -- I cannot be sued. I should be so lucky. When God breaks me out, I can demand anything that God vouches for, like www.temple.org God says... fall opposed pitiful limed org/gutenberg/etext02 male oft-times China worshipping Shhh cutting recur differ joys right-doing fornication reconcilement status Coeternal you're_so_screwed grown set stopped 97 inspiration reasons reverence evened ANY filthy cogito spider's intemperance Jerusalem chastenest mistaken hope supplies chant declare water-brooks showedst garlands funding community trademark vast Tunisia ftp 96 accord geek clearness wholly unutterably can't initiatory skills known current surpassed consequences wretchedness stunning jobs incorrect fearfully I'm_done Scotland Moreover wine-bibbing mingling deservedly
- neverm0re 14y agoI prefer language agnosticism. Work being done by Nix-OS, a fork of Plan 9, is moving into similar spaces by merely offering you ways of getting direct access to CPU cores for HPC loads: http://code.google.com/p/nix-os/ http://code.google.com/p/nix-os/ It also has the benefit of having actually existing code right now.
- jff 14y agoThat repo is extremely out of date. The lsub.org guys (http://lsub.org/ls/nix.html http://lsub.org/ls/nix.html) decided they'd prefer to roll their own "code review" system, and the nix-os repo was essentially abandoned. Apparently they've recently decided to stop using their own code review thing, and just make changes directly in their local tree while taking emailed diffs from everyone else, so there's that. Edit: oh, and Erik Quanstrom has started shipping the Nix kernel with his own set of patches in the 9atom distribution, if you can handle the hour+ download time for his ISO. There are some other nice improvements in 9atom, too.
- andrewflnr 14y agoNot to be confused with "NixOS", no dash, a Linux distro based on the Nix package manager. Sheesh, how many annoying names can we get into one thread? http://nixos.org/nixos/ http://nixos.org/nixos/
- jlarocco 14y agoOkay, I understand developers can spend their time doing whatever they want. Nobody can (or even should) tell other people how they're going to spend their spare time. But... wouldn't the effort be better spent improving the existing interpreter? The vague description makes it sound like they're trying to run Erlang without an OS "getting in the way". Looking at the language shootout benchmarks [1], it seems like there's still a lot of room for improvement before the OS would be slowing them down. Operating system services aren't causing Erlang to take 3-25x as long as the C code. [1] http://benchmarksgame.alioth.debian.org/u32/benchmark.php?test=all&lang=hipe&lang2=gcc http://benchmarksgame.alioth.debian.org/u32/benchmark.php?te...
- mrmekon 14y agoWe are completely surrounded by embedded devices, and their numbers are growing exponentially. Everything has a computer in it. You might be shocked to know the power that's in some of these devices... your cable box might be a quad-core 1.2GHz ARM. Currently, your common options for embedded software are: * bare-metal with C, C++, or ASM * load a full OS (i.e. Linux) and run an interpreter to get access to higher-level languages There are a lot of domains where the problem being solved doesn't require particularly high hardware resources, but requires quite advanced software design. Implementing something like a modern Blu-Ray player's UI and high-bandwidth/low-latency AV in C is rough. You spend a LOT up front on software development costs, and pay more in the maintenance phase. Alternatively, you can save on development and maintenance by running an OS and using high-level languages and existing libraries... but then your board price per unit skyrockets because of the extra processing power, RAM, and flash that you need to support the OS and interpreters. Access to a modern programming language at a "bare-metal" environment is the holy grain for embedded. It would change the world. Unfortunately, it has always been practically impossible. And likely still is.
- kabdib 14y agoOther options: FORTH. Icky, but I thought I'd mention it; very fast bringup time . . . but then it's FORTH. A commercial RTOS. There are lots of these, some of them are even pretty good. We went the C-and-ASM route and ported over some USB drivers from another in-house OS. Worked okay. USB is horrible to work with, and that was actually the hardest part to get right on our system.
- ricardobeat 14y agoWouldn't it be more feasible to make the Erlang VM run on Baremetal OS[1], and contribute to it? Thousands of hours will be spent on kernel development and hardware issues for what is mostly duplicated effort. [1] http://www.returninfinity.com/baremetal.html http://www.returninfinity.com/baremetal.html
- darrenmackay 14y agoPorting the erlangvm to bmos is what I am currently working on - llvm optimisations or compiling erlang to native code will come later. Cheers, Darren
- darrenmackay 14y agoRunning erlang on bare metal has been a side project of my thesis for a while. I love erlang, and I also love getting close to the hardware, which is perhaps why the Ian Seyler's ReturnInfinity Baremetal-OS has drawn my attention and efforts over the past 8 months or so. Over this time, my colleague Andrew has been tinkering also, and we seem to have a fair degree of overlap in our end goals, hence kERLnel came about. It is great to see a lot of excellent discussion, both positive and negative. I think all open discussion is useful, even negative comments have a positive affect in their own way. Although I admit I am surprised that kERLnel was even mentioned here, particularly, as we haven't actually dropped any code publicly yet ;-) There are other exokernel projects out there that may also be suitable also. In fact, not just exokernels, but also a number of {nano|pico|micro} kernels too, however, the idea of hand coded assembler kind of grabbed me completely for some reason. For me, the project is not about whether X or faster than Y, or anything like that. It is about what I like doing myself and what I find interesting. I think its great for some many people to comment and have different views, particularly different views to myself - this makes the world a better place! Cheers, Darren