17 ms·

Arrakis

308 points by conductor 12y ago

npsimons 12y ago
Been done before (some would say to death), and the reason we have memory protection between applications is forgotten because people don't realize how nice it is. Sure, sure, your big "well engineered" web browser needs direct access to the hardware for speed, but painful experience has taught us that giving apps programmers direct access to hardware is a recipe for failure. Besides, there are already plenty of workarounds to get faster (eg, mmap) or even direct access to hardware from userland, not to mention the myriad of virtualization and protection schemes and levels in userland (eg, SELinux). This seems like a solution in search of already solved problems. Although as a research project, it does seem interesting . . .
- rbanffy 12y ago
  > Been done before (some would say to death) How is that George Santayana thing? "Those who cannot remember the past are condemned to repeat it"? I like seeing new ideas being tested in the OS arena. It's a real shame the two dominant OSs these days are Unix and VMS. I refuse to believe these two are the best humans can come up with.
  - mikeash 12y ago
    From "The Rise of 'Worse is Better'", written in 1989: "The good news is that in 1995 we will have a good operating system and programming language; the bad news is that they will be Unix and C++." The predicted date was a little early, but otherwise he pretty much got it right. I think that, from a late 80s Lisp perspective, modern Windows fits into the "UNIX" category, and Java/C#/ObjC/whatever are close enough to C++ to count.
    - trhway 12y ago
      >The predicted date was a little early wouldn't say so - my first Linux machine (slackware, shoebox of diskettes :) i installed exactly in the 95 :) >from a late 80s Lisp perspective, modern Windows fits into the "UNIX" category while psychotropic mind-altering effects of Lisp are well known, i'd doubt that it can produce such perception changes or even in such a direction.
      - mikeash 12y ago
        Windows and UNIX are not so different. Monolithic kernels written in C or C++, permissions largely done with user granularity, byte-addressed memory, virtual memory with per-process address spaces, no hardware support for tagged pointers or garbage collection.... Compared to the variety that's gone before, they look nearly identical in their fundamentals.
  - Executor 12y ago
    Thucydides?
- chadgeidel 12y ago
  I would go even further and say that apps developers don't want to write a networking stack, display driver, roll their own sound implementation, etc. It sounds like the "bad old days" of DOS where games had to have their own implementation of these things.
  - wmf 12y ago
    That's why Arrakis provides that functionality through a library.
- sounds 12y ago
  Good points. Indulge me for a minute while I try to formulate what they might say in response: most of the complexity of an OS is device drivers. You're right, the basics of process isolation, scheduling, and resource allocation are pretty dull. Maybe the UW project makes the move to userspace official? At 1,000,000 packets per second it is more efficient to put the whole network stack in userspace. Assume similar gains can be had for all types of hardware, and shared libraries eliminate the overhead of loading the whole network stack every time you run 'ping'. That's just a 30-second pitch for the idea. Userspace is not new (see: Mach). I think what is changing is the average lifetime of each process. Both desktops and servers are beginning to become a commodity to the point where the old reasons for not using a microkernel are starting to become irrelevant: e.g. message passing cost isn't as much of a problem if the file server runs 'smbd' 99% of the time and that is the only really performance-sensitive application. Likewise for a desktop, except the running process is 'Chrome', or for a tablet running 'dalvikvm'. Really it's a move away from code complexity that wasn't possible before but is becoming possible.
  - npsimons 12y ago
    So are we talking Microkernel here or something else? Sorry to constrain by categorizing, but it's what I do sometimes :) and I am thinking of some OSes (mostly research/academic) that went this way (of no layers between userland and the hardware) that weren't Microkernels, but I can't remember any names right now. I have to admit, part of what keeps my interest in this is that I used to dabble in OSes and I haven't gotten to for quite some time :( EDIT: Exokernel was what I was remembering (I think; https://en.wikipedia.org/wiki/Exokernel https://en.wikipedia.org/wiki/Exokernel). I can see some of the advantages, but as a current maintenance programmer and previous RTLinux hacker, I have to say I have my doubts about giving most programmers direct access to hardware :) That being said, giving programmers (and admins!) easier access to containerization, virtualization, or just intra-process protection in general would be a good idea. As I've said, there are plenty of options right now, but none seem to be well documented or advertised. There was a CACM article a while back that predicted a future in which you had one smartphone, but it had split domains for work and personal apps and data. Interesting stuff, and as you say, finally becoming possible.
- bri3d 12y ago
  Arrakis doesn't eliminate memory protection or process isolation, nor does it give applications direct hardware access. It takes advantage of modern hardware virtualization capabilities to present certain virtual devices to the guest applications, eliminating the kernel from the I/O path for critical operations like receiving a packet. Read this paper: http://arrakis.cs.washington.edu/wp-content/uploads/2013/04/arrakis-tr-ver2.pdf http://arrakis.cs.washington.edu/wp-content/uploads/2013/04/...
peterwwillis 12y ago
If I understand this correctly, they want all userspace applications to be Ring 0? That sounds.... problematic, to put it lightly.
- SEJeff 12y ago
  goto fail means an entirely different thing in that operating system
  - trhway 12y ago
    until your application implements memory protection, etc... ie. their premise is that modern complex applications contains an OS inside it [ i don't fully agree with such premise, though remembering "nspr" state even back in 1999 i'm not surprised that 10+ years later Mozilla dind't have a problem coming up with their own OS, i'm more surprised that it didn't happen much earlier :) ]
- steveklabnik 12y ago
  One might say solving that problem is the entire point of the project.
ChuckMcM 12y ago
Interesting, its a fork of Barrelfish [1] which is the one-core-one-OS OS. When I first heard about it, it sounded like Multi-DOS (several instances of MS-DOS running at once) but its a bit more sophisticated than that :-). Other than cache contention (which is always going to be a problem) its an interesting approach. [1] http://www.barrelfish.org/TN-000-Overview.pdf http://www.barrelfish.org/TN-000-Overview.pdf
akavel 12y ago
Is this the same idea as "exokernel", and thus a (probably valuable) attempt at implementation of one, or does it differ in some important points? https://en.wikipedia.org/wiki/Exokernel https://en.wikipedia.org/wiki/Exokernel
- teraflop 12y ago
  It looks similar. MIT's Exokernel seems to use kernel-level hooks to control access to things like disk blocks and network packets. Arrakis takes a similar approach, but restricts those hooks to a subset that can be implemented in hardware instead of in the kernel.
- nostrademons 12y ago
  It's interesting to see many concepts and general design philosophy of exokernels make their way into modern systems. Zero-copy, mmap, RDMA, vectored I/O, fibers/switchto, FUSE - all of these are attempts to push as many policy decisions into user space as possible so that the OS only deals with securely multiplexing the hardware. The irony is that rather than new kernels, these are being added on as new APIs to the Linux kernel. I suppose that makes a lot of sense because it's much easier to expose a new syscall and see if it gets any adoption rather than convincing everybody to switch to a whole new OS.
teraflop 12y ago
This is pretty interesting, but unfortunately their intro page doesn't do a good job of explaining why. (For instance, they use the word "unprecedented" without really saying what makes Arrakis different from previous exokernel-like designs.) I'll try to summarize what I got out of skimming the paper[1]: When you have multiple applications running on the same machine, you need some way to safely share resources between them; for example, incoming network packets are a resource. A kernel handles this by keeping a data structure mapping sockets to processes, and demultiplexing data that comes in from the network card. Hypervisors work the same way, except at the level of virtual machines rather than processes. Arrakis does the same thing, but relies on hardware support in the network card to dispatch packets to the right process. This relies on a standard called SR-IOV[2] which allows the OS to configure a PCI device to present itself as multiple virtual subdevices. The kernel programs the NIC to dispatch packets to different buffers depending on the incoming MAC address; after that, packets can be dispatched with no kernel involvement at all. Similarly, you can tell a disk controller to present a particular extent of a disk as a new virtual storage device. The blurb about memory protection seems to be a red herring, because as far as I can see they haven't done anything to change that. There's still a kernel, which handles requests for resource mappings, and processes are still isolated from each other. But once they've requested the mappings that they need, the normal execution path doesn't involve any syscalls, and so there's no kernel overhead. The real contribution of the paper is designing an API around this idea and proving that real applications like Redis can be ported to it. [1] http://arrakis.cs.washington.edu/wp-content/uploads/2013/04/arrakis-tr-ver2.pdf http://arrakis.cs.washington.edu/wp-content/uploads/2013/04/... [2] http://blog.scottlowe.org/2009/12/02/what-is-sr-iov/ http://blog.scottlowe.org/2009/12/02/what-is-sr-iov/
- jws 12y ago
  But where does one get SR-IOV devices to experiment with? It looks like the Intel 82576 chipset has SR-IOV, and can be had in a $50 card, two ports, 8 filters per port. The Intel 82599 is a 10Gb chip with more filters per port. (with linux support for the SR-IOV, it can manifest as multiple devices) Disk storage is less obvious. The LSI 2308 and 3008 controller chips probably support it, but I'm not finding a commodity card or a motherboard integrating one. They mention in the paper that existing devices have problems which prevent them from actually securely isolating clients of the sub devices. Combine this with seeing a lot of web activity about SR-IOV in 2009 and not much now. Either it became too common to mention, or is dwindling into an idea that didn't catch on. The wait for secure SR-IOV might be interminable.
pygy_ 12y ago
In reply to many comments about the lack of safety of the approach: They claim that "[they] demonstrate that operating system protection is not contradictory with high performance". Abstract of their latest paper: Recent device hardware trends enable a new approach to the design of network server operating systems. In a traditional operating system, the kernel mediates access to device hardware by server applications, to enforce pro- cess isolation as well as network and disk security. We have designed and implemented a new operating system, Arrakis, that splits the traditional role of the kernel in two. Applications have direct access to virtualized I/O devices, allowing most I/O operations to skip the ker- nel entirely, while the kernel is re-engineered to provide network and disk protection without kernel mediation of every operation. We describe the hardware and software changes needed to take advantage of this new abstraction, and we illustrate its power by showing 2-5x end-to-end latency and 9x throughput improvements for a popular persistent NoSQL store [i.e. Redis] relative to a well-tuned Linux implementation. http://arrakis.cs.washington.edu/wp-content/uploads/2013/04/arrakis-tr-ver2.pdf http://arrakis.cs.washington.edu/wp-content/uploads/2013/04/...
michaelmior 12y ago
If you like this concept, you may also find Mirage[1] interesting. Mirage compiles the application code into the kernel to run directly on the Xen hypervisor. (Thus system calls become ordinary function calls. They do some tricks to maintain security.) [1] http://www.openmirage.org/ http://www.openmirage.org/
GuiA 12y ago
"Applications are becoming so complex that they are miniature operating systems in their own right and are hampered by the existing OS protection model" Sure, that's true for browsers, as they mention, and a few other degenerate cases (eg. virtualization software?) - but that's certainly not the case for the vast majority of applications I run (text editor, terminal, mail client, IM client, etc.). How does this argument hold?
- nekopa 12y ago
  Emacs.
  - magpi3 12y ago
    And let's be honest: VIM has been heading in this direction for a long time as well, especially now that it had a built in python interpreter.
- pyre 12y ago
  It's possible that the operating system on your desktop could take on an entirely different paradigm... E.g.: Hardware | Arrakis | __|__ / | \ Browser | System Services... Wrapper / | \ IM EMail Terminal
  - nealabq 12y ago
    As the OS gets thinner and more like a hypervisor, the apps and app-like VMs get more OS-like. One day the browser might have daemons. I'd guess at the same time all these pieces will continue to become more distributed. Hypervisors and apps will present distributed environments (storage, processing, failover), even more than they do today.
    - stephengillie 12y ago
      At one time, all an Operating System did was load other programs into memory so the processor would run them instead. All of the sandboxing, garbage collection, security, and other features were added to make it more usable.
phillmv 12y ago
Heh, this sounds analogous to what Gary Bernhardt finished "The Birth & Death of Javascript" with: https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript https://www.destroyallsoftware.com/talks/the-birth-and-death...
- thirsteh 12y ago
  Or what a bunch of projects have been doing for years -- Erlang on Xen, HalVM, etc. etc.
leorocky 12y ago
Hrm, I wonder what they were using for their source control before making it available on GitHub, they've squashed all commits into one giant commit which is really, really unfortunate, especially for people who might want to contribute or understand the code base better. There are tools to port commit history over into git, they should have used such a tool. https://github.com/UWNetworksLab/arrakis/commits/master https://github.com/UWNetworksLab/arrakis/commits/master
- Argorak 12y ago
  Or they found the history too messy, didn't want to publish full author information with email (possibly, because it would mean getting permission) and similar cases.
- cwmma 12y ago
  It can be bad to do that if there might have been some private stuff in the history especially if its pretty long. This is one of the things that led to the 4chan hack (private key deep in history)
mahyarm 12y ago
This is embracing the second system effect & NIH syndrome that many large apps eventually get.
falconfunction 12y ago
what language is it being written in?
- McP 12y ago
  C
dllthomas 12y ago
I see the benefits to performance and arguably flexibility. I don't see why this provides "unprecedented reliability".
EGreg 12y ago
OK but how do you do pre emptive multitasking?
- wmf 12y ago
  I think the idea is to give each app dedicated cores. (And maybe save one core for miscellaneous stuff.) If the app wants threads it can use a libOS that provides whatever flavor of threads it prefers.
jmtame 12y ago
Just wanted to say that I love the naming of the OS. Brings back memories to the classic Dune games I played when I was younger.
- gabriel34 12y ago
  In case you didn't know, the game is part of a whole franchise that started with Frank Herbert's Novels. https://en.wikipedia.org/wiki/Dune_%28franchise%29#Games https://en.wikipedia.org/wiki/Dune_%28franchise%29#Games But yeah, the game was pretty awesome, even on it's own.
spiritplumber 12y ago
Isn't this what DOS did?
drivingmenuts 12y ago
Pretty much sounds like they're trying to turn my laptop into a PS3.
gumby 12y ago
Looks like they have reinvented part of a capability-based Multics system. Perhaps they should go read Organick: http://dl.acm.org/citation.cfm?id=1095599 http://dl.acm.org/citation.cfm?id=1095599
yazaddaruvala 12y ago
Where exactly is the framework to build operating systems? I just want a layer (ie all current linux drivers) without virtual memory or a protected mode (context switches) or process management. Just a nice API to run code on a core of my choosing, and read and write data from device "streams".
- wmf 12y ago
  These aren't exactly what you're asking for, but check out http://www.cs.utah.edu/flux/oskit/ http://www.cs.utah.edu/flux/oskit/ and http://wiki.osdev.org/Main_Page http://wiki.osdev.org/Main_Page
- justincormack 12y ago
  That is what you get with rump kernel http://rumpkernel.org/ http://rumpkernel.org/ although they are the NetBSD drivers not Linux.
cobolorum 12y ago
As an operating system for a dedicated, single purpose server this may be okay. As an operating system for mobile phones, this may one day be alright (when phones get about 12 cores). As an operating system for workstations and desktops, this is probably the worst idea I have heard in a long time. It sounds like a hipster version of multidos. At one application per core (or at least I think that's the idea), you are severely limiting the ability to multitask. So, on an 8 core system I have one core running the exokernel (1), another core running a gui (2), another core running an audio application (3), another running my web browser (4), another with my editor (5), another with git (6), another with a torrent going (7), and another with email (8). Due to the description, I am hoping that the GUI using a core and other applications having access to it is possible. I also hope that audio services don't need a core, or else the audio application developer will need to reimpelement OSS4 and/or ALSA in his/her application. That's just about idiotic... oh well...
na85 12y ago
>"The application gets the full power of the unmediated hardware, through an application-specific library linked into the application address space." This is pretty concerning, actually. I don't think I want or trust shady companies like Adobe to be running DRM-laden code directly on my hardware. Vendor lock-in is an increasingly common phenomenon and I'm picturing a really alarming future if this sort of OS takes off. I like that the linux kernel sits between my software and my hardware. Want to watch a Sony DVD? Better hope you have a webcam so that the media player application can directly access your facial reactions to the media being played and upload it to Sony's servers.
- anon4 12y ago
  And if it detects more than one person behind the screen it won't play unless you upgrade your movie license for public performance.
- icebraining 12y ago
  There's no reason why you couldn't run those apps on a sandbox, even inside Arrakis. And if Sony wanted to force you to have a webcam, they could do it now - they don't need Arrakis.
deleted 12y ago
[deleted]
tempodox 12y ago
I wish our office coffee machine had a direct demultiplexing port into my mouth. For the coffee, that is. Not the waste. application-specific library: Yeah, all my circles are squares, too.
getmailpin 12y ago
No abstraction is better than having abstraction. See http://pdos.csail.mit.edu/exo.html http://pdos.csail.mit.edu/exo.html
erikb 12y ago
Although this might be the best operating system ever in existance, I think it's quite hard to get anywhere in the desktop OS market. How can they find users? And if they don't find users, how can they find people who work with them on their software?
- wmf 12y ago
  Arrakis is more of a server OS.
  - erikb 12y ago
    Well, if that's the case I misunderstood the landing page. Thanks for the correction!
auggierose 12y ago
Isn't Arrakis something out of Dune?
- untothebreach 12y ago
  Yes, it is the name of the planet that is home to the Fremen and the sandworms. In the book(s), the name "Dune" is a nickname for Arrakis.
  - deleted 12y ago
    [deleted]
paraiuspau 12y ago
arrakis...dune...desert planet... ducks
- tempodox 12y ago
  To be more exact: Duckies! See: http://www.zefrank.com/theshow/ http://www.zefrank.com/theshow/
molsongolden 12y ago
I was really excited when I saw "Arrakis" on the front page but after looking at the project I'm not sure why they chose the name. If anything should be named Arrakis it should be a terraforming project...or a worm farm.

308 points by conductor 12y ago

Been done before (some would say to death), and the reason we have memory protection between applications is forgotten because people don't realize how nice it is. Sure, sure, your big "well engineered" web browser needs direct access to the hardware for speed, but painful experience has taught us that giving apps programmers direct access to hardware is a recipe for failure. Besides, there are already plenty of workarounds to get faster (eg, mmap) or even direct access to hardware from userland, not to mention the myriad of virtualization and protection schemes and levels in userland (eg, SELinux). This seems like a solution in search of already solved problems. Although as a research project, it does seem interesting . . .

rbanffy 12y ago
> Been done before (some would say to death) How is that George Santayana thing? "Those who cannot remember the past are condemned to repeat it"? I like seeing new ideas being tested in the OS arena. It's a real shame the two dominant OSs these days are Unix and VMS. I refuse to believe these two are the best humans can come up with.
- mikeash 12y ago
  From "The Rise of 'Worse is Better'", written in 1989: "The good news is that in 1995 we will have a good operating system and programming language; the bad news is that they will be Unix and C++." The predicted date was a little early, but otherwise he pretty much got it right. I think that, from a late 80s Lisp perspective, modern Windows fits into the "UNIX" category, and Java/C#/ObjC/whatever are close enough to C++ to count.
  - trhway 12y ago
    >The predicted date was a little early wouldn't say so - my first Linux machine (slackware, shoebox of diskettes :) i installed exactly in the 95 :) >from a late 80s Lisp perspective, modern Windows fits into the "UNIX" category while psychotropic mind-altering effects of Lisp are well known, i'd doubt that it can produce such perception changes or even in such a direction.
    - mikeash 12y ago
      Windows and UNIX are not so different. Monolithic kernels written in C or C++, permissions largely done with user granularity, byte-addressed memory, virtual memory with per-process address spaces, no hardware support for tagged pointers or garbage collection.... Compared to the variety that's gone before, they look nearly identical in their fundamentals.
- Executor 12y ago
  Thucydides?
chadgeidel 12y ago
I would go even further and say that apps developers don't want to write a networking stack, display driver, roll their own sound implementation, etc. It sounds like the "bad old days" of DOS where games had to have their own implementation of these things.
- wmf 12y ago
  That's why Arrakis provides that functionality through a library.
sounds 12y ago
Good points. Indulge me for a minute while I try to formulate what they might say in response: most of the complexity of an OS is device drivers. You're right, the basics of process isolation, scheduling, and resource allocation are pretty dull. Maybe the UW project makes the move to userspace official? At 1,000,000 packets per second it is more efficient to put the whole network stack in userspace. Assume similar gains can be had for all types of hardware, and shared libraries eliminate the overhead of loading the whole network stack every time you run 'ping'. That's just a 30-second pitch for the idea. Userspace is not new (see: Mach). I think what is changing is the average lifetime of each process. Both desktops and servers are beginning to become a commodity to the point where the old reasons for not using a microkernel are starting to become irrelevant: e.g. message passing cost isn't as much of a problem if the file server runs 'smbd' 99% of the time and that is the only really performance-sensitive application. Likewise for a desktop, except the running process is 'Chrome', or for a tablet running 'dalvikvm'. Really it's a move away from code complexity that wasn't possible before but is becoming possible.
- npsimons 12y ago
  So are we talking Microkernel here or something else? Sorry to constrain by categorizing, but it's what I do sometimes :) and I am thinking of some OSes (mostly research/academic) that went this way (of no layers between userland and the hardware) that weren't Microkernels, but I can't remember any names right now. I have to admit, part of what keeps my interest in this is that I used to dabble in OSes and I haven't gotten to for quite some time :( EDIT: Exokernel was what I was remembering (I think; https://en.wikipedia.org/wiki/Exokernel https://en.wikipedia.org/wiki/Exokernel). I can see some of the advantages, but as a current maintenance programmer and previous RTLinux hacker, I have to say I have my doubts about giving most programmers direct access to hardware :) That being said, giving programmers (and admins!) easier access to containerization, virtualization, or just intra-process protection in general would be a good idea. As I've said, there are plenty of options right now, but none seem to be well documented or advertised. There was a CACM article a while back that predicted a future in which you had one smartphone, but it had split domains for work and personal apps and data. Interesting stuff, and as you say, finally becoming possible.
bri3d 12y ago
Arrakis doesn't eliminate memory protection or process isolation, nor does it give applications direct hardware access. It takes advantage of modern hardware virtualization capabilities to present certain virtual devices to the guest applications, eliminating the kernel from the I/O path for critical operations like receiving a packet. Read this paper: http://arrakis.cs.washington.edu/wp-content/uploads/2013/04/arrakis-tr-ver2.pdf http://arrakis.cs.washington.edu/wp-content/uploads/2013/04/...

peterwwillis 12y ago

If I understand this correctly, they want all userspace applications to be Ring 0? That sounds.... problematic, to put it lightly.

SEJeff 12y ago
goto fail means an entirely different thing in that operating system
- trhway 12y ago
  until your application implements memory protection, etc... ie. their premise is that modern complex applications contains an OS inside it [ i don't fully agree with such premise, though remembering "nspr" state even back in 1999 i'm not surprised that 10+ years later Mozilla dind't have a problem coming up with their own OS, i'm more surprised that it didn't happen much earlier :) ]
steveklabnik 12y ago
One might say solving that problem is the entire point of the project.

ChuckMcM 12y ago

Interesting, its a fork of Barrelfish [1] which is the one-core-one-OS OS. When I first heard about it, it sounded like Multi-DOS (several instances of MS-DOS running at once) but its a bit more sophisticated than that :-). Other than cache contention (which is always going to be a problem) its an interesting approach. [1] http://www.barrelfish.org/TN-000-Overview.pdf http://www.barrelfish.org/TN-000-Overview.pdf

akavel 12y ago

Is this the same idea as "exokernel", and thus a (probably valuable) attempt at implementation of one, or does it differ in some important points? https://en.wikipedia.org/wiki/Exokernel https://en.wikipedia.org/wiki/Exokernel

teraflop 12y ago
It looks similar. MIT's Exokernel seems to use kernel-level hooks to control access to things like disk blocks and network packets. Arrakis takes a similar approach, but restricts those hooks to a subset that can be implemented in hardware instead of in the kernel.
nostrademons 12y ago
It's interesting to see many concepts and general design philosophy of exokernels make their way into modern systems. Zero-copy, mmap, RDMA, vectored I/O, fibers/switchto, FUSE - all of these are attempts to push as many policy decisions into user space as possible so that the OS only deals with securely multiplexing the hardware. The irony is that rather than new kernels, these are being added on as new APIs to the Linux kernel. I suppose that makes a lot of sense because it's much easier to expose a new syscall and see if it gets any adoption rather than convincing everybody to switch to a whole new OS.

teraflop 12y ago

This is pretty interesting, but unfortunately their intro page doesn't do a good job of explaining why. (For instance, they use the word "unprecedented" without really saying what makes Arrakis different from previous exokernel-like designs.) I'll try to summarize what I got out of skimming the paper[1]: When you have multiple applications running on the same machine, you need some way to safely share resources between them; for example, incoming network packets are a resource. A kernel handles this by keeping a data structure mapping sockets to processes, and demultiplexing data that comes in from the network card. Hypervisors work the same way, except at the level of virtual machines rather than processes. Arrakis does the same thing, but relies on hardware support in the network card to dispatch packets to the right process. This relies on a standard called SR-IOV[2] which allows the OS to configure a PCI device to present itself as multiple virtual subdevices. The kernel programs the NIC to dispatch packets to different buffers depending on the incoming MAC address; after that, packets can be dispatched with no kernel involvement at all. Similarly, you can tell a disk controller to present a particular extent of a disk as a new virtual storage device. The blurb about memory protection seems to be a red herring, because as far as I can see they haven't done anything to change that. There's still a kernel, which handles requests for resource mappings, and processes are still isolated from each other. But once they've requested the mappings that they need, the normal execution path doesn't involve any syscalls, and so there's no kernel overhead. The real contribution of the paper is designing an API around this idea and proving that real applications like Redis can be ported to it. [1] http://arrakis.cs.washington.edu/wp-content/uploads/2013/04/arrakis-tr-ver2.pdf http://arrakis.cs.washington.edu/wp-content/uploads/2013/04/... [2] http://blog.scottlowe.org/2009/12/02/what-is-sr-iov/ http://blog.scottlowe.org/2009/12/02/what-is-sr-iov/

jws 12y ago
But where does one get SR-IOV devices to experiment with? It looks like the Intel 82576 chipset has SR-IOV, and can be had in a $50 card, two ports, 8 filters per port. The Intel 82599 is a 10Gb chip with more filters per port. (with linux support for the SR-IOV, it can manifest as multiple devices) Disk storage is less obvious. The LSI 2308 and 3008 controller chips probably support it, but I'm not finding a commodity card or a motherboard integrating one. They mention in the paper that existing devices have problems which prevent them from actually securely isolating clients of the sub devices. Combine this with seeing a lot of web activity about SR-IOV in 2009 and not much now. Either it became too common to mention, or is dwindling into an idea that didn't catch on. The wait for secure SR-IOV might be interminable.

pygy_ 12y ago

In reply to many comments about the lack of safety of the approach: They claim that "[they] demonstrate that operating system protection is not contradictory with high performance". Abstract of their latest paper: Recent device hardware trends enable a new approach to the design of network server operating systems. In a traditional operating system, the kernel mediates access to device hardware by server applications, to enforce pro- cess isolation as well as network and disk security. We have designed and implemented a new operating system, Arrakis, that splits the traditional role of the kernel in two. Applications have direct access to virtualized I/O devices, allowing most I/O operations to skip the ker- nel entirely, while the kernel is re-engineered to provide network and disk protection without kernel mediation of every operation. We describe the hardware and software changes needed to take advantage of this new abstraction, and we illustrate its power by showing 2-5x end-to-end latency and 9x throughput improvements for a popular persistent NoSQL store [i.e. Redis] relative to a well-tuned Linux implementation. http://arrakis.cs.washington.edu/wp-content/uploads/2013/04/arrakis-tr-ver2.pdf http://arrakis.cs.washington.edu/wp-content/uploads/2013/04/...

michaelmior 12y ago

If you like this concept, you may also find Mirage[1] interesting. Mirage compiles the application code into the kernel to run directly on the Xen hypervisor. (Thus system calls become ordinary function calls. They do some tricks to maintain security.) [1] http://www.openmirage.org/ http://www.openmirage.org/

GuiA 12y ago

"Applications are becoming so complex that they are miniature operating systems in their own right and are hampered by the existing OS protection model" Sure, that's true for browsers, as they mention, and a few other degenerate cases (eg. virtualization software?) - but that's certainly not the case for the vast majority of applications I run (text editor, terminal, mail client, IM client, etc.). How does this argument hold?

nekopa 12y ago
Emacs.
- magpi3 12y ago
  And let's be honest: VIM has been heading in this direction for a long time as well, especially now that it had a built in python interpreter.
pyre 12y ago
It's possible that the operating system on your desktop could take on an entirely different paradigm... E.g.: Hardware | Arrakis | __|__ / | \ Browser | System Services... Wrapper / | \ IM EMail Terminal
- nealabq 12y ago
  As the OS gets thinner and more like a hypervisor, the apps and app-like VMs get more OS-like. One day the browser might have daemons. I'd guess at the same time all these pieces will continue to become more distributed. Hypervisors and apps will present distributed environments (storage, processing, failover), even more than they do today.
  - stephengillie 12y ago
    At one time, all an Operating System did was load other programs into memory so the processor would run them instead. All of the sandboxing, garbage collection, security, and other features were added to make it more usable.

phillmv 12y ago

Heh, this sounds analogous to what Gary Bernhardt finished "The Birth & Death of Javascript" with: https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript https://www.destroyallsoftware.com/talks/the-birth-and-death...

thirsteh 12y ago
Or what a bunch of projects have been doing for years -- Erlang on Xen, HalVM, etc. etc.

leorocky 12y ago

Hrm, I wonder what they were using for their source control before making it available on GitHub, they've squashed all commits into one giant commit which is really, really unfortunate, especially for people who might want to contribute or understand the code base better. There are tools to port commit history over into git, they should have used such a tool. https://github.com/UWNetworksLab/arrakis/commits/master https://github.com/UWNetworksLab/arrakis/commits/master

Argorak 12y ago
Or they found the history too messy, didn't want to publish full author information with email (possibly, because it would mean getting permission) and similar cases.
cwmma 12y ago
It can be bad to do that if there might have been some private stuff in the history especially if its pretty long. This is one of the things that led to the 4chan hack (private key deep in history)

mahyarm 12y ago

This is embracing the second system effect & NIH syndrome that many large apps eventually get.

falconfunction 12y ago

what language is it being written in?

McP 12y ago
C

dllthomas 12y ago

I see the benefits to performance and arguably flexibility. I don't see why this provides "unprecedented reliability".

EGreg 12y ago

OK but how do you do pre emptive multitasking?

wmf 12y ago
I think the idea is to give each app dedicated cores. (And maybe save one core for miscellaneous stuff.) If the app wants threads it can use a libOS that provides whatever flavor of threads it prefers.

jmtame 12y ago

Just wanted to say that I love the naming of the OS. Brings back memories to the classic Dune games I played when I was younger.

gabriel34 12y ago
In case you didn't know, the game is part of a whole franchise that started with Frank Herbert's Novels. https://en.wikipedia.org/wiki/Dune_%28franchise%29#Games https://en.wikipedia.org/wiki/Dune_%28franchise%29#Games But yeah, the game was pretty awesome, even on it's own.

spiritplumber 12y ago

Isn't this what DOS did?

drivingmenuts 12y ago

Pretty much sounds like they're trying to turn my laptop into a PS3.

gumby 12y ago

Looks like they have reinvented part of a capability-based Multics system. Perhaps they should go read Organick: http://dl.acm.org/citation.cfm?id=1095599 http://dl.acm.org/citation.cfm?id=1095599

yazaddaruvala 12y ago

Where exactly is the framework to build operating systems? I just want a layer (ie all current linux drivers) without virtual memory or a protected mode (context switches) or process management. Just a nice API to run code on a core of my choosing, and read and write data from device "streams".

wmf 12y ago
These aren't exactly what you're asking for, but check out http://www.cs.utah.edu/flux/oskit/ http://www.cs.utah.edu/flux/oskit/ and http://wiki.osdev.org/Main_Page http://wiki.osdev.org/Main_Page
justincormack 12y ago
That is what you get with rump kernel http://rumpkernel.org/ http://rumpkernel.org/ although they are the NetBSD drivers not Linux.

cobolorum 12y ago

As an operating system for a dedicated, single purpose server this may be okay. As an operating system for mobile phones, this may one day be alright (when phones get about 12 cores). As an operating system for workstations and desktops, this is probably the worst idea I have heard in a long time. It sounds like a hipster version of multidos. At one application per core (or at least I think that's the idea), you are severely limiting the ability to multitask. So, on an 8 core system I have one core running the exokernel (1), another core running a gui (2), another core running an audio application (3), another running my web browser (4), another with my editor (5), another with git (6), another with a torrent going (7), and another with email (8). Due to the description, I am hoping that the GUI using a core and other applications having access to it is possible. I also hope that audio services don't need a core, or else the audio application developer will need to reimpelement OSS4 and/or ALSA in his/her application. That's just about idiotic... oh well...

na85 12y ago

>"The application gets the full power of the unmediated hardware, through an application-specific library linked into the application address space." This is pretty concerning, actually. I don't think I want or trust shady companies like Adobe to be running DRM-laden code directly on my hardware. Vendor lock-in is an increasingly common phenomenon and I'm picturing a really alarming future if this sort of OS takes off. I like that the linux kernel sits between my software and my hardware. Want to watch a Sony DVD? Better hope you have a webcam so that the media player application can directly access your facial reactions to the media being played and upload it to Sony's servers.

anon4 12y ago
And if it detects more than one person behind the screen it won't play unless you upgrade your movie license for public performance.
icebraining 12y ago
There's no reason why you couldn't run those apps on a sandbox, even inside Arrakis. And if Sony wanted to force you to have a webcam, they could do it now - they don't need Arrakis.

deleted 12y ago

[deleted]

tempodox 12y ago

I wish our office coffee machine had a direct demultiplexing port into my mouth. For the coffee, that is. Not the waste. application-specific library: Yeah, all my circles are squares, too.

getmailpin 12y ago

No abstraction is better than having abstraction. See http://pdos.csail.mit.edu/exo.html http://pdos.csail.mit.edu/exo.html

erikb 12y ago

Although this might be the best operating system ever in existance, I think it's quite hard to get anywhere in the desktop OS market. How can they find users? And if they don't find users, how can they find people who work with them on their software?

wmf 12y ago
Arrakis is more of a server OS.
- erikb 12y ago
  Well, if that's the case I misunderstood the landing page. Thanks for the correction!

auggierose 12y ago

Isn't Arrakis something out of Dune?

untothebreach 12y ago
Yes, it is the name of the planet that is home to the Fremen and the sandworms. In the book(s), the name "Dune" is a nickname for Arrakis.
- deleted 12y ago
  [deleted]

paraiuspau 12y ago

arrakis...dune...desert planet... ducks

tempodox 12y ago
To be more exact: Duckies! See: http://www.zefrank.com/theshow/ http://www.zefrank.com/theshow/

molsongolden 12y ago

I was really excited when I saw "Arrakis" on the front page but after looking at the project I'm not sure why they chose the name. If anything should be named Arrakis it should be a terraforming project...or a worm farm.