5 ms·

What is the likelihood that we will see a major operating system written in a safe language such as Rust in the next 10 years?

saghm 9y ago
That depends; do you mean that it has to be written in the next ten years and then later becomes widely used, or that it has to become widely used in the next ten years? If the former, I'd say maybe if Redox becomes mature enough to be widely used maybe a decade or so down the line (with "widely used" meaning "used as much as at least one of the major BSDs", since those are what are being discussed in the PDF), and if the latter, I'd say almost definitely not (and this is coming from a huge Rust fanatic)
- Skunkleton 9y ago
  The best case scenario in that time frame would be to implement portions/new features of an existing kernel in a safe language. Maybe it could start with officially supported driver modules in rust?
  - mpe 9y ago
    Yeah I agree. It would be great if parts of the kernel could be written in a safer language. I think in 10 years that will be happening in at least some widely used OS, but most likely one that only supports a small range of architectures.
bluejekyll 9y ago
I'm really excited for Redox. That being said, I still think the best strategy will be to start porting different portions of the Kernels to Rust. Perhaps starting with drivers. There have been a few examples of Rust drivers for the various OSes out there. First, the maintainers have to believe that this is better than continuing with C. That doesn't seem to be something everyone believes.
yellowapple 9y ago
Various OpenBSD devs have had mixed (at best) feelings about Rust, recognizing it as a language that is by no means a silver bullet for safe code. Part of this was/is also flavored by the fact that LLVM (and therefore Rust) supports only a subset of the hardware platforms supported by OpenBSD. However, now that OpenBSD is continuing to switch from GCC to LLVM/clang as the default C compiler for an increasing number of targets and as an increasing number of non-LLVM-supported targets are discontinued, I reckon the idea of some or all of OpenBSD being incrementally reimplemented in Rust is - bit by bit - becoming a little less far-fetched. It also helps that Rust is liberally-licensed. That all said, I'm not really holding my breath for that happening anytime soon. Until then, Redox continues to look interesting.
pjmlp 9y ago
The problem is more political than technical. There have been operating systems written in mostly safe systems programming languages since the 60s, 10 years before C was born. Burroughs created ESPOL and NEWP, IBM did their RISC research with PL/8, Xerox had Mesa/Cedar, UK Royal Navy Algol-68RS, and many many others with Midori's System C# being the latest example. If there isn't political will, there isn't the motivation to overcome any technical hurdles that might appear. Now even with Microsoft adopting a Linux compatibility layer we are reaching the age of UNIX mono-culture, and its dependency to C for kernel development. So the question is who will have the political will to break this reality, and drive the technical adoption regardless of the hurdles that might appear along the way.
mike_hearn 9y ago
Well, assuming by "major OS" you mean a full blown OS and not just a kernel, Android is a major operating system and large chunks of it are written in Java. So I think that is the closest humanity has got so far. I was tempted to say ChromeOS but I don't think anything non-trivial in ChromeOS is actually written in JavaScript: that's C++ for the OS people and JS for everyone else. Android is more consistent in that core OS components are written in the same way as user apps are. If I had to place bets, therefore, I'd say that Android is best placed to get closest to the ideal of a fully safe OS within 10 years. I have no idea if Google would do it, but they have the funds, the security commitment and there's a clear path opening up for them to make nearly all userland code in Android safe. But it doesn't look like "rewrite everything in Rust". It looks like this: Continue to improve ART's JIT and AOT compilers, possibly teaming up with the Graal project. The Java guys are doing a lot of work right now on adding support for value types and subsequent large upgrades to support for vectorisation (as with value types you can represent 256-bit wide primitives and things like that). This unlocks the potential to port parts of the most security-vulnerable and performance sensitive components like the media server and media codecs into Java or Kotlin, without much performance slowdown (bulk of the most important h264 decoding is done in hardware anyway). But there will still be lots of C/C++ code in Android and rewriting it would be very expensive. So a feasible next step is to swallow the bitter political pill and team up with Oracle Labs to get access to SubstrateVM + Managed Sulong (or wait and see if they are open sourced). This rather magical combination of tools appears to offer the potential to run anything LLVM can compile in a way that eliminates all forms of memory unsafety including buffer overflows, out of range reads/writes, double frees etc at a nearly trivial cost (around 10-20% at most): http://www.ssw.uni-linz.ac.at/General/Staff/ManuelRigger/ECOOP16-DS.pdf http://www.ssw.uni-linz.ac.at/General/Staff/ManuelRigger/ECO... If this technology lives up to its potential, and it seems like it is in with a very good chance based on progress so far, I suspect we'll see a loss of interest in Rust over time ... at least for the "make existing software secure" use case. Managed Sulong + SubstrateVM can run C/C++ code that is strictly standards compliant (it struggles with code that makes spec-violating assumptions as it exploits UB very aggressively), but it's much easier to fix compliance issues in an existing codebase than rewrite it from scratch. The Android team have already switched to OpenJDK and abandoned their plans to rewrite the Java compiler, so with the resolution of the Oracle lawsuits they do seem to be technically moving closer to the Java platform rather than away from it. Collaboration between Google and Oracle on this doesn't seem as unthinkable as it once did.
nickpsecurity 9y ago
It's been done in the past. OS's come to dominate mostly for marketing rather than technical reasons. Much like startups. Integration with legacy systems and rapid development increase chance they'll get uptake. The existing stuff is mostly C, C++, etc. So, the new stuff that dominates tends to reuse that. Redoing it all in a safe language will be a lot of work. That's why high-assurance security has focused on redoing the smallest, most-trusted parts of the stack or working on transformations of legacy code like Softbound+CETS or CHERI CPU that address root problems. Although I doubt we'll see a big one, there's already smaller projects that do useful stuff with safer languages. ExpressOS comes to mind given the kernel is C# and Dafny running Android apps that are Java. The L4 kernel and Linux remain in the system with L4 replaceable by safe stuff like Muen in SPARK language. Replacing Linux will be a lot harder. ;) However, if wanting mobile or other apps that aren't Android compatible, one might be able to use work like Redox, JX, or Oberon to eliminate unsafe language entirely. https://haohuim.appspot.com/static/pubs/expressos-asplos13-slides.pdf https://haohuim.appspot.com/static/pubs/expressos-asplos13-s... Another in alternative OS's is A2 Bluebottle from Wirth's people at ETH written in an Oberon dialect. Main Oberon OS is designed for simplicity and teaching. A2 is a batteries-included OS with a GUI designed to be useful. When I tried it, it ran snappy fast with networking, editor, compiler, and so on. Very minimal given only a few students worked on it but it is usable and extendable. Below was first link I found in Google so I haven't vetted it. https://github.com/Ignat99/a2 https://github.com/Ignat99/a2 Redox OS in Rust can do a lot currently. They plan to expand it more. Already has a GUI, shell, networking, and the standard library of Rust. http://redox-os.org/ http://redox-os.org/

saghm 9y ago

That depends; do you mean that it has to be written in the next ten years and then later becomes widely used, or that it has to become widely used in the next ten years? If the former, I'd say maybe if Redox becomes mature enough to be widely used maybe a decade or so down the line (with "widely used" meaning "used as much as at least one of the major BSDs", since those are what are being discussed in the PDF), and if the latter, I'd say almost definitely not (and this is coming from a huge Rust fanatic)

Skunkleton 9y ago
The best case scenario in that time frame would be to implement portions/new features of an existing kernel in a safe language. Maybe it could start with officially supported driver modules in rust?
- mpe 9y ago
  Yeah I agree. It would be great if parts of the kernel could be written in a safer language. I think in 10 years that will be happening in at least some widely used OS, but most likely one that only supports a small range of architectures.

bluejekyll 9y ago

I'm really excited for Redox. That being said, I still think the best strategy will be to start porting different portions of the Kernels to Rust. Perhaps starting with drivers. There have been a few examples of Rust drivers for the various OSes out there. First, the maintainers have to believe that this is better than continuing with C. That doesn't seem to be something everyone believes.

yellowapple 9y ago

Various OpenBSD devs have had mixed (at best) feelings about Rust, recognizing it as a language that is by no means a silver bullet for safe code. Part of this was/is also flavored by the fact that LLVM (and therefore Rust) supports only a subset of the hardware platforms supported by OpenBSD. However, now that OpenBSD is continuing to switch from GCC to LLVM/clang as the default C compiler for an increasing number of targets and as an increasing number of non-LLVM-supported targets are discontinued, I reckon the idea of some or all of OpenBSD being incrementally reimplemented in Rust is - bit by bit - becoming a little less far-fetched. It also helps that Rust is liberally-licensed. That all said, I'm not really holding my breath for that happening anytime soon. Until then, Redox continues to look interesting.

pjmlp 9y ago

The problem is more political than technical. There have been operating systems written in mostly safe systems programming languages since the 60s, 10 years before C was born. Burroughs created ESPOL and NEWP, IBM did their RISC research with PL/8, Xerox had Mesa/Cedar, UK Royal Navy Algol-68RS, and many many others with Midori's System C# being the latest example. If there isn't political will, there isn't the motivation to overcome any technical hurdles that might appear. Now even with Microsoft adopting a Linux compatibility layer we are reaching the age of UNIX mono-culture, and its dependency to C for kernel development. So the question is who will have the political will to break this reality, and drive the technical adoption regardless of the hurdles that might appear along the way.

mike_hearn 9y ago

Well, assuming by "major OS" you mean a full blown OS and not just a kernel, Android is a major operating system and large chunks of it are written in Java. So I think that is the closest humanity has got so far. I was tempted to say ChromeOS but I don't think anything non-trivial in ChromeOS is actually written in JavaScript: that's C++ for the OS people and JS for everyone else. Android is more consistent in that core OS components are written in the same way as user apps are. If I had to place bets, therefore, I'd say that Android is best placed to get closest to the ideal of a fully safe OS within 10 years. I have no idea if Google would do it, but they have the funds, the security commitment and there's a clear path opening up for them to make nearly all userland code in Android safe. But it doesn't look like "rewrite everything in Rust". It looks like this: Continue to improve ART's JIT and AOT compilers, possibly teaming up with the Graal project. The Java guys are doing a lot of work right now on adding support for value types and subsequent large upgrades to support for vectorisation (as with value types you can represent 256-bit wide primitives and things like that). This unlocks the potential to port parts of the most security-vulnerable and performance sensitive components like the media server and media codecs into Java or Kotlin, without much performance slowdown (bulk of the most important h264 decoding is done in hardware anyway). But there will still be lots of C/C++ code in Android and rewriting it would be very expensive. So a feasible next step is to swallow the bitter political pill and team up with Oracle Labs to get access to SubstrateVM + Managed Sulong (or wait and see if they are open sourced). This rather magical combination of tools appears to offer the potential to run anything LLVM can compile in a way that eliminates all forms of memory unsafety including buffer overflows, out of range reads/writes, double frees etc at a nearly trivial cost (around 10-20% at most): http://www.ssw.uni-linz.ac.at/General/Staff/ManuelRigger/ECOOP16-DS.pdf http://www.ssw.uni-linz.ac.at/General/Staff/ManuelRigger/ECO... If this technology lives up to its potential, and it seems like it is in with a very good chance based on progress so far, I suspect we'll see a loss of interest in Rust over time ... at least for the "make existing software secure" use case. Managed Sulong + SubstrateVM can run C/C++ code that is strictly standards compliant (it struggles with code that makes spec-violating assumptions as it exploits UB very aggressively), but it's much easier to fix compliance issues in an existing codebase than rewrite it from scratch. The Android team have already switched to OpenJDK and abandoned their plans to rewrite the Java compiler, so with the resolution of the Oracle lawsuits they do seem to be technically moving closer to the Java platform rather than away from it. Collaboration between Google and Oracle on this doesn't seem as unthinkable as it once did.

nickpsecurity 9y ago

It's been done in the past. OS's come to dominate mostly for marketing rather than technical reasons. Much like startups. Integration with legacy systems and rapid development increase chance they'll get uptake. The existing stuff is mostly C, C++, etc. So, the new stuff that dominates tends to reuse that. Redoing it all in a safe language will be a lot of work. That's why high-assurance security has focused on redoing the smallest, most-trusted parts of the stack or working on transformations of legacy code like Softbound+CETS or CHERI CPU that address root problems. Although I doubt we'll see a big one, there's already smaller projects that do useful stuff with safer languages. ExpressOS comes to mind given the kernel is C# and Dafny running Android apps that are Java. The L4 kernel and Linux remain in the system with L4 replaceable by safe stuff like Muen in SPARK language. Replacing Linux will be a lot harder. ;) However, if wanting mobile or other apps that aren't Android compatible, one might be able to use work like Redox, JX, or Oberon to eliminate unsafe language entirely. https://haohuim.appspot.com/static/pubs/expressos-asplos13-slides.pdf https://haohuim.appspot.com/static/pubs/expressos-asplos13-s... Another in alternative OS's is A2 Bluebottle from Wirth's people at ETH written in an Oberon dialect. Main Oberon OS is designed for simplicity and teaching. A2 is a batteries-included OS with a GUI designed to be useful. When I tried it, it ran snappy fast with networking, editor, compiler, and so on. Very minimal given only a few students worked on it but it is usable and extendable. Below was first link I found in Google so I haven't vetted it. https://github.com/Ignat99/a2 https://github.com/Ignat99/a2 Redox OS in Rust can do a lot currently. They plan to expand it more. Already has a GUI, shell, networking, and the standard library of Rust. http://redox-os.org/ http://redox-os.org/