6 ms·

They should rewrite Go in Rust! That way we can avoid all this Go vs Rust discussions ;) Problem with your idea, is that low level kernel will use a lot of uns

by Ygg2 9y ago

They should rewrite Go in Rust! That way we can avoid all this Go vs Rust discussions ;) Problem with your idea, is that low level kernel will use a lot of unsafe Rust, which will lose lot of benefits.

cyber1 9y ago
According to memory management, yes but you have a lot of benefit from the strong type-safe system, ecosystem, testing and etc.
ekidd 9y ago
> Problem with your idea, is that low level kernel will use a lot of unsafe Rust, which will lose lot of benefits. I've actually worked on a toy kernel in Rust (using the excellent tutorial at https://os.phil-opp.com/ https://os.phil-opp.com/), and it turns out that, yes, you obviously need to use unsafe code to talk to the actual hardware. But in most cases, you can encapsulate the low-level hardware inside a safe API: https://github.com/emk/toyos-rs/blob/fdc5fb8cc8152a63d1b6c85cd357737e6b1aebb5/crates/cpuio/src/lib.rs#L43-L75 https://github.com/emk/toyos-rs/blob/fdc5fb8cc8152a63d1b6c85... In this example, only I/O port creation is an unsafe API, because you need to specify a memory address to read and write. But once the port is created (pointed at an appropriate address!), it's perfectly safe to use. So, yes, kernel-space Rust will use "unsafe" far more often than regular Rust code. But you can still make at least 80% of your code safe, and maybe much more. And the remaining "unsafe" APIs act as a useful warning to pay attention to what you're doing. Plus, Rust is a really nice language to write kernel code in, anyway.
- angrygoat 9y ago
  Nice to have the dangerous bits annotated – in a C/C++/... kernel, everything is "unsafe". "Given enough eyeballs, all bugs are shallow" - but it helps when the eyeballs are focussed! :-)
  - staticassertion 9y ago
    It's hard to emphasize the auditability enough. With auditing a C/C++ codebase, in my experience, you rely a lot on intuition. "Well, there's parsing done here, so I'll focus my efforts" or "Historically we've had a lot of CVEs from this part of code so I'll start there". I'm not a professional RE so my experience is limited, but when I went looking for vulns that's how I went about it, and I think that's generally the case. With rust there's significantly less guesswork. That parser doesn't use unsafe? OK, let's start elsewhere. That seemingly innocent code uses unsafe? Great, check that out. You can grep for vulnerabilities, basically.
- Spiritus 9y ago
  >But in most cases, you can encapsulate the low-level hardware inside a safe API I'm probably missing something obvious. But isn't that true for most languages?
  - Ygg2 9y ago
    Not really. I mean let's say you program in C, how will you enforce some pointer is never null? In Rust you can say &Object and that reference is never null (modulo any unsafe shenanigans).
    - megous 9y ago
      Check it when you use it?
      - sgift 9y ago
        Experience tells us that this idea doesn't work so well in practice without a compiler yelling if you don't do it.
        megous 9y ago
        Compiler doesn't yell, because it's unnecessary in most cases. My experience is that huge amount of C code running on my computer had exactly zero issues like this today. It has been working perfectly fine. I run GNOME which uses a paradigm of checking all inputs to a function on entry via g_return(_val)_if_fail(assertion_expression). It helps the programmer do the right thing when it comes to using APIs that are disallowing NULL input. Two days ago, the code on my workstation hit one of those "assertions": Sep 26 23:40:31 core transmission-gt[1084]: g_file_test: assertion 'filename != NULL' failed No oops in the kernel recently. Millions of lines of desktop and kernel code, and one failed assertion in two days for using NULL incorrectly in the API. So unless your standard is absolute perfection, it works fine as is.
    - Jyaif 9y ago
      thankfully C++ has references.
- Thaxll 9y ago
  So you could do the same in C/C++, if you use unsafe it's unsafe that it's. I can't believe people arguing over unsafe used in a safe way.
  - TuringTest 9y ago
    There's unsafe hidden under an API that you know where you're using, and unsafe in every single line in your software. Just like there's changing state encapsulated in object methods, and changing state by assigning to global variables. Theoretically it's just all the same, but in practice having reduced interfaces does wonders to reduce the complexity and being able to reason about what brings problems, and what doesn't.
  - sa1 9y ago
    The responsibility of proving safety can belong to the compiler or it could belong to the programmer. The compiler can do a very good job, but there are edge cases where your code can become convoluted and things aren't expressed easily. The programmer can do a good job of expressing things, but is usually not able to prove safety of large blocks. unsafe blocks are just a compromise where the programmer only has to prove safety of small blocks of code which aren't easily expressed in the type system.
cyphar 9y ago
I would recommend looking at RedoxOS[1] (a kernel+userspace written in Rust) to see what a fairly reasonably featured operating system written in Rust looks like. "Using a lot of unsafe Rust" doesn't actually lose the benefits of Rust -- in fact annotating some code as unsafe is one of the benefits of Rust (you can much more easily tell where certain classes of bugs must originate from). [1]: https://www.redox-os.org/ https://www.redox-os.org/
tmzt 9y ago
It would be kind of cool to see a `go gen`-syntax based Go + generics implemented in Rust. Another thing that has been attempted is a stdlib for Rust that uses syscalls into the kernel much like Go's.
- ekidd 9y ago
  > Another thing that has been attempted is a stdlib for Rust that uses syscalls into the kernel much like Go's. With Rust, it's easier just to replace the C library! In particular, cross-compiling with a static musl-libc has been supported out of the box with for a while now. For pure Rust programs (or ones with small amounts of C handled by cargo), just write: cargo build --target=x86_64-unknown-linux-musl This will produce a 100% static binary that relies on nothing except the kernel. For programs which require external C libraries, it's a bit trickier, because you need a static version of those libraries built against musl-libc. For common libraries like OpenSSL and libpq, I have a Docker container that makes this easier: https://github.com/emk/rust-musl-builder https://github.com/emk/rust-musl-builder
nickpsecurity 9y ago
Both House OS in Haskell and SVA-OS handled that by cleanly hiding unsafe details behind an interface that could be used safely: http://programatica.cs.pdx.edu/House/ http://programatica.cs.pdx.edu/House/ https://llvm.org/pubs/2009-08-12-UsenixSecurity-SafeSVAOS.html https://llvm.org/pubs/2009-08-12-UsenixSecurity-SafeSVAOS.ht... So, that's not really a limitation. Worst-case scenario is proving those primitives correct with external tools whose preconditions and invariants are just checked by memory-safe code calling it. The above work shows worst case might not happen, though.
mtgx 9y ago
Well...maybe just the compiler? ;)

by Ygg2 9y ago

cyber1 9y ago

According to memory management, yes but you have a lot of benefit from the strong type-safe system, ecosystem, testing and etc.

ekidd 9y ago

> Problem with your idea, is that low level kernel will use a lot of unsafe Rust, which will lose lot of benefits. I've actually worked on a toy kernel in Rust (using the excellent tutorial at https://os.phil-opp.com/ https://os.phil-opp.com/), and it turns out that, yes, you obviously need to use unsafe code to talk to the actual hardware. But in most cases, you can encapsulate the low-level hardware inside a safe API: https://github.com/emk/toyos-rs/blob/fdc5fb8cc8152a63d1b6c85cd357737e6b1aebb5/crates/cpuio/src/lib.rs#L43-L75 https://github.com/emk/toyos-rs/blob/fdc5fb8cc8152a63d1b6c85... In this example, only I/O port creation is an unsafe API, because you need to specify a memory address to read and write. But once the port is created (pointed at an appropriate address!), it's perfectly safe to use. So, yes, kernel-space Rust will use "unsafe" far more often than regular Rust code. But you can still make at least 80% of your code safe, and maybe much more. And the remaining "unsafe" APIs act as a useful warning to pay attention to what you're doing. Plus, Rust is a really nice language to write kernel code in, anyway.

angrygoat 9y ago
Nice to have the dangerous bits annotated – in a C/C++/... kernel, everything is "unsafe". "Given enough eyeballs, all bugs are shallow" - but it helps when the eyeballs are focussed! :-)
- staticassertion 9y ago
  It's hard to emphasize the auditability enough. With auditing a C/C++ codebase, in my experience, you rely a lot on intuition. "Well, there's parsing done here, so I'll focus my efforts" or "Historically we've had a lot of CVEs from this part of code so I'll start there". I'm not a professional RE so my experience is limited, but when I went looking for vulns that's how I went about it, and I think that's generally the case. With rust there's significantly less guesswork. That parser doesn't use unsafe? OK, let's start elsewhere. That seemingly innocent code uses unsafe? Great, check that out. You can grep for vulnerabilities, basically.
Spiritus 9y ago
>But in most cases, you can encapsulate the low-level hardware inside a safe API I'm probably missing something obvious. But isn't that true for most languages?
- Ygg2 9y ago
  Not really. I mean let's say you program in C, how will you enforce some pointer is never null? In Rust you can say &Object and that reference is never null (modulo any unsafe shenanigans).
  - megous 9y ago
    Check it when you use it?
    - sgift 9y ago
      Experience tells us that this idea doesn't work so well in practice without a compiler yelling if you don't do it.
      - megous 9y ago
        Compiler doesn't yell, because it's unnecessary in most cases. My experience is that huge amount of C code running on my computer had exactly zero issues like this today. It has been working perfectly fine. I run GNOME which uses a paradigm of checking all inputs to a function on entry via g_return(_val)_if_fail(assertion_expression). It helps the programmer do the right thing when it comes to using APIs that are disallowing NULL input. Two days ago, the code on my workstation hit one of those "assertions": Sep 26 23:40:31 core transmission-gt[1084]: g_file_test: assertion 'filename != NULL' failed No oops in the kernel recently. Millions of lines of desktop and kernel code, and one failed assertion in two days for using NULL incorrectly in the API. So unless your standard is absolute perfection, it works fine as is.
  - Jyaif 9y ago
    thankfully C++ has references.
Thaxll 9y ago
So you could do the same in C/C++, if you use unsafe it's unsafe that it's. I can't believe people arguing over unsafe used in a safe way.
- TuringTest 9y ago
  There's unsafe hidden under an API that you know where you're using, and unsafe in every single line in your software. Just like there's changing state encapsulated in object methods, and changing state by assigning to global variables. Theoretically it's just all the same, but in practice having reduced interfaces does wonders to reduce the complexity and being able to reason about what brings problems, and what doesn't.
- sa1 9y ago
  The responsibility of proving safety can belong to the compiler or it could belong to the programmer. The compiler can do a very good job, but there are edge cases where your code can become convoluted and things aren't expressed easily. The programmer can do a good job of expressing things, but is usually not able to prove safety of large blocks. unsafe blocks are just a compromise where the programmer only has to prove safety of small blocks of code which aren't easily expressed in the type system.

cyphar 9y ago

I would recommend looking at RedoxOS[1] (a kernel+userspace written in Rust) to see what a fairly reasonably featured operating system written in Rust looks like. "Using a lot of unsafe Rust" doesn't actually lose the benefits of Rust -- in fact annotating some code as unsafe is one of the benefits of Rust (you can much more easily tell where certain classes of bugs must originate from). [1]: https://www.redox-os.org/ https://www.redox-os.org/

tmzt 9y ago

It would be kind of cool to see a `go gen`-syntax based Go + generics implemented in Rust. Another thing that has been attempted is a stdlib for Rust that uses syscalls into the kernel much like Go's.

ekidd 9y ago
> Another thing that has been attempted is a stdlib for Rust that uses syscalls into the kernel much like Go's. With Rust, it's easier just to replace the C library! In particular, cross-compiling with a static musl-libc has been supported out of the box with for a while now. For pure Rust programs (or ones with small amounts of C handled by cargo), just write: cargo build --target=x86_64-unknown-linux-musl This will produce a 100% static binary that relies on nothing except the kernel. For programs which require external C libraries, it's a bit trickier, because you need a static version of those libraries built against musl-libc. For common libraries like OpenSSL and libpq, I have a Docker container that makes this easier: https://github.com/emk/rust-musl-builder https://github.com/emk/rust-musl-builder

nickpsecurity 9y ago

Both House OS in Haskell and SVA-OS handled that by cleanly hiding unsafe details behind an interface that could be used safely: http://programatica.cs.pdx.edu/House/ http://programatica.cs.pdx.edu/House/ https://llvm.org/pubs/2009-08-12-UsenixSecurity-SafeSVAOS.html https://llvm.org/pubs/2009-08-12-UsenixSecurity-SafeSVAOS.ht... So, that's not really a limitation. Worst-case scenario is proving those primitives correct with external tools whose preconditions and invariants are just checked by memory-safe code calling it. The above work shows worst case might not happen, though.

mtgx 9y ago

Well...maybe just the compiler? ;)