4 ms·

What exactly makes Rust attractive for low level stuff like this? Just wondering why Rust/D get lumped into OS and low level development in general when it see

What exactly makes Rust attractive for low level stuff like this? Just wondering why Rust/D get lumped into OS and low level development in general when it seems more syntactic than anything else. Do you have to manually manage memory in Rust?

xenadu02 11y ago
Because the Rust type system is memory-safe by default. It aims to have similar performance characteristics as C, but support higher level abstractions without compromising safety. If you don't understand why that is critically needed, I'd like to introduce you to /almost every RCE, buffer overflow, and stack smashing bug/ that has plagued us for the past 20 years.
- DonaldFisk 11y ago
  These are problems limited to C, C++, and assembly language. Even C programs can be kept reasonably secure by following the MISRA-C standard. Very secure and reliable operating systems have been written in existing languages such as ESPOL (MCP) and PL1 (MULTICS). There's also Zetalisp (Genera) and Oberon (Oberon), though they might not appeal if you dislike garbage collection. You might also consider Ada, designed with type and memory safety in mind and used in defence and avionics.
  - pcwalton 11y ago
    > Even C programs can be kept reasonably secure by following the MISRA-C standard. You might also consider Ada, designed with type and memory safety in mind and used in defence and avionics. The problem with these older systems is that they don't support heap allocation while remaining memory-safe. (A quick Google search seems to confirm that MISRA-C is in this category.) Or, if they do support heap allocation, they use a GC to manage it. Both of these restrictions are often considered undesirable for a kernel, except in specific embedded situations in which you can get away with statically allocating all your variables. Rust's lifetime system can ensure that you use malloc/free correctly, so it doesn't have to wall dynamic memory allocation off into unsafe code.
    - deleted 11y ago
      [deleted]
deleted 11y ago
[deleted]
pshc 11y ago
What do you mean by "more syntactic than anything else"?
pcwalton 11y ago
> Do you have to manually manage memory in Rust? Yes.
steveklabnik 11y ago
Rust was specifically designed as a language for doing low-level stuff. Or at least, to fit squarely in C's domain. Building an OS rather than applications still has some rough edges, but we're working on them. One of the reasons that I'm playing with OS stuff is to make sure we have a core team member to advocate for those features, as they are fairly niche. Most people are interacting with their OS, rather than writing one, so it's easy to put them off.
p0nce 11y ago
D offers a number of improvements in all directions: http://p0nce.github.io/d-idioms/#How-does-D-improve-on-C++17 http://p0nce.github.io/d-idioms/#How-does-D-improve-on-C++17? The fact that is has better syntax is almost anecdotal. Same story for Rust.
phil-opp 11y ago
Yes, you need to manually manage memory. But you can't do it wrong in Rust¹. It's an error at compile time. No use after free, no double free, and no dangling pointers are possible. The same applies to iterator invalidation and data races. Coupled with sane defaults for things such as array bounds checking and value initialization, it results in a great language to write a safe kernel. ¹: Well, you can mess up in `unsafe` blocks. It's basically an “I'm smarter than the compiler” annotation. You can use it when Rust can't guarantee that something is memory safe. For example writing to the VGA text buffer at the special address `0xb8000` is totally safe, but Rust can't know that. We try to minimize the number of `unsafe` and to double check each use, but of course you can make mistakes and accidentally violate memory safety. But even then you just need to look at `unsafe` blocks to find the bug, which is still a big improvement over languages such as C.
- tomjakubowski 11y ago
  > But even then you just need to look at `unsafe` blocks to find the bug, which is still a big improvement over languages such as C. This is often said about Rust, but isn't exactly true. If you'd like this to be true in your own code, you have to have be discipline and put all operations which may violate invariants required by unsafe code inside unsafe blocks, even if the compiler considers that code to be safe. Imagine you're implementing a type like Vec<T> that has a (raw) pointer 'buf: *mut T' to some slab of memory it owns, and another field 'length: usize'. A safe method of Vec<T> could make a mistake while mutating the length field and violate memory safety in a completely correct 'unsafe' block that assumes '[buf, buf + length)' is a valid range of memory.
  - ianhedoesit 11y ago
    Isn't this still a useful feature? You still at least have a better idea of where to look compared to C. With the feature of `unsafe` blocks in Rust you have points of entry to look at, and can be relatively sure that you only need to follow the several methods that are used in the `unsafe` block at most. In C you can happily have `some_function_1()` which causes the error you're trying to debug happen but is never called anywhere near where the error gets thrown. Tracking a semantic error in a call that's three calls away from where your unsafe block lives is much easier and structured than going through everything everywhere for any error from anything. (I haven't done much work at all in Rust and avoid `unsafe` - and `mut` and `.unwrap`, for that matter - like the plague when I do use it, so I'm very open to being wrong in my interpretation.) EDIT: I also think a reasonable comparison would be Haskell's `do` blocks. It's guaranteed that anything with side effects will live in a `do` block, but of course you can still have semantic errors. Again, you know exactly the "functions of interest" to look at if this is the case - there's no chance of `someFunc1` to be doing anything funny if it's never called by anything in the `do` block.
Manishearth 11y ago
> Do you have to manually manage memory in Rust? For some definition of "have to" and "manual". Rust uses RAII and ownership/borrowing for safe, performant memory management (no GC). At the same time, it doesn't automatically use memory. So, stuff goes on the stack unless I explicitly put it inside a heap-based thing (e.g. a box or a vector). These things clean up after themselves and can be used cleanly. Like Sam said here (https://news.ycombinator.com/item?id=10711997 https://news.ycombinator.com/item?id=10711997), "You have to think about it. You don't have to worry about it." At the same time, since you don't have to use the heap, and low level code isn't too hard to write in Rust, you can easily write non-allocating OS-level code if you wish. And with the appropriate (unsafely implemented, but verifiable) zero-cost abstractions, you can still have safe code at this level.

xenadu02 11y ago

Because the Rust type system is memory-safe by default. It aims to have similar performance characteristics as C, but support higher level abstractions without compromising safety. If you don't understand why that is critically needed, I'd like to introduce you to /almost every RCE, buffer overflow, and stack smashing bug/ that has plagued us for the past 20 years.

DonaldFisk 11y ago
These are problems limited to C, C++, and assembly language. Even C programs can be kept reasonably secure by following the MISRA-C standard. Very secure and reliable operating systems have been written in existing languages such as ESPOL (MCP) and PL1 (MULTICS). There's also Zetalisp (Genera) and Oberon (Oberon), though they might not appeal if you dislike garbage collection. You might also consider Ada, designed with type and memory safety in mind and used in defence and avionics.
- pcwalton 11y ago
  > Even C programs can be kept reasonably secure by following the MISRA-C standard. You might also consider Ada, designed with type and memory safety in mind and used in defence and avionics. The problem with these older systems is that they don't support heap allocation while remaining memory-safe. (A quick Google search seems to confirm that MISRA-C is in this category.) Or, if they do support heap allocation, they use a GC to manage it. Both of these restrictions are often considered undesirable for a kernel, except in specific embedded situations in which you can get away with statically allocating all your variables. Rust's lifetime system can ensure that you use malloc/free correctly, so it doesn't have to wall dynamic memory allocation off into unsafe code.
  - deleted 11y ago
    [deleted]

deleted 11y ago

[deleted]

pshc 11y ago

What do you mean by "more syntactic than anything else"?

pcwalton 11y ago

> Do you have to manually manage memory in Rust? Yes.

steveklabnik 11y ago

Rust was specifically designed as a language for doing low-level stuff. Or at least, to fit squarely in C's domain. Building an OS rather than applications still has some rough edges, but we're working on them. One of the reasons that I'm playing with OS stuff is to make sure we have a core team member to advocate for those features, as they are fairly niche. Most people are interacting with their OS, rather than writing one, so it's easy to put them off.

p0nce 11y ago

D offers a number of improvements in all directions: http://p0nce.github.io/d-idioms/#How-does-D-improve-on-C++17 http://p0nce.github.io/d-idioms/#How-does-D-improve-on-C++17? The fact that is has better syntax is almost anecdotal. Same story for Rust.

phil-opp 11y ago

Yes, you need to manually manage memory. But you can't do it wrong in Rust¹. It's an error at compile time. No use after free, no double free, and no dangling pointers are possible. The same applies to iterator invalidation and data races. Coupled with sane defaults for things such as array bounds checking and value initialization, it results in a great language to write a safe kernel. ¹: Well, you can mess up in `unsafe` blocks. It's basically an “I'm smarter than the compiler” annotation. You can use it when Rust can't guarantee that something is memory safe. For example writing to the VGA text buffer at the special address `0xb8000` is totally safe, but Rust can't know that. We try to minimize the number of `unsafe` and to double check each use, but of course you can make mistakes and accidentally violate memory safety. But even then you just need to look at `unsafe` blocks to find the bug, which is still a big improvement over languages such as C.

tomjakubowski 11y ago
> But even then you just need to look at `unsafe` blocks to find the bug, which is still a big improvement over languages such as C. This is often said about Rust, but isn't exactly true. If you'd like this to be true in your own code, you have to have be discipline and put all operations which may violate invariants required by unsafe code inside unsafe blocks, even if the compiler considers that code to be safe. Imagine you're implementing a type like Vec<T> that has a (raw) pointer 'buf: *mut T' to some slab of memory it owns, and another field 'length: usize'. A safe method of Vec<T> could make a mistake while mutating the length field and violate memory safety in a completely correct 'unsafe' block that assumes '[buf, buf + length)' is a valid range of memory.
- ianhedoesit 11y ago
  Isn't this still a useful feature? You still at least have a better idea of where to look compared to C. With the feature of `unsafe` blocks in Rust you have points of entry to look at, and can be relatively sure that you only need to follow the several methods that are used in the `unsafe` block at most. In C you can happily have `some_function_1()` which causes the error you're trying to debug happen but is never called anywhere near where the error gets thrown. Tracking a semantic error in a call that's three calls away from where your unsafe block lives is much easier and structured than going through everything everywhere for any error from anything. (I haven't done much work at all in Rust and avoid `unsafe` - and `mut` and `.unwrap`, for that matter - like the plague when I do use it, so I'm very open to being wrong in my interpretation.) EDIT: I also think a reasonable comparison would be Haskell's `do` blocks. It's guaranteed that anything with side effects will live in a `do` block, but of course you can still have semantic errors. Again, you know exactly the "functions of interest" to look at if this is the case - there's no chance of `someFunc1` to be doing anything funny if it's never called by anything in the `do` block.

Manishearth 11y ago

> Do you have to manually manage memory in Rust? For some definition of "have to" and "manual". Rust uses RAII and ownership/borrowing for safe, performant memory management (no GC). At the same time, it doesn't automatically use memory. So, stuff goes on the stack unless I explicitly put it inside a heap-based thing (e.g. a box or a vector). These things clean up after themselves and can be used cleanly. Like Sam said here (https://news.ycombinator.com/item?id=10711997 https://news.ycombinator.com/item?id=10711997), "You have to think about it. You don't have to worry about it." At the same time, since you don't have to use the heap, and low level code isn't too hard to write in Rust, you can easily write non-allocating OS-level code if you wish. And with the appropriate (unsafely implemented, but verifiable) zero-cost abstractions, you can still have safe code at this level.