14 ms·

Why Rust helps even if you have to use a lot of `unsafe`

101 points by chriskrycho 3y ago

lordnacho 3y ago
Do people generally say they need a lot of unsafe? Hasn't come up much for me. Also I'm not sure the cognitive overhead is really that big. The compiler helps a lot with its useful error messages.
- slashdev 3y ago
  Depends how much performance matters and what the problem domain is, and how low level it is. It can vary a lot.
- andrewfurey2003 3y ago
  Been using ash for vulkan so lots of unsafe here
- ActorNightly 3y ago
  If you chose Rust because you want speed that comes with natively compiled code, and you want optimal performance, in general you will probably come across a place where you need to dereference pointers without anything extra running.
  - jjnoakes 3y ago
    I haven't really had this experience. Usually a subtle reorg of how I iterate, for example, allows the compiler to eliminate bounds checks, effectively giving me full speed without unsafe in tight loops.
  - jamesmunns 3y ago
    For what it's worth: references in Rust (both `&` and `&mut`) are just pointers with stronger static (compile time only) analysis. They have zero runtime impact vs raw pointers. Even heap allocations like `Box` are still just pointers, that have destructors that run when they fall out of scope, but access is just a pointer dereference, the same as you could do with unsafe.
    - mtsr 3y ago
      Indeed. Raw pointers are mostly when either crossing FFI barriers (say, calling or being called by C) or when what you want to do can’t be expressed within Rust’s ownership rules without taking on overhead (Rc, cloning, indirections, etc) that’s unacceptable to your specific use case.
    - ActorNightly 3y ago
      If you are just working with data, you can avoid unsafe completely. The issue is that for performant code, you often have to tap into linux libc stuff which doesn't guarantee safety. For example, if you try to to write something with packet_mmap for fast processing of traffic, you will have to use unsafes. Look at the Firecracker codebase. There are a lot of unsafe calls into libc. There is also a runtime reference counter which takes up cycles since it has to operate at run time.
      - __jem 3y ago
        Okay, but using unsafe for FFI is entirely different than the claim that you can only achieve performance optimizations through "dereferencing pointers without anything extra running." The use of unsafe for FFI is unavoidable for obvious reasons.
  - sophacles 3y ago
    There's nothing "extra running" when you dereference a rust reference either, its just a pointer that follows the semantics that can be checked at compile time.
- EnergyAmy 3y ago
  I've used some unsafe, but only when using Rust for embedded programming. The amount of unsafe I have to use is surprisingly low, since a lot of what I'd have to use it for is wrapped up into libraries like RTIC or the various hal/pac board-specific libraries.
- sophacles 3y ago
  A lot of people say that they need it because they are trying to code C in rust. Often I see it as analogous to trying to learn an SOV spoken language but insisting on using SVO grammar still - sure you can muddle through to a mutual understanding with a conversant, but it's high friction and has a lot of confusion along the way. I'm not trying to invalidate all claims that unsafe is necessary, nor even that there are cases where unsafe is quite helpful for performance. It's just that a little unsafe used deliberately and thoughtfully can go a long way, and accomplish those goals in a largely safe codebase.
- chriskrycho 3y ago
  I mostly hear it from folks who are explaining why they use Zig or Odin instead, or aren't why they are not interested in switching from C or C++. Re: overhead—I think the compiler helps a ton here, but I do think that it makes a lot of memory-related concerns explicit where they are usually implicit, and dealing with lifetimes can be quite burdensome. Now: I also think that those concerns still exist when you're using other systems-level languages, and that the tradeoff therefore often ends up looking like "Deal with it up front, or deal with it on hard mode later."
  - josephg 3y ago
    I’ve spent a fair bit of time in rust and written more than my fair share of unsafe code. Unsafe is super necessary sometimes. But there are definitely reasons I’d consider using Zig or C or something if I needed a lot of unsafe in my project. - From a language point of view, the syntax for dereferencing pointers in rust is awful. I don’t mind the unsafe keyword. But in rust you can’t do a->b. You need to write (*a).b. And that gets increasingly awful as you have more complex pointer expressions. Unsafe code in rust looks significantly uglier than the equivalent C. - The Rust community - particularly on Reddit - seems to get very snide annd condescending about programmers using any amount of unsafe in their rust code. Even when it’s totally necessary. I’m not convinced the community will is there to fix rust’s bad pointer syntax, or other missteps with unsafe. Apparently having unreadable unsafe code is my atonement for wanting high performance code. Never mind that the harder my code is to read, the more likely I am to make mistakes. And in exactly the part of my codebase where bugs would be the most damaging to safety! Argh. I love rust. I love the fresh ideas it’s brought to programming as a whole. But I can’t wait for some other, newer languages to iterate on these features & the language syntax some more. It’s not the top of the mountain.
    - estebank 3y ago
      > But in rust you can’t do a->b. You need to write (*a).b. For what is worth, I think that we'll add that at some point.
    - mtsr 3y ago
      I think some of the snideness is probably due to different senses of what amount of unsafe is actually needed. There’s so much premature optimization in many cases that wouldn’t even come up if the language were Java, Go or whatever. It’s totally fine to initially accept some known overhead from cloning or Rc if you don’t know yet it it’s going to significantly impact performance.
- wokwokwok 3y ago
  Yes. It’s pretty common to need to interact with existing APIs or libraries that only have a c interface. Fortunately, rust is good at that. Unfortunately, it’s impossible to do without unsafe code. You can wrap libraries in safe wrappers, or if you’re lucky someone has a wrapper you can use… …but I’ve heard it several times for this reason, especially in shops that have existing internal code bases (Ie. 0% chance of an existing safe wrapper) and want to incrementally adopt rust. Never heard someone say it’s because of perf tho; that sounds like a strawman argument to me; if your code is seriously too slow because of the bounds checking you’re doing something crazy and probably want to be using a gpu instead. It’s usually io bound, and unsafe code doesn’t fix that.
- zer8k 3y ago
  > The compiler helps a lot with its useful error messages. As someone trying very hard to love Rust this is just simply not the case. It takes the shortest path to most answers. The most trivial example where the compiler becomes less helpful is a self-balancing binary tree. It will constantly suggest explicit lifetimes and other magic when in reality you probably want `Box<Rc<..>>`. It will suggest Boxing for recursive types...but that's about the end of it. In many cases you're better off knowing what to do. But the problem is knowing what to do requires you to have done what you need...and hence the Rust Ouroboros is born. The nice thing about unsafe code (C, for example) is the cognitive overhead is initially very low. That allows you to get an idea down and then worry about stuff later. Of course, if you have a gun to your head from management you never get to the "worry about stuff later" stage and introduce all sorts of nasty bugs. Rust frontloads all of this making it sometimes very difficult to use without having everything already thought out way ahead of time. Both a benefit and a curse.
  - mtsr 3y ago
    It’s just the compiler assuming you know what you’re doing. You’re building this data structure on the stack, so it’ll point out where you’re not sufficiently explaining to it how the lifetimes of your code are safe. And even if you build it on the heap (using Box) you’ll still need to get the ownership right (using e.g. Rc). There’s only so many variations that work for something like trees (especially with doubly linked nodes) in any language, without either inherently unsafe lifetimes (because the particular language can’t express them as part of the API) or some kind of indirection (such as ECS style indices). Or garbage collection or ref counting, but one probably uses Rust to be able to control that kind of overhead instead of having to use whatever the language creators picked. If you want to learn about this, instead of using a ‘blessed’ implementation, read “too many linked lists” https://rust-unofficial.github.io/too-many-lists/ https://rust-unofficial.github.io/too-many-lists/. As for the front-loading comment: yes, Rust requires you to get some things right from the start, particularly ownership and lifetimes. If you want to work in Rust like in a scripting language, you can just use Box, Rc and clone() everywhere. But getting it right without those might require a significantly different API. Which isn’t that much different with your rough C implementation. To get any kind of safe API there will require just as many changes (if not more).
- jackmott42 3y ago
  Depends what you are doing, you might have 0.0 lines of unsafe, or you might be doing some custom SIMD string parsing code and its 100% unsafe. Or anywhere in between.
- insanitybit 3y ago
  I don't really need unsafe but I do enjoy using it. In particular, `unwrap_unchecked` is nice for when I'm 100% sure that an unwrap can't fail. I don't see any reason to leave that to the compiler to figure out.
- empath-nirvana 3y ago
  I've put a bunch of rust code into production and I've never written unsafe code. It primarily comes up when you're 1) interfacing with hardware 2) interfacing with C APIs 3) doing something weird with pointers.
- jandrewrogers 3y ago
  Most software does not need a lot of unsafe. There are a handful of software domains where you do need a lot of unsafe if you care about performance, but in these cases languages like C++ are often more ergonomic anyway.
fnfjfk 3y ago
This article could be 1 sentence: “it has sum types” :) I find programming in a language without sum types is like rock climbing in ankle weights. I just can’t model many types of data correctly, and I hate using product types for something that is clearly a sum type. Even if Rust didn’t make any safety guarantees over other languages (although sum types provide safety guarantees through exhaustiveness!) I would still think Rust helps a lot because of that. I guess Swift and Kotlin are basically “it has sum types” languages too, although they both come with baggage (Apple-only in practice, JVM).
- tines 3y ago
  This is a huge deal, I find it so difficult to choose a language that doesn't have sum types and destructuring via pattern matching for the kinds of projects I like to work on (compiler/interpreter implementation).
  - flohofwoe 3y ago
    IMHO the one downside with sum types in a performance-oriented systems programming language (or as I prefer to call them 'tagged unions' because that makes it clearer how they actually look like in memory) is that they are inherently cache-unfriendly, because they always take up as much space as the largest wrapped type (plus 8 bytes for the tag). When you have an array of those you need to be aware of the potential 'waste' between array items. (now of course an automatic transformation into a structure-of-arrays would be pretty cool, I actually wonder what Zig's MultiArrayList does with tagged unions). PS: in general I agree though, for instance in Typescript they are fantastic, but there I usually don't care much about memory layout and performance.
    - eddd-ddde 3y ago
      If you are dealing with always-same-type arrays then you can express so as a sum type of arrays, not an array of sum types.
    - widdershins 3y ago
      That depends on the sizes of the objects. If the objects in the union are generally small, cache coherence is better than if you have polymorphic objects allocated on the heap.
pornel 3y ago
The second point — ability to wrap `unsafe` code in safe APIs — is especially important. Ability to isolate high-risk "hold my beer" code from boring glue code has many nice side effects. In a team, you can agree who can write and review `unsafe` code, and not be afraid that your junior team members will cause some horror heisenbug (there can always be other types of higher-level bugs in safe code, but safe code doesn't cause them, you'd have them anyway, on top of memory issues). People often move abstractions built from unsafe code (like clever data structures, concurrency primitives, OS or C library wrappers) into separate crates. This makes it easier to thoroughly test and fuzz just the unsafe code, without having to test a larger project as a whole (once you ensure your safe API is indeed upholding safety invariants, the compiler will prove all uses of that code are safe too).
- throwaway894345 3y ago
  I'm curious if a linter + the new C23 annotation syntax could be used for a similar purpose in C? EDIT: I'm rate limited. If we assume an `[[unsafe]]` attribute would only apply to things like pointer arithmetic and unchecked array access (not lifetimes), could such a linter exist?
  - Conscat 3y ago
    Unfortunately the attributes are not nearly rich enough. There are many proposals for massively increasing the information that you can give C compilers, such as the lifetime safety profile and n3005, but it's a very difficult problem to do this in a language that makes pointer aliasing and pointer arithmetic very easy. n3005: https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3005.pdf https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3005.pdf
- galangalalgol 3y ago
  Even of the error originates in unsafe code, the UB can manifest elsewhere. The optimizer makes assumptions that you can easily invalidate in an unsafe block leading to UB. In many ways it is mich easier to introduce UB in an umsafe block than it is in C++. But at the very least you know the UB originated in one of your unsafe blocks. Its just figuring out which one. I very rarely use unsafe outside bindings though, so that helps too.
amelius 3y ago
Apart from "safe" and "unsafe" we should also have "trusted" and "untrusted". Software is getting complicated and supply chain attacks are real.
- GuB-42 3y ago
  Isn't it what "git blame" is for?
  - amelius 3y ago
    Basically, if you use a library that you don't trust, you have to run it inside a sandbox so that it cannot do stuff that you don't want it to do, e.g. access the entire filesystem or peek/poke in another library's memory. The sandbox should be part of the language.
- massysett 3y ago
  Safe Haskell sort of addressed this, though I didn’t find it useful in practice. https://downloads.haskell.org/~ghc/7.8.4/docs/html/users_guide/safe-haskell.html https://downloads.haskell.org/~ghc/7.8.4/docs/html/users_gui...
- ReactiveJelly 3y ago
  Agreed. 5 years ago I thought that Haskell's IO monad was genius. I hope it trickles into a mainstream language before I retire. Sure, it's an extra arg to every non-pure function. So what? Write more pure functions! If a function shouldn't use the network, pass it an IO that can't do so. If it shouldn't touch disk, pass it an IO that can't do so. This may complicated printf logging... Or maybe there is an IO that only allows passing messages to a logging module, and the logging module can write to disk. So the logging module gets more auditing attention than pure functions.
  - empath-nirvana 3y ago
    I think where this is headed is algebraic effects, not using an io monad.
- 3y ago
hydroid7 3y ago
Having `derive(Debug)` is the dream of all C devs.
- constantcrying 3y ago
  Not having to deal with a completely asinine build chain is another.
deleted 3y ago
[deleted]
FrustratedMonky 3y ago
Looking up links from the poster, Chris. I didn't realize languages should have theme songs until now. Think all languages should have a theme. https://soundcloud.com/chriskrycho/new-rustacean-theme https://soundcloud.com/chriskrycho/new-rustacean-theme Other Languages: C++, some classic rock? C#, some re-cover of a classic rock song by a corporate group? Python, dubstep?
- indy 3y ago
  PHP: A one-man band trying to play a song while being attacked by an angry goose.
zengid 3y ago
Babe, wake up, a new Rust post from Chris Krycho just dropped.
ActorNightly 3y ago
>When dealing in unsafe code — whether in Rust unsafe blocks or in all the code in languages like C and C++3 — the responsibility falls to the programmer to write code which is still safe. This is possible! It is simply very difficult. This is an interesting Catch 22. If you accept this premise as true, then Rust becomes ultimately pointless. Because the prescription would be to become a very good programmer and learn how to write memory safe code, at which point you really don't need any other language except C. You will likely write memorypools that have mechanisms to make sure that you never access memory you are not supposed to. An alternative prescription is to dedicate effort to making higher level language compilers/transpilers better. That way everyone codes in Python, the compiler recognizes common access patterns, and does iterative optimization (probably ML powered) to get you the most optimal machine code. If you accept the premise as false, i.e writing memory safe code is easy, then Rust doesn't really offer any benefits.
- constantcrying 3y ago
  >Because the prescription would be to become a very good programmer and learn how to write memory safe code, at which point you really don't need any other language except C. Yes. Indeed you don't need any other language than C. Every program could be written in it. The point of rust is to enforce strict rules on code that guarantees certain behaviour, but still allow operations which could violate those behaviours. >Because the prescription would be to become a very good programmer and learn how to write memory safe code No. The prescription is to be careful when dealing with memory. This is true for literally any language, the goal of rust is that the compiler can take away some of that workload.
- adgjlsfhk1 3y ago
  > Because the prescription would be to become a very good programmer and learn how to write memory safe code The problem isn't becoming a very good programmer. That's not difficult. The problem is becoming a very good programmer who is never tired or rushed. Any correct code in C will have implicit rules about how the memory is managed that are either written in comments or reside in the programmer's head. This doesn't work because eventually some will make a code change without understanding fully the implicit rule (often this person is the same person who figured out the rule).
constantcrying 3y ago
One misconception is also that in unsafe block everything goes and all checks for memory are disabled. This just isn't true. Unsafe rust allows you to make operations which could invalidate all of rusts memory guarantees by granting you access to raw pointers, but that is (almost) it. It doesn't mean that rust suddenly does not enforce any of its rules. I also can not believe that that much code actually needs raw pointer operations if the developer considers alternatives.
zengid 3y ago
> I do not take Rust to be remotely the final word in this space. It is the first memory safe systems language to be successful at industrial scale — not the last! I think this side note is the most interesting tidbit, because I agree. I think Rust is a beautiful thing, but it's starting to get more competition now, and the "memory safe systems language" space is really starting to heat up. I'm particularly interested in seeing whether Val becomes its own thing, or if Swift and Rust (or another language) just absorb it's good ideas. Rust is amazing and the community is full of great people, but there are a lot of rough edges and places that feel awkward for new folks. I appreciate seeing all of the constructive criticisms that have come out against Rust lately, and for folks like Tim McNamara who are pointing out all of Rust's rough edges in hopes of improving the language. If I were able to contribute more to the project, that is where I would focus, personally. Also, Rust needs to get it's leadership issues sorted out. More and more projects are starting to sneak up on Rust, and if the folks in charge of the project keep fumbling the bag and causing rifts in the community (like with the Rust licensing, and driving phantomderp away), it's going to keep folks from wanting to do the hard work to improve the language.
- c_crank 3y ago
  Ignoring the long usage of Ada in the (military) industrial scale, or predecessors.
  - nicce 3y ago
    A bit late reply, but that is the core problem for adaption. It is very commercial language after all. There is not big open source community. On military and aerospace level there is a lot of money to throw for language tooling. For indiciduals, not much is possible.
  - retrac 3y ago
    Ada still has killer features that haven't made it into C-like languages. One of my favourite is the split between the abstract/programmer's view of a type, and the machine implementation/representation of the type. There's no bit twiddling, no masking with 0xFFFF..., no worries about order in a struct or packing. If you have a 4 bit value, and want to store it in bits 27 ... 31 of a word, Ada lets you specify that. From the programmer's perspective, you work on the abstract type, perhaps defined as range 0 .. 15 in this case. (It can be any type that fits into the number of bits available, by the way.) It packs/unpacks transparently behind the scenes. And if you overflow that 0 .. 15 range it will raise an overflow/underflow exception. If you were trying to cram that 0 .. 15 range into only 3 bits, that'll raise a compile time error. Ever since I've encountered that its absence from the C family seems like a very obvious oversight.
pjmlp 3y ago
System languages with unsafe code blocks exist for decades, with ESPOL in 1961 being one of the first ones. In fact, had languages like Modula-2 and Ada not lost to UNIX/C/C++ hegemony, this would already be quite common. It is kind of tragic that we still need to keep discussing the value of this.

101 points by chriskrycho 3y ago

Do people generally say they need a lot of unsafe? Hasn't come up much for me. Also I'm not sure the cognitive overhead is really that big. The compiler helps a lot with its useful error messages.

slashdev 3y ago
Depends how much performance matters and what the problem domain is, and how low level it is. It can vary a lot.
andrewfurey2003 3y ago
Been using ash for vulkan so lots of unsafe here
ActorNightly 3y ago
If you chose Rust because you want speed that comes with natively compiled code, and you want optimal performance, in general you will probably come across a place where you need to dereference pointers without anything extra running.
- jjnoakes 3y ago
  I haven't really had this experience. Usually a subtle reorg of how I iterate, for example, allows the compiler to eliminate bounds checks, effectively giving me full speed without unsafe in tight loops.
- jamesmunns 3y ago
  For what it's worth: references in Rust (both `&` and `&mut`) are just pointers with stronger static (compile time only) analysis. They have zero runtime impact vs raw pointers. Even heap allocations like `Box` are still just pointers, that have destructors that run when they fall out of scope, but access is just a pointer dereference, the same as you could do with unsafe.
  - mtsr 3y ago
    Indeed. Raw pointers are mostly when either crossing FFI barriers (say, calling or being called by C) or when what you want to do can’t be expressed within Rust’s ownership rules without taking on overhead (Rc, cloning, indirections, etc) that’s unacceptable to your specific use case.
  - ActorNightly 3y ago
    If you are just working with data, you can avoid unsafe completely. The issue is that for performant code, you often have to tap into linux libc stuff which doesn't guarantee safety. For example, if you try to to write something with packet_mmap for fast processing of traffic, you will have to use unsafes. Look at the Firecracker codebase. There are a lot of unsafe calls into libc. There is also a runtime reference counter which takes up cycles since it has to operate at run time.
    - __jem 3y ago
      Okay, but using unsafe for FFI is entirely different than the claim that you can only achieve performance optimizations through "dereferencing pointers without anything extra running." The use of unsafe for FFI is unavoidable for obvious reasons.
- sophacles 3y ago
  There's nothing "extra running" when you dereference a rust reference either, its just a pointer that follows the semantics that can be checked at compile time.
EnergyAmy 3y ago
I've used some unsafe, but only when using Rust for embedded programming. The amount of unsafe I have to use is surprisingly low, since a lot of what I'd have to use it for is wrapped up into libraries like RTIC or the various hal/pac board-specific libraries.
sophacles 3y ago
A lot of people say that they need it because they are trying to code C in rust. Often I see it as analogous to trying to learn an SOV spoken language but insisting on using SVO grammar still - sure you can muddle through to a mutual understanding with a conversant, but it's high friction and has a lot of confusion along the way. I'm not trying to invalidate all claims that unsafe is necessary, nor even that there are cases where unsafe is quite helpful for performance. It's just that a little unsafe used deliberately and thoughtfully can go a long way, and accomplish those goals in a largely safe codebase.
chriskrycho 3y ago
I mostly hear it from folks who are explaining why they use Zig or Odin instead, or aren't why they are not interested in switching from C or C++. Re: overhead—I think the compiler helps a ton here, but I do think that it makes a lot of memory-related concerns explicit where they are usually implicit, and dealing with lifetimes can be quite burdensome. Now: I also think that those concerns still exist when you're using other systems-level languages, and that the tradeoff therefore often ends up looking like "Deal with it up front, or deal with it on hard mode later."
- josephg 3y ago
  I’ve spent a fair bit of time in rust and written more than my fair share of unsafe code. Unsafe is super necessary sometimes. But there are definitely reasons I’d consider using Zig or C or something if I needed a lot of unsafe in my project. - From a language point of view, the syntax for dereferencing pointers in rust is awful. I don’t mind the unsafe keyword. But in rust you can’t do a->b. You need to write (*a).b. And that gets increasingly awful as you have more complex pointer expressions. Unsafe code in rust looks significantly uglier than the equivalent C. - The Rust community - particularly on Reddit - seems to get very snide annd condescending about programmers using any amount of unsafe in their rust code. Even when it’s totally necessary. I’m not convinced the community will is there to fix rust’s bad pointer syntax, or other missteps with unsafe. Apparently having unreadable unsafe code is my atonement for wanting high performance code. Never mind that the harder my code is to read, the more likely I am to make mistakes. And in exactly the part of my codebase where bugs would be the most damaging to safety! Argh. I love rust. I love the fresh ideas it’s brought to programming as a whole. But I can’t wait for some other, newer languages to iterate on these features & the language syntax some more. It’s not the top of the mountain.
  - estebank 3y ago
    > But in rust you can’t do a->b. You need to write (*a).b. For what is worth, I think that we'll add that at some point.
  - mtsr 3y ago
    I think some of the snideness is probably due to different senses of what amount of unsafe is actually needed. There’s so much premature optimization in many cases that wouldn’t even come up if the language were Java, Go or whatever. It’s totally fine to initially accept some known overhead from cloning or Rc if you don’t know yet it it’s going to significantly impact performance.
wokwokwok 3y ago
Yes. It’s pretty common to need to interact with existing APIs or libraries that only have a c interface. Fortunately, rust is good at that. Unfortunately, it’s impossible to do without unsafe code. You can wrap libraries in safe wrappers, or if you’re lucky someone has a wrapper you can use… …but I’ve heard it several times for this reason, especially in shops that have existing internal code bases (Ie. 0% chance of an existing safe wrapper) and want to incrementally adopt rust. Never heard someone say it’s because of perf tho; that sounds like a strawman argument to me; if your code is seriously too slow because of the bounds checking you’re doing something crazy and probably want to be using a gpu instead. It’s usually io bound, and unsafe code doesn’t fix that.
zer8k 3y ago
> The compiler helps a lot with its useful error messages. As someone trying very hard to love Rust this is just simply not the case. It takes the shortest path to most answers. The most trivial example where the compiler becomes less helpful is a self-balancing binary tree. It will constantly suggest explicit lifetimes and other magic when in reality you probably want `Box<Rc<..>>`. It will suggest Boxing for recursive types...but that's about the end of it. In many cases you're better off knowing what to do. But the problem is knowing what to do requires you to have done what you need...and hence the Rust Ouroboros is born. The nice thing about unsafe code (C, for example) is the cognitive overhead is initially very low. That allows you to get an idea down and then worry about stuff later. Of course, if you have a gun to your head from management you never get to the "worry about stuff later" stage and introduce all sorts of nasty bugs. Rust frontloads all of this making it sometimes very difficult to use without having everything already thought out way ahead of time. Both a benefit and a curse.
- mtsr 3y ago
  It’s just the compiler assuming you know what you’re doing. You’re building this data structure on the stack, so it’ll point out where you’re not sufficiently explaining to it how the lifetimes of your code are safe. And even if you build it on the heap (using Box) you’ll still need to get the ownership right (using e.g. Rc). There’s only so many variations that work for something like trees (especially with doubly linked nodes) in any language, without either inherently unsafe lifetimes (because the particular language can’t express them as part of the API) or some kind of indirection (such as ECS style indices). Or garbage collection or ref counting, but one probably uses Rust to be able to control that kind of overhead instead of having to use whatever the language creators picked. If you want to learn about this, instead of using a ‘blessed’ implementation, read “too many linked lists” https://rust-unofficial.github.io/too-many-lists/ https://rust-unofficial.github.io/too-many-lists/. As for the front-loading comment: yes, Rust requires you to get some things right from the start, particularly ownership and lifetimes. If you want to work in Rust like in a scripting language, you can just use Box, Rc and clone() everywhere. But getting it right without those might require a significantly different API. Which isn’t that much different with your rough C implementation. To get any kind of safe API there will require just as many changes (if not more).
jackmott42 3y ago
Depends what you are doing, you might have 0.0 lines of unsafe, or you might be doing some custom SIMD string parsing code and its 100% unsafe. Or anywhere in between.
insanitybit 3y ago
I don't really need unsafe but I do enjoy using it. In particular, `unwrap_unchecked` is nice for when I'm 100% sure that an unwrap can't fail. I don't see any reason to leave that to the compiler to figure out.
empath-nirvana 3y ago
I've put a bunch of rust code into production and I've never written unsafe code. It primarily comes up when you're 1) interfacing with hardware 2) interfacing with C APIs 3) doing something weird with pointers.
jandrewrogers 3y ago
Most software does not need a lot of unsafe. There are a handful of software domains where you do need a lot of unsafe if you care about performance, but in these cases languages like C++ are often more ergonomic anyway.

fnfjfk 3y ago

This article could be 1 sentence: “it has sum types” :) I find programming in a language without sum types is like rock climbing in ankle weights. I just can’t model many types of data correctly, and I hate using product types for something that is clearly a sum type. Even if Rust didn’t make any safety guarantees over other languages (although sum types provide safety guarantees through exhaustiveness!) I would still think Rust helps a lot because of that. I guess Swift and Kotlin are basically “it has sum types” languages too, although they both come with baggage (Apple-only in practice, JVM).

tines 3y ago
This is a huge deal, I find it so difficult to choose a language that doesn't have sum types and destructuring via pattern matching for the kinds of projects I like to work on (compiler/interpreter implementation).
- flohofwoe 3y ago
  IMHO the one downside with sum types in a performance-oriented systems programming language (or as I prefer to call them 'tagged unions' because that makes it clearer how they actually look like in memory) is that they are inherently cache-unfriendly, because they always take up as much space as the largest wrapped type (plus 8 bytes for the tag). When you have an array of those you need to be aware of the potential 'waste' between array items. (now of course an automatic transformation into a structure-of-arrays would be pretty cool, I actually wonder what Zig's MultiArrayList does with tagged unions). PS: in general I agree though, for instance in Typescript they are fantastic, but there I usually don't care much about memory layout and performance.
  - eddd-ddde 3y ago
    If you are dealing with always-same-type arrays then you can express so as a sum type of arrays, not an array of sum types.
  - widdershins 3y ago
    That depends on the sizes of the objects. If the objects in the union are generally small, cache coherence is better than if you have polymorphic objects allocated on the heap.

pornel 3y ago

The second point — ability to wrap `unsafe` code in safe APIs — is especially important. Ability to isolate high-risk "hold my beer" code from boring glue code has many nice side effects. In a team, you can agree who can write and review `unsafe` code, and not be afraid that your junior team members will cause some horror heisenbug (there can always be other types of higher-level bugs in safe code, but safe code doesn't cause them, you'd have them anyway, on top of memory issues). People often move abstractions built from unsafe code (like clever data structures, concurrency primitives, OS or C library wrappers) into separate crates. This makes it easier to thoroughly test and fuzz just the unsafe code, without having to test a larger project as a whole (once you ensure your safe API is indeed upholding safety invariants, the compiler will prove all uses of that code are safe too).

throwaway894345 3y ago
I'm curious if a linter + the new C23 annotation syntax could be used for a similar purpose in C? EDIT: I'm rate limited. If we assume an `[[unsafe]]` attribute would only apply to things like pointer arithmetic and unchecked array access (not lifetimes), could such a linter exist?
- Conscat 3y ago
  Unfortunately the attributes are not nearly rich enough. There are many proposals for massively increasing the information that you can give C compilers, such as the lifetime safety profile and n3005, but it's a very difficult problem to do this in a language that makes pointer aliasing and pointer arithmetic very easy. n3005: https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3005.pdf https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3005.pdf
galangalalgol 3y ago
Even of the error originates in unsafe code, the UB can manifest elsewhere. The optimizer makes assumptions that you can easily invalidate in an unsafe block leading to UB. In many ways it is mich easier to introduce UB in an umsafe block than it is in C++. But at the very least you know the UB originated in one of your unsafe blocks. Its just figuring out which one. I very rarely use unsafe outside bindings though, so that helps too.

amelius 3y ago

Apart from "safe" and "unsafe" we should also have "trusted" and "untrusted". Software is getting complicated and supply chain attacks are real.

GuB-42 3y ago
Isn't it what "git blame" is for?
- amelius 3y ago
  Basically, if you use a library that you don't trust, you have to run it inside a sandbox so that it cannot do stuff that you don't want it to do, e.g. access the entire filesystem or peek/poke in another library's memory. The sandbox should be part of the language.
massysett 3y ago
Safe Haskell sort of addressed this, though I didn’t find it useful in practice. https://downloads.haskell.org/~ghc/7.8.4/docs/html/users_guide/safe-haskell.html https://downloads.haskell.org/~ghc/7.8.4/docs/html/users_gui...
ReactiveJelly 3y ago
Agreed. 5 years ago I thought that Haskell's IO monad was genius. I hope it trickles into a mainstream language before I retire. Sure, it's an extra arg to every non-pure function. So what? Write more pure functions! If a function shouldn't use the network, pass it an IO that can't do so. If it shouldn't touch disk, pass it an IO that can't do so. This may complicated printf logging... Or maybe there is an IO that only allows passing messages to a logging module, and the logging module can write to disk. So the logging module gets more auditing attention than pure functions.
- empath-nirvana 3y ago
  I think where this is headed is algebraic effects, not using an io monad.
3y ago

hydroid7 3y ago

Having `derive(Debug)` is the dream of all C devs.

constantcrying 3y ago
Not having to deal with a completely asinine build chain is another.

deleted 3y ago

[deleted]

FrustratedMonky 3y ago

Looking up links from the poster, Chris. I didn't realize languages should have theme songs until now. Think all languages should have a theme. https://soundcloud.com/chriskrycho/new-rustacean-theme https://soundcloud.com/chriskrycho/new-rustacean-theme Other Languages: C++, some classic rock? C#, some re-cover of a classic rock song by a corporate group? Python, dubstep?

indy 3y ago
PHP: A one-man band trying to play a song while being attacked by an angry goose.

zengid 3y ago

Babe, wake up, a new Rust post from Chris Krycho just dropped.

ActorNightly 3y ago

>When dealing in unsafe code — whether in Rust unsafe blocks or in all the code in languages like C and C++3 — the responsibility falls to the programmer to write code which is still safe. This is possible! It is simply very difficult. This is an interesting Catch 22. If you accept this premise as true, then Rust becomes ultimately pointless. Because the prescription would be to become a very good programmer and learn how to write memory safe code, at which point you really don't need any other language except C. You will likely write memorypools that have mechanisms to make sure that you never access memory you are not supposed to. An alternative prescription is to dedicate effort to making higher level language compilers/transpilers better. That way everyone codes in Python, the compiler recognizes common access patterns, and does iterative optimization (probably ML powered) to get you the most optimal machine code. If you accept the premise as false, i.e writing memory safe code is easy, then Rust doesn't really offer any benefits.

constantcrying 3y ago
>Because the prescription would be to become a very good programmer and learn how to write memory safe code, at which point you really don't need any other language except C. Yes. Indeed you don't need any other language than C. Every program could be written in it. The point of rust is to enforce strict rules on code that guarantees certain behaviour, but still allow operations which could violate those behaviours. >Because the prescription would be to become a very good programmer and learn how to write memory safe code No. The prescription is to be careful when dealing with memory. This is true for literally any language, the goal of rust is that the compiler can take away some of that workload.
adgjlsfhk1 3y ago
> Because the prescription would be to become a very good programmer and learn how to write memory safe code The problem isn't becoming a very good programmer. That's not difficult. The problem is becoming a very good programmer who is never tired or rushed. Any correct code in C will have implicit rules about how the memory is managed that are either written in comments or reside in the programmer's head. This doesn't work because eventually some will make a code change without understanding fully the implicit rule (often this person is the same person who figured out the rule).

constantcrying 3y ago

One misconception is also that in unsafe block everything goes and all checks for memory are disabled. This just isn't true. Unsafe rust allows you to make operations which could invalidate all of rusts memory guarantees by granting you access to raw pointers, but that is (almost) it. It doesn't mean that rust suddenly does not enforce any of its rules. I also can not believe that that much code actually needs raw pointer operations if the developer considers alternatives.

zengid 3y ago

> I do not take Rust to be remotely the final word in this space. It is the first memory safe systems language to be successful at industrial scale — not the last! I think this side note is the most interesting tidbit, because I agree. I think Rust is a beautiful thing, but it's starting to get more competition now, and the "memory safe systems language" space is really starting to heat up. I'm particularly interested in seeing whether Val becomes its own thing, or if Swift and Rust (or another language) just absorb it's good ideas. Rust is amazing and the community is full of great people, but there are a lot of rough edges and places that feel awkward for new folks. I appreciate seeing all of the constructive criticisms that have come out against Rust lately, and for folks like Tim McNamara who are pointing out all of Rust's rough edges in hopes of improving the language. If I were able to contribute more to the project, that is where I would focus, personally. Also, Rust needs to get it's leadership issues sorted out. More and more projects are starting to sneak up on Rust, and if the folks in charge of the project keep fumbling the bag and causing rifts in the community (like with the Rust licensing, and driving phantomderp away), it's going to keep folks from wanting to do the hard work to improve the language.

c_crank 3y ago
Ignoring the long usage of Ada in the (military) industrial scale, or predecessors.
- nicce 3y ago
  A bit late reply, but that is the core problem for adaption. It is very commercial language after all. There is not big open source community. On military and aerospace level there is a lot of money to throw for language tooling. For indiciduals, not much is possible.
- retrac 3y ago
  Ada still has killer features that haven't made it into C-like languages. One of my favourite is the split between the abstract/programmer's view of a type, and the machine implementation/representation of the type. There's no bit twiddling, no masking with 0xFFFF..., no worries about order in a struct or packing. If you have a 4 bit value, and want to store it in bits 27 ... 31 of a word, Ada lets you specify that. From the programmer's perspective, you work on the abstract type, perhaps defined as range 0 .. 15 in this case. (It can be any type that fits into the number of bits available, by the way.) It packs/unpacks transparently behind the scenes. And if you overflow that 0 .. 15 range it will raise an overflow/underflow exception. If you were trying to cram that 0 .. 15 range into only 3 bits, that'll raise a compile time error. Ever since I've encountered that its absence from the C family seems like a very obvious oversight.

pjmlp 3y ago

System languages with unsafe code blocks exist for decades, with ESPOL in 1961 being one of the first ones. In fact, had languages like Modula-2 and Ada not lost to UNIX/C/C++ hegemony, this would already be quite common. It is kind of tragic that we still need to keep discussing the value of this.