8 ms·

> In fact the pattern described in the article is a common pattern in Rust and I make use of it all the time; the library for making use of it is `slotmap`. Sl

• dralley 3y ago
  > Slotmap uses unsafe everywhere, it's a memory usage pattern not supported by the borrow checker. Is disabling the borrow checker really a common pattern in Rust? Wrapping "unsafe" code in a safe interface is a common pattern in Rust, yes. There is absolutely nothing wrong with using "unsafe" so long as you are diligent about checking invariants, and keep it contained as much as possible. Obviously the standard library uses some "unsafe" as well, for instance. "unsafe" just means "safe but the compiler cannot verify it". Unsafe does not disable the borrow checker, though. All of the restrictions of safe Rust still apply. All "unsafe" does is unlock the ability to use raw pointers and a few other constructs. https://doc.rust-lang.org/book/ch19-01-unsafe-rust.html#unsafe-superpowers https://doc.rust-lang.org/book/ch19-01-unsafe-rust.html#unsa...
  • dxhdr 3y ago
    It's essentially a "user-space" memory allocator with it's own use-after-free and double-free checks, apparently because the language implementation isn't adequate. If anything it just reinforces the articles point that "borrow checking is incompatible with some useful patterns and optimizations."
    • junon 3y ago
      Eh? This is a wild take. How do you draw the conclusion the default implementation is inadequate?
      • dymk 3y ago
        Because something like slotmap has to use `unsafe` to get around the inadequacies of the borrow checker...
        • burntsushi 3y ago
          A downside for sure, but one that, at least in this specific example, has limited downsides. If you can button it up into a safe abstraction that you can share with others, then I don't really see what the huge problem is. The fact that you might need to write `unsafe` inside of a well optimized data structure isn't a weakness of Rust, it's the entire point: you use it to encapsulate an unsafe core within a safe interface. The standard library is full of these things. Now if you're trying to do something that you can't button up into a safe abstraction for others to use, then that's a different story.
        • orlp 3y ago
          Author of slotmap here. There is absolutely no need for unsafe in slotmap. I chose to use unsafe (wrapped in a safe API) to reduce memory usage using intrusive linked freelists. If done using safe Rust this would involve `enum`s that would take up extra space.
          • pjmlp 3y ago
            Thus not adequate for performance requirements.
            • junon 3y ago
              .... that has nothing to do with the allocator though
          • junon 3y ago
            Slotmap is one of my favorite crates, by the way. Thank you for putting it out there!
  • mr_00ff00 3y ago
    If unsafe means “safe but the compiler cannot verify” then I guess just consider .cpp to mean “safe but the compiler cannot verify” and we have suddenly made C++ memory safe
    • Ygg2 3y ago
      It's not what unsafe means. Unsafe means this might cause UB for some invocations (accessing raw pointers, calling into another language, etc.). Safe means it will not cause UB for any invocations (it may panic or abort).
    • ammar2 3y ago
      Sure but you're missing the > so long as you are diligent about checking invariants part. Could you go through and check all the parts of a huge C++ codebase to make sure invariants are held as opposed to a few hundred lines of unsafe Rust code?
      • mr_00ff00 3y ago
        Sure, but I think the point here is the degree. Presumably if it takes a lot of unsafe rust lines to build something, it won’t matter if it’s 30% safe or whatever. I just see the point of “unsafe is fine” a lot when the whole point of rust is that memory safety issues are never worth the cost.
        • ammar2 3y ago
          Right, I guess the question is what will that proportion be when Rust is used for things like operating systems and web browsers. 30% would be untenable but a few hundred/thousand lines of unsafe code is fairly easy to put under a microscope. For some current day research into this, there is the paper "How Do Programmers Use Unsafe Rust?"[1] which I'll drop a quote from here: > The majority of crates (76.4%) contain no unsafe features at all. Even in most crates that do contain unsafe blocks or functions, only a small fraction of the code is unsafe: for 92.3% of all crates, the unsafe statement ratio is at most 10%, i.e., up to 10% of the codebase consists of unsafe blocks and unsafe functions That paper is definitely worth reading and goes into why programmers use unsafe. e.g 5% of the crates at that time were using it to perform FFI. In writing "RUDRA: Finding Memory Safety Bugs in Rust at the Ecosystem Scale" [2], I recreated this data and year-by-year the % of crates using unsafe is going down. And for what it's worth, crates are probably a bad data-set for this. crates tend to be libraries which are exactly where we would expect to find unsafe code encapsulated to be used safely. There's also plenty of experimental and hobby crates. A large dataset of actual binaries would be way more interesting to look at. [1] https://dl.acm.org/doi/10.1145/3428204 https://dl.acm.org/doi/10.1145/3428204 [2] https://taesoo.kim/pubs/2021/bae:rudra.pdf https://taesoo.kim/pubs/2021/bae:rudra.pdf
  • tialaramex 3y ago
    > Obviously the standard library uses some "unsafe" as well, for instance. Most beautifully, MaybeUninit<T>::assume_init() -> T This unsafe Rust method says "I promise that I actually did initialize this MaybeUninit<T>, so give me the T". In terms of the resulting program the machine is not going to do any work whatsoever, a MaybeUninit<T> and a T are the same size, they're in the same place, your CPU doesn't care that this is a T not a MaybeUninit<T> now. But from a type safety point of view, there's all the difference in the world. Even though it won't result in emitting any actual CPU instructions, MaybeUninit::assume_init has to be unsafe. Most of the rest of that API surface is not. Because that API call, the one which emitted no CPU instructions, is where you took responsibility for type correctness. If you were wrong, if you haven't initialized T properly, everything may be about to go spectacularly wrong and there's no-one else to blame but you.
    • dcow 3y ago
      Exactly. People miss this all the time when they write off Rust for "needing unsafe to do real programming" or whatever uninformed criticism they're parroting (they've clearly never actually done this "real programming" in Rust). The whole point is to reduce the opportunity for unforced errors by marginalizing the cognitive load required for the programmer to ensure the program is correct. And a program with a few unsafe blocks to `assume_init` some memory that e.g. a driver initialized for you is still infinitely better in that regard than a program that's littered with `void*` everywhere.
      • mike_hock 3y ago
        > than a program that's littered with `void*` everywhere Strawman argument. A properly written C++ program isn't littered with `void*` everywhere in the same way that a properly written Rust program isn't littered with `unsafe` everywhere. You build safe abstractions around the ugly low-level pointer handling, you just don't have a keyword for a clear delineation. > People miss this all the time when they write off Rust for "needing unsafe to do real programming" or whatever uninformed criticism they're parroting Hard-core Rust proponents also seem to miss this all the time. Because "you basically write the same unsafe code that you would write in C++ but you now have a keyword to mark it" just doesn't imply the same urgency for adopting the language than "you only need unsafe to implement a few primitives in the standard library" does, which always seems to be tacitly implied until called out, and then the critics are "misinformed."
  • runeks 3y ago
    > "unsafe" just means "safe but the compiler cannot verify it". "unsafe" means "safe"? I would say "unsafe" means "only safe if used in a manner that cannot be checked by the compiler".
    • ChrisSD 3y ago
      There are two things here. The `unsafe` in an `unsafe { ... }` block is referring to the contents of the block. From the outside it is indeed safe to use as if it were safe code. No special requirements necessary. So, yes, from a certain point of view `safe` would have been a better name (albeit confusing in a different way). An `unsafe fn` however does need to be used correctly (and should document those requirements). However, these can only be called within `unsafe` blocks, so see above.
      • cyber_kinetist 3y ago
        Not entirely correct, Rust’s “unsafe” marker doesn’t pollute only its scope, it actually pollutes the whole module; You need to make sure that the invariants in unsafe code are met even in safe code. (An explanation of this in the Rustonomicon: https://doc.rust-lang.org/nomicon/working-with-unsafe.html https://doc.rust-lang.org/nomicon/working-with-unsafe.html)
• orlp 3y ago
  > Slotmap uses unsafe everywhere, it's a memory usage pattern not supported by the borrow checker. Author of slotmap here. This is patently false. Yes, the slotmap crate uses a lot of unsafe to squeeze out maximum performance. But it is not 'a memory usage pattern not supported by the borrow checker'. You can absolutely write a crate with an API identical to slotmap without using unsafe.
  • dxhdr 3y ago
    > But it is not 'a memory usage pattern not supported by the borrow checker'. You can absolutely write a crate with an API identical to slotmap without using unsafe. I think that might actually be worse though, performance aside. You're performing memory / object lifetime management but the Rust borrow checker still would have no idea what's going on because now you've tricked it by using indices or an opaque handle instead of references. The program may compile just fine but could have use-after-free bugs. At least with unsafe there's an explicit acknowledgement that the borrow checker is turned off.
    • orlp 3y ago
      Yes, using slotmap you can get "use after free"-style bugs that you would not encounter if you strictly stayed with the borrow checker. So if the borrow checker fits your purpose, by all means, go ahead. But the borrow checker can't represent circular/self-referential structures you see very often in graphs. Nor is it convenient in some cases as it has a strict separation between references that can mutate, and those that can't, which doesn't fit all problems either because the mutable references are by necessity unique. Note that a "use after free" in slotmap results in a None value, or a panic (exception for the C++ people), depending on which API you use. In other words, it is detected and you can handle it. It does not trigger undefined behavior, you don't get ABA-style spurious references, there are no security issues. It is not the same as the issues pointers have at all.
• chlorion 3y ago
  I have implemented my own slotmap crate for a lisp interpreter that uses no unsafe code and provides exactly the same features as the "standard" slotmap crate. There is nothing inherent to the slotmap that requires unsafe code! It's only used for optimizations purposes. Mine works in a similar way to the "standard" slotmap. It's a vec of slots, slot is an enum that can be occupied or vacant, the occupied variant is a two tuple containing the value and generation, vacant holds just a generation. Inserting into the slotmap simply switches the variant of the slot from vacant to occupied, and popping does the reverse. If there is no currently vacant slots, we just use the underlying push method on the vec of slots which will handle resizing for us! I also store a stack of indexes to vacant slots to make insertion fast. When you insert into the slotmap, it provides a opaque key, but the data inside is an index and a generation. When you attempt to retrieve a value with a key, the slotmap checks if the slot is occupied and if the generation matches, and if so returns the value, otherwise returns none. There is also a indirect slotmap, that adds an extra layer of indirection, so rather than the key being an index directly into the underlying vec of slots, its an index into a vec of indexes, this allows moving the slots around without invaliding currently living keys. The indirect slotmap has the advantage of faster iteration, since it doesn't have to skip over empty "holes" of vacant slots in the vec of slots. The tradeoff is that insertion is slightly slower! Anyways, no unsafe is required to implement a performant slotmap data structure! I have not uploaded my slotmap to crates.io because I didn't think anyone would find it useful, but maybe I should reconsider this!
  • deleted 3y ago
    [deleted]