5 ms·

There's probably no way for the compiler to prove safety. Rust is designed to allow 100% safe bare metal development, like a perfectly safe C that still allows

by api 2y ago

There's probably no way for the compiler to prove safety. Rust is designed to allow 100% safe bare metal development, like a perfectly safe C that still allows you to get close to the hardware, and that's tough.

ajross 2y ago
I'm failing to understand the correlation to "safety" here. Reading a byte for which you don't know the value isn't "unsafe". It's literally (!) the desired behavior of foreign data being read from an external source, which is in fact the use case in the article. There's no safety problem as long as the arbitrary value is deterministic, which it is, being process RAM. The "uninitialized data read" bugs reported from instrumentation tools in C code are because the code is assuming the value has some semantics. The read itself has no value and is presumably an artifact of the bug, but it is safe.
- gpm 2y ago
  > There's no safety problem as long as the arbitrary value is deterministic, which it is, being process RAM. The article discusses how it is in fact, on Linux with memory returned from at least one very common allocator, not deterministic. Ctrl-f tautology.
  - ajross 2y ago
    That's just a terminology collision. All RAM access is deterministic in the sense that the value will not change until written. It's not "predictable" in the sense that the value could be anything. C code that reads uninitialized data is presumed to be buggy, because it wouldn't be doing that unless it thought the memory was initialized. But the read itself is not unsafe. Rust is just confused, and is preventing all reads from uninitialized data a-priori instead of relying on its perfectly working type system to tell it whether the uninitialized data is safe to use. And that has performance impact, as described in the linked article, which has then resulted in some terrible API choices to evade.
    - gpm 2y ago
      > All RAM access is deterministic in the sense that the value will not change until written. Again, the article literally points to how this is not true given modern allocators. The memory that Linux exposes to processes will change without being written to prior to being initialized given how allocators manage it. This isn't a fiction of the C-standard or rust reference, it's what actually happens in the real world on a regular basis. Rust is not confused, it is correctly observing what is allowed to actually happen to uninitialized memory while the process does nothing to it. You could change the C/Rust specification of that memory. You could in your C/rust implementation declare that the OS swapping out pages of uninitialized memory counts as a write just like any other, and that it's the programmers (allocators) responsibility to make sure those writes obey the normal aliasing rules. Doing so would be giving up performance though, because the fact that writing to memory has the side-effect of cancelling collection of freed pages is a powerful way for processes to quickly communicate with the OS. (You'd probably also cause other issues with memory mapped IO, values after the end of the stack changing, and so on, but we can just focus on this one issue for now).
    - vlovich123 2y ago
      I don’t understand your point and you’re wrong on a couple of things. > C code that reads uninitialized data is presumed to be buggy, because it wouldn't be doing that unless it thought the memory was initialized. But the read itself is not unsafe. Rust is just confused, and is preventing all reads from uninitialized data a-priori instead of relying on its perfectly working type system to tell it whether the uninitialized data is safe to use Reads of uninitialized memory is unsafe full stop. That’s literally what Rust’s memory safety is about. If you give that up you’re not in safe land and you can always use unsafe & all the risks that come with that to try to write more optimal code / abstractions. This article is literally about the mechanisms Rust is trying to stabilize how you use the type system to go from a block of uninitialized memory & is aware of writes occuring so that it can take a &[MaybeUninit<T>] and give you back a &[T] after a call to read which wrote into the slice. But reading uninitialized memory by definition is tautologically unsafe. It doesn’t mean that the computer will pull out a knife and kill you, but it does mean you’re no longer memory safe.
MrBuddyCasino 2y ago
What does safe mean here? Everything can be interpreted as a [u8], right?
- lidavidm 2y ago
  https://www.ralfj.de/blog/2019/07/14/uninit.html https://www.ralfj.de/blog/2019/07/14/uninit.html perhaps (the OP also talks about this when linking to a talk about jemalloc)
  - MrBuddyCasino 2y ago
    Interesting. Tl;dr: its not to do with any hardware concept, the compiler can substitute any value for a read of uninitialised memory, and the value does not have to be stable.
- gpm 2y ago
  [u8] guarantees to the compiler that two reads through the array at the same location without any intervening writes return the same value. Turns out that's not the case on freshly returned uninitiated allocations. The first read could return old data (say "1"), and the second read could return a freshly zeroed page ("0").
  - ridiculous_fish 2y ago
    No, it's not so. If the allocation is backed by the kernel, then it will be zero-filled for security reasons. If it's backed by user-space malloc then who knows; but there's never a scenario where a mallocated page is quietly replaced by a zero-filled page behind the scenes.
    - gpm 2y ago
      Ctrl-f tautology in the article, it turns out that is not the case because of madvise free.
lmm 2y ago
> There's probably no way for the compiler to prove safety. That's already the case for the AnyBitPattern stuff though. (Indeed according to the docs AnyBitPattern traits already get cast from uninitialized bytes, which in C/LLVM semantics are not necessarily frozen, even if in practice Linux would not be remapping the pages they're in). > Rust is designed to allow 100% safe bare metal development I wouldn't say that since bare metal rust always needs some unsafe; rather it's designed to allow managed, contained use of unsafe constructs in code that's say 98% safe. The whole purpose of this BorrowedBuf is already something like that.

by api 2y ago

ajross 2y ago

I'm failing to understand the correlation to "safety" here. Reading a byte for which you don't know the value isn't "unsafe". It's literally (!) the desired behavior of foreign data being read from an external source, which is in fact the use case in the article. There's no safety problem as long as the arbitrary value is deterministic, which it is, being process RAM. The "uninitialized data read" bugs reported from instrumentation tools in C code are because the code is assuming the value has some semantics. The read itself has no value and is presumably an artifact of the bug, but it is safe.

gpm 2y ago
> There's no safety problem as long as the arbitrary value is deterministic, which it is, being process RAM. The article discusses how it is in fact, on Linux with memory returned from at least one very common allocator, not deterministic. Ctrl-f tautology.
- ajross 2y ago
  That's just a terminology collision. All RAM access is deterministic in the sense that the value will not change until written. It's not "predictable" in the sense that the value could be anything. C code that reads uninitialized data is presumed to be buggy, because it wouldn't be doing that unless it thought the memory was initialized. But the read itself is not unsafe. Rust is just confused, and is preventing all reads from uninitialized data a-priori instead of relying on its perfectly working type system to tell it whether the uninitialized data is safe to use. And that has performance impact, as described in the linked article, which has then resulted in some terrible API choices to evade.
  - gpm 2y ago
    > All RAM access is deterministic in the sense that the value will not change until written. Again, the article literally points to how this is not true given modern allocators. The memory that Linux exposes to processes will change without being written to prior to being initialized given how allocators manage it. This isn't a fiction of the C-standard or rust reference, it's what actually happens in the real world on a regular basis. Rust is not confused, it is correctly observing what is allowed to actually happen to uninitialized memory while the process does nothing to it. You could change the C/Rust specification of that memory. You could in your C/rust implementation declare that the OS swapping out pages of uninitialized memory counts as a write just like any other, and that it's the programmers (allocators) responsibility to make sure those writes obey the normal aliasing rules. Doing so would be giving up performance though, because the fact that writing to memory has the side-effect of cancelling collection of freed pages is a powerful way for processes to quickly communicate with the OS. (You'd probably also cause other issues with memory mapped IO, values after the end of the stack changing, and so on, but we can just focus on this one issue for now).
  - vlovich123 2y ago
    I don’t understand your point and you’re wrong on a couple of things. > C code that reads uninitialized data is presumed to be buggy, because it wouldn't be doing that unless it thought the memory was initialized. But the read itself is not unsafe. Rust is just confused, and is preventing all reads from uninitialized data a-priori instead of relying on its perfectly working type system to tell it whether the uninitialized data is safe to use Reads of uninitialized memory is unsafe full stop. That’s literally what Rust’s memory safety is about. If you give that up you’re not in safe land and you can always use unsafe & all the risks that come with that to try to write more optimal code / abstractions. This article is literally about the mechanisms Rust is trying to stabilize how you use the type system to go from a block of uninitialized memory & is aware of writes occuring so that it can take a &[MaybeUninit<T>] and give you back a &[T] after a call to read which wrote into the slice. But reading uninitialized memory by definition is tautologically unsafe. It doesn’t mean that the computer will pull out a knife and kill you, but it does mean you’re no longer memory safe.

MrBuddyCasino 2y ago

What does safe mean here? Everything can be interpreted as a [u8], right?

lidavidm 2y ago
https://www.ralfj.de/blog/2019/07/14/uninit.html https://www.ralfj.de/blog/2019/07/14/uninit.html perhaps (the OP also talks about this when linking to a talk about jemalloc)
- MrBuddyCasino 2y ago
  Interesting. Tl;dr: its not to do with any hardware concept, the compiler can substitute any value for a read of uninitialised memory, and the value does not have to be stable.
gpm 2y ago
[u8] guarantees to the compiler that two reads through the array at the same location without any intervening writes return the same value. Turns out that's not the case on freshly returned uninitiated allocations. The first read could return old data (say "1"), and the second read could return a freshly zeroed page ("0").
- ridiculous_fish 2y ago
  No, it's not so. If the allocation is backed by the kernel, then it will be zero-filled for security reasons. If it's backed by user-space malloc then who knows; but there's never a scenario where a mallocated page is quietly replaced by a zero-filled page behind the scenes.
  - gpm 2y ago
    Ctrl-f tautology in the article, it turns out that is not the case because of madvise free.

lmm 2y ago

> There's probably no way for the compiler to prove safety. That's already the case for the AnyBitPattern stuff though. (Indeed according to the docs AnyBitPattern traits already get cast from uninitialized bytes, which in C/LLVM semantics are not necessarily frozen, even if in practice Linux would not be remapping the pages they're in). > Rust is designed to allow 100% safe bare metal development I wouldn't say that since bare metal rust always needs some unsafe; rather it's designed to allow managed, contained use of unsafe constructs in code that's say 98% safe. The whole purpose of this BorrowedBuf is already something like that.