6 ms·

I know, the goal is (in short) a listing of expected undefined behavior, which could improve security if developers follow along. But isn't putting undefined b

I know, the goal is (in short) a listing of expected undefined behavior, which could improve security if developers follow along. But isn't putting undefined behavior into the standard an oxymoron, kinda?

mike_hock 3y ago
No, because UB is not avoidable in principle unless you create a walled garden/sandbox that must be enforced at non-zero runtime cost.
- vlovich123 3y ago
  I don’t know. Safe rust has 0 UB although if I recall correctly things are still a bit messy with integer overflow. I could imagine a world where we define annotations around “signed integer will never overflow” that you can add to hit paths but otherwise disallow optimizing around that UB and for all other UB require an explicit annotation acknowledging it or just a warning and a missed optimization instead. The current status quo is not the best.
  - aw1621107 3y ago
    The key is "that must be enforced at non-zero runtime cost." Safe Rust is indeed generally free of UB, but that requires bounds checks at a minimum. > I could imagine a world where we define annotations around “signed integer will never overflow” that you can add to hit paths but otherwise disallow optimizing around that UB That's kind of what Rust does - overflow is not UB, but you can use unchecked_add/mul/div/sub to get UB-on-overflow if you explicitly want it.
    - jeffbee 3y ago
      How does Rust as a language avoid the "UB" traps that are fundamentally just compiler optimizations? For example in the case where you have an array of 8 elements, you access the element at index x, then (in program order) test whether x is less than 8. It seems than an optimizing compiler with a known-bits analysis could conclude that x is always less than 8, and remove that test. I don't know much about languages, so I am interested in how that trap is a consequence of C and avoided by Rust.
      - aw1621107 3y ago
        The Rust compiler/stdlib inserts bounds checks for accesses that cannot be proven to be safe, so in this example you'd get a panic on the out-of-bounds access itself instead of hitting the assert afterwards. If you use get_unchecked() to intentionally bypass those bounds checks, the assert is indeed removed, but that requires an unsafe block. Speaking more generally, Rust gates UB behavior behind unsafe blocks, so it's much harder to unintentionally hit UB than in C.
      - Kranar 3y ago
        Rust performs a runtime check to protect against out of bounds array accesses. In some cases those checks can be optimized out if the compiler can prove it's not needed.
        jeffbee 3y ago
        This is my point precisely. Having proven to itself that 0 <= x < 8, the compiler could go on to remove any test for x < 8, right?
        heinrich5991 3y ago
        Yes. It even does so in some cases.
        sgift 3y ago
        Yes, and the compiler does that. That's one of the reasons that iterators are preferred over for-loops with manual access in Rust. While compilers can for very simple cases determine that you will always stay within the bounds (as in your example, where it can find at compile time that the length will always be 8 and no access outside happens) with iterators the compiler doesn't need to know the actual length and can always elide bound checks.
        GrumpySloth 3y ago
        Pedantic note: compiler doesn’t elide bounds checks in code using iterators. They’re not there to begin with. Iterators in the standard library are implemented using unsafe code, which calls non-bounds-checking variants of functions getting elements from collections. There is nothing for the compiler left to do here regarding bounds-checking.
    - vlovich123 3y ago
      There's lots of things beyond bounds checks that constitute UB and bounds checks are but 1 small item. Generally Rust avoids the runtime cost of that through a three prong strategy of idiomatic Rust amortizing the cost of a bounds check to 0 (e.g. efficient iterator implementations that don't need bounds checks), eliding the check altogether if the compiler can prove it's duplicate / unneeded for some reason, or providing unsafe Rust as an escape hatch when you absolutely need it. C++ already has something similar via `.at` vs `[]` although the shorter notation being the unsafer but faster option is debatable & likely the thing most people use by default. Explicit annotations are probably the better approach. It's also got a great ecosystem. I love the assume crate to do these annotations instead of writing unsafe code explicitly: assume!(unsafe: i < v.len()) Now you've explicitly written an assumption that will cause the compiler to elide the bounds check in release mode but still assert it in debug mode (vs just doing unsafe & using variants that bypass the bounds check).
      - aw1621107 3y ago
        > There's lots of things beyond bounds checks that constitute UB and bounds checks are but 1 small item. Indeed, hence "at a minimum". The type system and choosing to avoid UB when defining some operations helps a lot as well, but those have no direct overhead so there's no runtime cost there. Bounds checks (and overflow checks, if those are ever added to release mode) are just the one thing that have a direct runtime cost when they aren't elided.
    - mike_hock 3y ago
      The key is that you can't get rid of it entirely, which is why Rust has `unsafe.`
  - jpcfl 3y ago
    > Safe rust has 0 UB Safe Rust aims for 0 UB, but I don't think you can make the claim that it absolutely has no UB. This program SEGFAULTs on my system (macOS), because it's reading an invalid memory address due to a stack overflow: const N: usize = 1024*1024*1024; fn main() { let var: [u8; N] = [0; N]; println!("var: {:?}", var); }
    - Kranar 3y ago
      Safe Rust has no undefined behavior. Undefined behavior does not mean no crashing, it means that the semantics of the program are undefined. Rust's semantics are to abort on a stack overflow. A language like C or C++ have no such semantics, they may abort or they may continue running and producing jibberish.
      - vlovich123 3y ago
        I don't know if it's technically UB or well defined. The crash is a SEGFAULT and not a panic/abort, but it's probably a SEGFAULT due to guard pages. Still, it's possible to evade guard pages so if you access var[X] such that X points to the heap, it's possible you're reading aliased memory which would be UB in safe Rust. EDIT: Going to take it back. I'm unable to create a situation where I create a large stack array that doesn't result in an immediate stack overflow. I even tried nightly MaybeUninit::uninit_array but that crashed explicitly with a "fatal runtime error: stack overflow" so it seems like the standard library has improved reporting instead of the old SEGFAULT. So no UB.
        deleted 3y ago
        [deleted]
        Kranar 3y ago
        Panics are not quite the same as an abort in Rust. Most notably a panic can be caught and execution can resume so as to gracefully terminate the application, but an abort is an immediate termination, a go to jail do not pass go kind of situation. An out of bounds access in Rust will result in a panic but a stack overflow is an abort.
  - crotchfire 3y ago
    things are still a bit messy with integer overflow No, they aren't messy, they're fully defined. Integer overflow in safe Rust works just like Java: the numbers wrap around. It's totally defined. You can, optionally, also enable a runtime check for this wraparound. This is usually enabled for debug builds. There is of course a performance cost to doing this.
    - vlovich123 3y ago
      I didn't say they're undefined but I guess we can agree to disagree that integer addition behavior depending on debug vs release is messy.
    - throwaway81523 3y ago
      > they aren't messy, they're fully defined. Integer overflow in safe Rust works just like Java: the numbers wrap around. That is both fully defined and messy. They are not mutually exclusive. It means there are integers where n+1 is less than n. That is messy. No integers that I learned about in math class work like that. The only two non-messy well-defined behaviours are 1) bignums by default like in Python, but not suitable for a low level language like Rust; or, 2) trap on overflow, like Ada is supposed to do though it is usually shut off by a pragma. Both of those have significant runtime cost.
      - mike_hock 3y ago
        > No integers that I learned about in math class work like that. Did you learn about modular arithmetic in math class? > The only two non-messy well-defined behaviours are 1) bignums by default like in Python, but not suitable for a low level language like Rust; or, 2) trap on overflow, like Ada is supposed to do though it is usually shut off by a pragma. Both of those have significant runtime cost. No, they're not the only "non-messy" behaviors. An example of where wraparound is the desired behavior is computing hash functions. An example of where saturating arithmetic is the desired behavior is processing audio samples.
        throwaway81523 3y ago
  - deathanatos 3y ago
    Safe Rust has 0 UB, but the person you're responding to qualified that with: > that must be enforced at non-zero runtime cost. Some of Rust's behavior that permits this is not zero runtime costs. Bounds-checking, for example, has a non-zero cost: the bounds check! Now, I greatly prefer Rust's approach: I'd rather have the marginal CPU cost: I value my time far higher, at least until the profiler speaks up, and in many cases the optimizer is pretty good at eliminating checks that I myself might look at and go "but I know $condition is true here!" — so too does the optimizer. (And these days, Godbolt makes testing that very simple.) And in the worse case, there's unsafe{}, and it is at least explicitly labelled as such to the next reader. > things are still a bit messy with integer overflow Integer overflow is well-defined behavior, but the behavior depends on compile settings. (https://doc.rust-lang.org/book/ch03-02-data-types.html#integer-overflow https://doc.rust-lang.org/book/ch03-02-data-types.html#integ...) I do hope that someday, the debug behavior becomes the behavior: IME most overflows are errors / the author did not intend for it to occur. And the "panic on overflow" behavior is removable by simply using one of the functions that specifies an overflow behavior, in which case then the author's intention is just explicitly stated.)
crotchfire 3y ago
What they are listing is not the undefined behaviors, but rather the C++ language constructs which might lead to them.
mort96 3y ago
There's already a bunch of stuff in the standard which explicitly says, "if you do xyz, the behavior is undefined". Stuff doesn't become "defined" just because the standard points out that it's not defined.
pornel 3y ago
It’s a poor name all around, because the spec already precisely defines boundaries of what is UB — “if this and this happens, the behavior is undefined”. These are precisely defined situations, that in modern compilers’ interpretation, are forbidden from ever happening in any program under any circumstances. UB is only a name for nebulous consequences of violating these very specific prohibitions.
- AlotOfReading 3y ago
  The spec explicitly does not define the boundaries of all UB. Look at the note on the definition in 1.3.24 and the context of the surrounding definitions: Undefined behavior may be expected when this International Standard omits any explicit definition of behavior... Rather than having specific causes, UB is simply the label given to "stuff outside the standard with no requirements whatsoever". Some of this stuff is important and much more of it isn't. A small subset of that infinite universe of UB is the set of enumerated undefined behaviors you're talking about, where the standard declines to specify any requirements in certain situations. These are what the annex is trying to organize.

mike_hock 3y ago

No, because UB is not avoidable in principle unless you create a walled garden/sandbox that must be enforced at non-zero runtime cost.

vlovich123 3y ago
I don’t know. Safe rust has 0 UB although if I recall correctly things are still a bit messy with integer overflow. I could imagine a world where we define annotations around “signed integer will never overflow” that you can add to hit paths but otherwise disallow optimizing around that UB and for all other UB require an explicit annotation acknowledging it or just a warning and a missed optimization instead. The current status quo is not the best.
- aw1621107 3y ago
  The key is "that must be enforced at non-zero runtime cost." Safe Rust is indeed generally free of UB, but that requires bounds checks at a minimum. > I could imagine a world where we define annotations around “signed integer will never overflow” that you can add to hit paths but otherwise disallow optimizing around that UB That's kind of what Rust does - overflow is not UB, but you can use unchecked_add/mul/div/sub to get UB-on-overflow if you explicitly want it.
  - jeffbee 3y ago
    How does Rust as a language avoid the "UB" traps that are fundamentally just compiler optimizations? For example in the case where you have an array of 8 elements, you access the element at index x, then (in program order) test whether x is less than 8. It seems than an optimizing compiler with a known-bits analysis could conclude that x is always less than 8, and remove that test. I don't know much about languages, so I am interested in how that trap is a consequence of C and avoided by Rust.
    - aw1621107 3y ago
      The Rust compiler/stdlib inserts bounds checks for accesses that cannot be proven to be safe, so in this example you'd get a panic on the out-of-bounds access itself instead of hitting the assert afterwards. If you use get_unchecked() to intentionally bypass those bounds checks, the assert is indeed removed, but that requires an unsafe block. Speaking more generally, Rust gates UB behavior behind unsafe blocks, so it's much harder to unintentionally hit UB than in C.
    - Kranar 3y ago
      Rust performs a runtime check to protect against out of bounds array accesses. In some cases those checks can be optimized out if the compiler can prove it's not needed.
      - jeffbee 3y ago
        This is my point precisely. Having proven to itself that 0 <= x < 8, the compiler could go on to remove any test for x < 8, right?
        heinrich5991 3y ago
        Yes. It even does so in some cases.
        sgift 3y ago
        Yes, and the compiler does that. That's one of the reasons that iterators are preferred over for-loops with manual access in Rust. While compilers can for very simple cases determine that you will always stay within the bounds (as in your example, where it can find at compile time that the length will always be 8 and no access outside happens) with iterators the compiler doesn't need to know the actual length and can always elide bound checks.
        GrumpySloth 3y ago
        Pedantic note: compiler doesn’t elide bounds checks in code using iterators. They’re not there to begin with. Iterators in the standard library are implemented using unsafe code, which calls non-bounds-checking variants of functions getting elements from collections. There is nothing for the compiler left to do here regarding bounds-checking.
  - vlovich123 3y ago
    There's lots of things beyond bounds checks that constitute UB and bounds checks are but 1 small item. Generally Rust avoids the runtime cost of that through a three prong strategy of idiomatic Rust amortizing the cost of a bounds check to 0 (e.g. efficient iterator implementations that don't need bounds checks), eliding the check altogether if the compiler can prove it's duplicate / unneeded for some reason, or providing unsafe Rust as an escape hatch when you absolutely need it. C++ already has something similar via `.at` vs `[]` although the shorter notation being the unsafer but faster option is debatable & likely the thing most people use by default. Explicit annotations are probably the better approach. It's also got a great ecosystem. I love the assume crate to do these annotations instead of writing unsafe code explicitly: assume!(unsafe: i < v.len()) Now you've explicitly written an assumption that will cause the compiler to elide the bounds check in release mode but still assert it in debug mode (vs just doing unsafe & using variants that bypass the bounds check).
    - aw1621107 3y ago
      > There's lots of things beyond bounds checks that constitute UB and bounds checks are but 1 small item. Indeed, hence "at a minimum". The type system and choosing to avoid UB when defining some operations helps a lot as well, but those have no direct overhead so there's no runtime cost there. Bounds checks (and overflow checks, if those are ever added to release mode) are just the one thing that have a direct runtime cost when they aren't elided.
  - mike_hock 3y ago
    The key is that you can't get rid of it entirely, which is why Rust has `unsafe.`
- jpcfl 3y ago
  > Safe rust has 0 UB Safe Rust aims for 0 UB, but I don't think you can make the claim that it absolutely has no UB. This program SEGFAULTs on my system (macOS), because it's reading an invalid memory address due to a stack overflow: const N: usize = 1024*1024*1024; fn main() { let var: [u8; N] = [0; N]; println!("var: {:?}", var); }
  - Kranar 3y ago
    Safe Rust has no undefined behavior. Undefined behavior does not mean no crashing, it means that the semantics of the program are undefined. Rust's semantics are to abort on a stack overflow. A language like C or C++ have no such semantics, they may abort or they may continue running and producing jibberish.
    - vlovich123 3y ago
      I don't know if it's technically UB or well defined. The crash is a SEGFAULT and not a panic/abort, but it's probably a SEGFAULT due to guard pages. Still, it's possible to evade guard pages so if you access var[X] such that X points to the heap, it's possible you're reading aliased memory which would be UB in safe Rust. EDIT: Going to take it back. I'm unable to create a situation where I create a large stack array that doesn't result in an immediate stack overflow. I even tried nightly MaybeUninit::uninit_array but that crashed explicitly with a "fatal runtime error: stack overflow" so it seems like the standard library has improved reporting instead of the old SEGFAULT. So no UB.
      - deleted 3y ago
        [deleted]
      - Kranar 3y ago
        Panics are not quite the same as an abort in Rust. Most notably a panic can be caught and execution can resume so as to gracefully terminate the application, but an abort is an immediate termination, a go to jail do not pass go kind of situation. An out of bounds access in Rust will result in a panic but a stack overflow is an abort.
- crotchfire 3y ago
  things are still a bit messy with integer overflow No, they aren't messy, they're fully defined. Integer overflow in safe Rust works just like Java: the numbers wrap around. It's totally defined. You can, optionally, also enable a runtime check for this wraparound. This is usually enabled for debug builds. There is of course a performance cost to doing this.
  - vlovich123 3y ago
    I didn't say they're undefined but I guess we can agree to disagree that integer addition behavior depending on debug vs release is messy.
  - throwaway81523 3y ago
    > they aren't messy, they're fully defined. Integer overflow in safe Rust works just like Java: the numbers wrap around. That is both fully defined and messy. They are not mutually exclusive. It means there are integers where n+1 is less than n. That is messy. No integers that I learned about in math class work like that. The only two non-messy well-defined behaviours are 1) bignums by default like in Python, but not suitable for a low level language like Rust; or, 2) trap on overflow, like Ada is supposed to do though it is usually shut off by a pragma. Both of those have significant runtime cost.
    - mike_hock 3y ago
      > No integers that I learned about in math class work like that. Did you learn about modular arithmetic in math class? > The only two non-messy well-defined behaviours are 1) bignums by default like in Python, but not suitable for a low level language like Rust; or, 2) trap on overflow, like Ada is supposed to do though it is usually shut off by a pragma. Both of those have significant runtime cost. No, they're not the only "non-messy" behaviors. An example of where wraparound is the desired behavior is computing hash functions. An example of where saturating arithmetic is the desired behavior is processing audio samples.
      - throwaway81523 3y ago
- deathanatos 3y ago
  Safe Rust has 0 UB, but the person you're responding to qualified that with: > that must be enforced at non-zero runtime cost. Some of Rust's behavior that permits this is not zero runtime costs. Bounds-checking, for example, has a non-zero cost: the bounds check! Now, I greatly prefer Rust's approach: I'd rather have the marginal CPU cost: I value my time far higher, at least until the profiler speaks up, and in many cases the optimizer is pretty good at eliminating checks that I myself might look at and go "but I know $condition is true here!" — so too does the optimizer. (And these days, Godbolt makes testing that very simple.) And in the worse case, there's unsafe{}, and it is at least explicitly labelled as such to the next reader. > things are still a bit messy with integer overflow Integer overflow is well-defined behavior, but the behavior depends on compile settings. (https://doc.rust-lang.org/book/ch03-02-data-types.html#integer-overflow https://doc.rust-lang.org/book/ch03-02-data-types.html#integ...) I do hope that someday, the debug behavior becomes the behavior: IME most overflows are errors / the author did not intend for it to occur. And the "panic on overflow" behavior is removable by simply using one of the functions that specifies an overflow behavior, in which case then the author's intention is just explicitly stated.)

crotchfire 3y ago

What they are listing is not the undefined behaviors, but rather the C++ language constructs which might lead to them.

mort96 3y ago

There's already a bunch of stuff in the standard which explicitly says, "if you do xyz, the behavior is undefined". Stuff doesn't become "defined" just because the standard points out that it's not defined.

pornel 3y ago

It’s a poor name all around, because the spec already precisely defines boundaries of what is UB — “if this and this happens, the behavior is undefined”. These are precisely defined situations, that in modern compilers’ interpretation, are forbidden from ever happening in any program under any circumstances. UB is only a name for nebulous consequences of violating these very specific prohibitions.

AlotOfReading 3y ago
The spec explicitly does not define the boundaries of all UB. Look at the note on the definition in 1.3.24 and the context of the surrounding definitions: Undefined behavior may be expected when this International Standard omits any explicit definition of behavior... Rather than having specific causes, UB is simply the label given to "stuff outside the standard with no requirements whatsoever". Some of this stuff is important and much more of it isn't. A small subset of that infinite universe of UB is the set of enumerated undefined behaviors you're talking about, where the standard declines to specify any requirements in certain situations. These are what the annex is trying to organize.