9 ms·

This rather niche fixing of unsafe behaviour is excellent: https://blog.rust-lang.org/2020/07/16/Rust-1.45.0.html#fixing-unsoundness-in-casts https://blog.rust-

This rather niche fixing of unsafe behaviour is excellent: https://blog.rust-lang.org/2020/07/16/Rust-1.45.0.html#fixing-unsoundness-in-casts https://blog.rust-lang.org/2020/07/16/Rust-1.45.0.html#fixin... I spent a few years as a scientific programmer and this is exactly the sort of thing that just bites you on the behind in C/C++/Fortran: the undefined behaviour can actually manifest as noise in your output, or just really hard to track down, intermittent problems. A big win to get rid of it.

ansible 6y ago
It is nice that they have defined behavior for that now. Though I try to always scrutinize any floating-point / integer conversions during code reviews. The default casting of a floating point value to integer is frequently not what you want, however. In the code we do, for example, you will usually round to the nearest integer instead, we don't normally need the more fancy rounding schemes.
davrosthedalek 6y ago
I'm not sure I understand this. Does it not produce a run time error? Why not? This looks very dangerous, because it essentially does the "nearest to right" thing. Say, you cast 256 to a u8, it's then saturated to 255. That's almost right, and a result might be wrong only by 0.5%. Much harder to detect than if it is set to 0.
- deleted 6y ago
  [deleted]
- deleted 6y ago
  [deleted]
- tinalumfoil 6y ago
  Most floating points aren't integers anyway, so if you think of it as casting 256 to the nearest u8 it's correct, same as rounding 0.25 to 0. NaN to 0 is a bit more concerning, but inconvienence and compatibility need to be weighed against catching every error (no type system will catch every bug).
- pwdisswordfish2 6y ago
  > I'm not sure I understand this. Does it not produce a run time error? Why not? It’s not supposed to. Type casting with ‘as’ is supposed to be lightweight and always succeed; there is no room in the type system to return an error. In case lossless casting is not possible, some value still has to be returned. Until now, this was outright UB — meaning the compiler is not even obligated to keep it consistent from one build to another. Saturating, while still not optimal, is at least deterministic. > This looks very dangerous, because it essentially does the "nearest to right" thing. That’s why the intention is to introduce more robust approximate conversion functions and eventually probably deprecate ‘as’ casts altogether. There has been a number of discussions about this; current disagreements seem to be about how to handle the various possible rounding modes.
  - chowells 6y ago
    > meaning the compiler is not even obligated to keep it consistent from one build to another. Way worse than that. The compiler wasn't obligated to act like anything at all. It would be totally legal to compile it so that the first time the value was accessed you got 0, the next time you got 1 - within the same program execution, with no mutation of the value. That is the sort of thing that is observed behavior of UB in the worst cases, and why it's so terrible to just pretend that UB is innocuous.
    - Sharlin 6y ago
      Way worse than that, even. UB poisons every state of the program that eventually results in UB. For example, the optimizer is well within its rights to remove as dead code any branch that, if taken, would provably lead to UB at some arbitrary future point of execution.
      - TomMarius 6y ago
        That could literally produce no output program?
        6y ago
- kibwen 6y ago
  > Why not? For better or worse, Rust 1.0 released with the philosophy that the `as` operator is for "fast and loose" conversions where accuracy is not prioritized; e.g. casting a u32 to a u8 would always risk silently truncating in the event the value was too large to represent. Over the years the language has added a lot of standard library support for bypassing the `as` operator entirely, and I think the prevailing opinion at this point might be that if they had the to do it all over again they might not have had the `as` operator at all, instead making do with a combination of ordinary error-checked conversion methods and the YOLO unsafe unchecked methods as seen here. Which is to say: it's not that they technically couldn't have gone with the panic approach, but (performance implications aside) I think they'd rather just start moving away from `as` in general wherever possible.
  - yazaddaruvala 6y ago
    Should `as` then be a candidate for deprecation in the next edition?
    - kibwen 6y ago
      I don't know about the next edition; `as` is also used for a small number of non-numeric conversions, like turning a reference into a raw pointer or for creating a trait object. In order to completely remove `as`, you would need to first introduce alternatives to those (the trait object use case specifically might be a bit weird as a method rather than a keyword, but I don't know if that's a big problem). Alternatively one could just deprecate `as` for numeric conversions, although there are still some library holes that would need to be patched up (e.g. other than `as` I don't know of an existing single method to say that you want a fast integer conversion routine that merely truncates a u64 into a u32; right now the alternative is `try_into`, which does a runtime check and returns a Result). It might also exacerbate tensions with some people who for quite a while have been grumpy that Rust requires frequent explicit numeric conversions when doing things like indexing (which always requires a `usize` type, so you see `foo[bar as usize]` unless people want to always work with usizes directly); would these people be happier if that were `foo[bar.as_usize()]` instead, or would this all be blocked on a discussion about being more lenient with numeric conversions?
milancurcic 6y ago
Fortran predictably overflows the result, in contrast to fptoui which gives a poison value. I agree that even a predictable overflow can bite you on the behind. But it's better than undefined behavior. I'm not a fan of saturated cast, but having a defined behavior is for sure a great improvement.
- nitrogen 6y ago
  When working with audio as an example, saturation is much less obnoxious than wraparound. You can potentially destroy speakers and your hearing that way.
  - pjmlp 6y ago
    Wraparound is also the bug that turns Ghandi into a huge dictator in Civilization, and killed a couple of people in cancer treatment devices.
    - nindalf 6y ago
      For those not in the know, computer controlled Civs would be assigned behaviour attributes based on historical personality. Each attribute would be 1-10, stored in an 8 bit integer. These attributes change in response to events in the game. When a civilization would reach the modern era, it's aggression attribute would be reduced by 2. Gandhi's aggression was reduced from 1 to 255 (underflow). He would rain down nukes on everything in sight, somewhat uncharacteristic of him. Fans liked this so much that Firaxis maintained the same behaviour in later games.
      - paavohtl 6y ago
        It's a cool story, but I don't think there is evidence that it actually happened (in Civ 1, certainly in later games as a homage to this potentially urban legend). The YouTube channel People Make Games did an investigation[1] about this a year ago, including interviews with the original developers. [1] https://www.youtube.com/watch?v=Ur3SdgkW8W4 https://www.youtube.com/watch?v=Ur3SdgkW8W4
yodelshady 6y ago
Out of interest, with (my instance of!) 1.44, "let i = 257.0; i as u8" casts to 1. "let i = usize::MAX as f64 + 1.0; i as usize" does likewise. If you assign i as an integer DIRECTLY, however, it saturates. To be honest... struggling to see why you'd do the former, outside of situations where you're happy with saturation, though I haven't thought about it a lot. Agreed that a consistent behaviour is a big help - I can work with "Rust does x in this scenario".
- Sharlin 6y ago
  Again, UB means the compiler is allowed to do anything at all with code that triggers UB. It’s not just that the value you get is undefined, the behavior of the whole program becomes suspect. Including anything that happens before the UB is actually triggered, not just after that.
gameswithgo 6y ago
Agree, a lot of people focus on Rust and memory safety vs C/C++, but the lack (nearly) of undefined behavior is at least as important/helpful.
- fluffy87 6y ago
  Undefined behavior is memory safety; at least from Rust POV. A program with UB is not memory safe because the compiler makes no guarantees about the behaviors of those programs.
crazypython 6y ago
Rust has no specification.
- steveklabnik 6y ago
  Sort of; we have more than nothing, but not a full thing. Some things are well specified. Some things are mostly specified. Some things are still very much up in the air.
  - mesaframe 6y ago
    Do you have specification of a memory model?
    - kibwen 6y ago
      I believe it's safe to assume that Rust's memory model is (a subset of) C++11's memory model, since that's what LLVM implements. At this point I don't see how any specification could deviate significantly from that without breaking tons of code.
      - Jweb_Guru 6y ago
        The Rust memory model is deviating from it a little in order to enable some norestrict-based optimizations that aren't really done for C, even though (as you know) LLVM can't really take advantage of them yet.
        staticassertion 6y ago
        I imagine that in C if the compiler can prove that two pointers don't alias than it can elide a load?
        pjmlp 6y ago
        It is very much compiler specific, and also depends if modifiers like volatile are present or not.
nullc 6y ago
It would be nice if there were both a saturating-as and a overflow-is-a-bug-as, the later of which is also saturating but in debug builds get instrumentation to panic if it ever actually saturates. The overflow-is-a-bug-saturating as would be the default, and there would be a separate sat_as for "I know this saturates, it isn't a bug.". ::sigh:: Rust went through this same debate for integers, initially rejecting the argument I'm giving here but switching back to it after silently-defined-integer-overflow concealed severe memory unsafty bugs in the standard library. Well-defining something like saturation actually reduces the power of static and dynamic program analysis because it can no longer tell if the overflow was a programmer-intended use of saturation or a bug. Having it undefined was better from a tools perspective, even if worse at runtime, because if a tool could prove that overflow would happen (statically) or that it does happen (dynamically) that would always be a bug, and always be worth bringing to the user's attention. So now you still get "noise" in your output, but it's the harder to detect noise of consistent saturation, and you've lost the ability to have instrumentation that tells you that you have a flaw in your code. So I think this is again an example of rust making a decision that hurts program correctness.
- estebank 6y ago
  `as` is not considered idiomatic in Rust code,. `.into()` for infallible cases and `.try_into()` for fallible cases are preferred in almost all cases (in the case of floats there's still discussion around the correct behavior of `.try_into()` for edge cases). > So I think this is again an example of rust making a decision that hurts program correctness. Could you expand on other examples?
  - saurik 6y ago
    It might not be "idiomatic", but it isn't "unsafe", and so it is now providing a guaranteed behavior that... is it a bug? I certainly think saturation should always be considered a bug... like that's even worse to me than truncation as the semantics of truncation are at least interesting and related to mathematical basis of von neumann computing and sometimes extremely useful as a way to interact with memory-oriented systems, while the semantics of saturation are just subtle corruption that is almost useless to "use" for anything valuable (and are inconsistent, unless they are also doing it for integers? like if I cast a floating point to a 64-bit number and then to an 8-bit number do I get a truncated result instead of a saturated one?... I hope not, even though I reject saturation as a reasonable semantic). But if it is defined legal and guaranteed behavior, what should a tool do in this scenario? It can't say "you have a bug here" as maybe you wanted that behavior.
TheRealKing 6y ago
In Fortran, there is the ieee_arithmetic intrinsic module that can very nicely and robustly handle undefined behavior. If people do not use it, it is their problems.

ansible 6y ago

It is nice that they have defined behavior for that now. Though I try to always scrutinize any floating-point / integer conversions during code reviews. The default casting of a floating point value to integer is frequently not what you want, however. In the code we do, for example, you will usually round to the nearest integer instead, we don't normally need the more fancy rounding schemes.

davrosthedalek 6y ago

I'm not sure I understand this. Does it not produce a run time error? Why not? This looks very dangerous, because it essentially does the "nearest to right" thing. Say, you cast 256 to a u8, it's then saturated to 255. That's almost right, and a result might be wrong only by 0.5%. Much harder to detect than if it is set to 0.

deleted 6y ago
[deleted]
deleted 6y ago
[deleted]
tinalumfoil 6y ago
Most floating points aren't integers anyway, so if you think of it as casting 256 to the nearest u8 it's correct, same as rounding 0.25 to 0. NaN to 0 is a bit more concerning, but inconvienence and compatibility need to be weighed against catching every error (no type system will catch every bug).
pwdisswordfish2 6y ago
> I'm not sure I understand this. Does it not produce a run time error? Why not? It’s not supposed to. Type casting with ‘as’ is supposed to be lightweight and always succeed; there is no room in the type system to return an error. In case lossless casting is not possible, some value still has to be returned. Until now, this was outright UB — meaning the compiler is not even obligated to keep it consistent from one build to another. Saturating, while still not optimal, is at least deterministic. > This looks very dangerous, because it essentially does the "nearest to right" thing. That’s why the intention is to introduce more robust approximate conversion functions and eventually probably deprecate ‘as’ casts altogether. There has been a number of discussions about this; current disagreements seem to be about how to handle the various possible rounding modes.
- chowells 6y ago
  > meaning the compiler is not even obligated to keep it consistent from one build to another. Way worse than that. The compiler wasn't obligated to act like anything at all. It would be totally legal to compile it so that the first time the value was accessed you got 0, the next time you got 1 - within the same program execution, with no mutation of the value. That is the sort of thing that is observed behavior of UB in the worst cases, and why it's so terrible to just pretend that UB is innocuous.
  - Sharlin 6y ago
    Way worse than that, even. UB poisons every state of the program that eventually results in UB. For example, the optimizer is well within its rights to remove as dead code any branch that, if taken, would provably lead to UB at some arbitrary future point of execution.
    - TomMarius 6y ago
      That could literally produce no output program?
      - 6y ago
kibwen 6y ago
> Why not? For better or worse, Rust 1.0 released with the philosophy that the `as` operator is for "fast and loose" conversions where accuracy is not prioritized; e.g. casting a u32 to a u8 would always risk silently truncating in the event the value was too large to represent. Over the years the language has added a lot of standard library support for bypassing the `as` operator entirely, and I think the prevailing opinion at this point might be that if they had the to do it all over again they might not have had the `as` operator at all, instead making do with a combination of ordinary error-checked conversion methods and the YOLO unsafe unchecked methods as seen here. Which is to say: it's not that they technically couldn't have gone with the panic approach, but (performance implications aside) I think they'd rather just start moving away from `as` in general wherever possible.
- yazaddaruvala 6y ago
  Should `as` then be a candidate for deprecation in the next edition?
  - kibwen 6y ago
    I don't know about the next edition; `as` is also used for a small number of non-numeric conversions, like turning a reference into a raw pointer or for creating a trait object. In order to completely remove `as`, you would need to first introduce alternatives to those (the trait object use case specifically might be a bit weird as a method rather than a keyword, but I don't know if that's a big problem). Alternatively one could just deprecate `as` for numeric conversions, although there are still some library holes that would need to be patched up (e.g. other than `as` I don't know of an existing single method to say that you want a fast integer conversion routine that merely truncates a u64 into a u32; right now the alternative is `try_into`, which does a runtime check and returns a Result). It might also exacerbate tensions with some people who for quite a while have been grumpy that Rust requires frequent explicit numeric conversions when doing things like indexing (which always requires a `usize` type, so you see `foo[bar as usize]` unless people want to always work with usizes directly); would these people be happier if that were `foo[bar.as_usize()]` instead, or would this all be blocked on a discussion about being more lenient with numeric conversions?

milancurcic 6y ago

Fortran predictably overflows the result, in contrast to fptoui which gives a poison value. I agree that even a predictable overflow can bite you on the behind. But it's better than undefined behavior. I'm not a fan of saturated cast, but having a defined behavior is for sure a great improvement.

nitrogen 6y ago
When working with audio as an example, saturation is much less obnoxious than wraparound. You can potentially destroy speakers and your hearing that way.
- pjmlp 6y ago
  Wraparound is also the bug that turns Ghandi into a huge dictator in Civilization, and killed a couple of people in cancer treatment devices.
  - nindalf 6y ago
    For those not in the know, computer controlled Civs would be assigned behaviour attributes based on historical personality. Each attribute would be 1-10, stored in an 8 bit integer. These attributes change in response to events in the game. When a civilization would reach the modern era, it's aggression attribute would be reduced by 2. Gandhi's aggression was reduced from 1 to 255 (underflow). He would rain down nukes on everything in sight, somewhat uncharacteristic of him. Fans liked this so much that Firaxis maintained the same behaviour in later games.
    - paavohtl 6y ago
      It's a cool story, but I don't think there is evidence that it actually happened (in Civ 1, certainly in later games as a homage to this potentially urban legend). The YouTube channel People Make Games did an investigation[1] about this a year ago, including interviews with the original developers. [1] https://www.youtube.com/watch?v=Ur3SdgkW8W4 https://www.youtube.com/watch?v=Ur3SdgkW8W4

yodelshady 6y ago

Out of interest, with (my instance of!) 1.44, "let i = 257.0; i as u8" casts to 1. "let i = usize::MAX as f64 + 1.0; i as usize" does likewise. If you assign i as an integer DIRECTLY, however, it saturates. To be honest... struggling to see why you'd do the former, outside of situations where you're happy with saturation, though I haven't thought about it a lot. Agreed that a consistent behaviour is a big help - I can work with "Rust does x in this scenario".

Sharlin 6y ago
Again, UB means the compiler is allowed to do anything at all with code that triggers UB. It’s not just that the value you get is undefined, the behavior of the whole program becomes suspect. Including anything that happens before the UB is actually triggered, not just after that.

gameswithgo 6y ago

Agree, a lot of people focus on Rust and memory safety vs C/C++, but the lack (nearly) of undefined behavior is at least as important/helpful.

fluffy87 6y ago
Undefined behavior is memory safety; at least from Rust POV. A program with UB is not memory safe because the compiler makes no guarantees about the behaviors of those programs.

crazypython 6y ago

Rust has no specification.

steveklabnik 6y ago
Sort of; we have more than nothing, but not a full thing. Some things are well specified. Some things are mostly specified. Some things are still very much up in the air.
- mesaframe 6y ago
  Do you have specification of a memory model?
  - kibwen 6y ago
    I believe it's safe to assume that Rust's memory model is (a subset of) C++11's memory model, since that's what LLVM implements. At this point I don't see how any specification could deviate significantly from that without breaking tons of code.
    - Jweb_Guru 6y ago
      The Rust memory model is deviating from it a little in order to enable some norestrict-based optimizations that aren't really done for C, even though (as you know) LLVM can't really take advantage of them yet.
      - staticassertion 6y ago
        I imagine that in C if the compiler can prove that two pointers don't alias than it can elide a load?
        pjmlp 6y ago
        It is very much compiler specific, and also depends if modifiers like volatile are present or not.

nullc 6y ago

It would be nice if there were both a saturating-as and a overflow-is-a-bug-as, the later of which is also saturating but in debug builds get instrumentation to panic if it ever actually saturates. The overflow-is-a-bug-saturating as would be the default, and there would be a separate sat_as for "I know this saturates, it isn't a bug.". ::sigh:: Rust went through this same debate for integers, initially rejecting the argument I'm giving here but switching back to it after silently-defined-integer-overflow concealed severe memory unsafty bugs in the standard library. Well-defining something like saturation actually reduces the power of static and dynamic program analysis because it can no longer tell if the overflow was a programmer-intended use of saturation or a bug. Having it undefined was better from a tools perspective, even if worse at runtime, because if a tool could prove that overflow would happen (statically) or that it does happen (dynamically) that would always be a bug, and always be worth bringing to the user's attention. So now you still get "noise" in your output, but it's the harder to detect noise of consistent saturation, and you've lost the ability to have instrumentation that tells you that you have a flaw in your code. So I think this is again an example of rust making a decision that hurts program correctness.

estebank 6y ago
`as` is not considered idiomatic in Rust code,. `.into()` for infallible cases and `.try_into()` for fallible cases are preferred in almost all cases (in the case of floats there's still discussion around the correct behavior of `.try_into()` for edge cases). > So I think this is again an example of rust making a decision that hurts program correctness. Could you expand on other examples?
- saurik 6y ago
  It might not be "idiomatic", but it isn't "unsafe", and so it is now providing a guaranteed behavior that... is it a bug? I certainly think saturation should always be considered a bug... like that's even worse to me than truncation as the semantics of truncation are at least interesting and related to mathematical basis of von neumann computing and sometimes extremely useful as a way to interact with memory-oriented systems, while the semantics of saturation are just subtle corruption that is almost useless to "use" for anything valuable (and are inconsistent, unless they are also doing it for integers? like if I cast a floating point to a 64-bit number and then to an 8-bit number do I get a truncated result instead of a saturated one?... I hope not, even though I reject saturation as a reasonable semantic). But if it is defined legal and guaranteed behavior, what should a tool do in this scenario? It can't say "you have a bug here" as maybe you wanted that behavior.

TheRealKing 6y ago

In Fortran, there is the ieee_arithmetic intrinsic module that can very nicely and robustly handle undefined behavior. If people do not use it, it is their problems.