5 ms·

If my intent is that a certain integer should never be negative, labelling it unsigned does a poor job at conveying this intent because any arithmetic just sile

by joppy 5y ago

If my intent is that a certain integer should never be negative, labelling it unsigned does a poor job at conveying this intent because any arithmetic just silently does the wrong thing. 3-4 is some huge number rather than “error”.

tromp 5y ago
In Rust, 3u64 - 4u64 is "error", specifically thread 'main' panicked at 'attempt to subtract with overflow'
- Macha 5y ago
  It panics _in debug builds_ but wraps in release builds because including the checks in every arithmetic operation is not zero cost.
  - masklinn 5y ago
    Though you can configure your release builds to panic as well. That can be quite useful if you don't expect trivial computations to be a large cost center (and you can always bench to check that assumption). Even more so now that custom profiles have been added to Cargo (yay), you can enable overflow checking in release, and add a `release-unchecked` or whatever.
    - tialaramex 5y ago
      Or, better in many cases, you can write what you meant explicitly in Rust. 3u64.checked_sub(4u64).expect("Overflow"); This allows you to spell out that you don't expect this to overflow, and Rust should panic if it does, regardless of your compiler flags.
      - masklinn 5y ago
        Of course you can do that, but I would say that doing so defensively for every operation is way more verbose and unwieldy than you'd want. Much simpler to just toggle on the relevant flag.
        eyelidlessness 5y ago
        I think GP was saying you can do that when you expect that the general case doesn’t need to be checked but a specific case does.
        masklinn 5y ago
        > when you expect that the general case doesn’t need to be checked but a specific case does. But surely that's almost never the case? Few programs actually desire or care for base 2 modular arithmetics, and when they do it tends to be for very specific tasks (usually cryptographic or cryptography-adjacent e.g. hashing, checksumming, ...).
        eyelidlessness 5y ago
        > But surely that's almost never the case? Which is why it’s not so impractical to make an occasional exception.
      - deepsun 5y ago
        I wish they had "unchecked_sub()" instead, and default "-" would work like checked_sub().
        tialaramex 5y ago
        unchecked_sub() exists in Nightly and so may some day come to stable However, changing - to perform checked_sub() leaves you with an Option, and this makes ordinary arithmetic look pretty clumsy: let x = (a-b).unwrap()-c; The intent is to land more types with intrinsic behaviour. Today only the Wrapping type is provided, so Wrapping<i32> is an i32 that definitely has Wrapping arithmetic and won't panic on overflow, but eventually Saturating<i16> will be possible (e.g. for CD audio PCM samples, saturating arithmetic is correct, if you try to make the loudest possible noise louder it just stays the same) and so will Unchecked<i8> if you really sure that you're doing 8-bit arithmetic that can't overflow and doesn't need Debug checks.
willis936 5y ago
This sounds like an issue with the language. 3-4 is not a large number and should not be by default. It should be a case handled by the developer. Maybe you want to do some weird bit banging trickery, and that's fine too, but it should require an out-of-the-way function call.
- dragonwriter 5y ago
  > This sounds like an issue with the language. 3-4 is not a large number and should not be by default Many languages optimize numeric operations and DX around numeric types for efficiency rather than correctness, and make the latter high-friction. (Wrap-around subtraction, decimal literals being treated as IEEE floats, etc.) The exceptions (or cases where it is merely less true) tend to be very high level, dynamic languages.
raverbashing 5y ago
But then your function should check if the value is within reasonable parameters. If it was an int at the function signature and you got -1 what would you expect to happen? It's the same thing.
- marginalia_nu 5y ago
  > But then your function should check if the value is within reasonable parameters How could you possibly check this using types that cannot represent what you are checking for?
  - chromatin 5y ago
    > How could you possibly check this using types that cannot represent what you are checking for? `f(u32 a, u32 b) { assert a > b ... ` (if the hypothetical following operation is `a-b` as discussed higher in the thread)
    - pjmlp 5y ago
      Better ensure that assert is not turned off in release mode then.
      - raverbashing 5y ago
        One of the reasons I don't like asserts. Log the error and act accordingly, instead of believing this kind of issue only happens during development
        jstimpfle 5y ago
        How do you "act" if you detected your logic is faulty? In many situations aborting is the best choice. If input data is faulty and the reason for that is not seen as part of your logic, then sure, log an error and skip. I used to define my own assert macro to "ensure" my asserts aren't disabled, but I don't bother anymore. There's nothing wrong with assert, and you needn't define NDEBUG. It's important to be aware that there are different situations, and not all warrant aborting, as described above. Another differentation is that there can be asserts that must be disabled in release builds for performance reasons, and others that won't affect performance and can stay enabled.
cle 5y ago
An unsigned integer will never be negative, that intent is expressed clearly and correctly. What the operational semantics should be is unclear, because of the tradeoffs involved in signaling an error. In terms of the C standard, "3-4" for unsigned ints is modular arithmetic, and the "wrong thing" is assuming that it will do anything other than wrap around. This is very clearly defined, and implied whenever you see an arithmetic expression on unsigned integers.
- kazinator 5y ago
  Conversion from signed to unsigned is implicit and silent. The expression being converted to your "can never be negative" type isn't nonnegative!!! Its value drastically changes; e.g. -3 becomes a huge number.
  - snovv_crash 5y ago
    Not if you pass -Wconversion to GCC
- dagss 5y ago
  Clearly defined, but very inconvenient and makes it too easy to write buggy code that looks correct on surface.
  - cle 5y ago
    Inconvenient to whom? It’s pretty inconvenient not to do that if you eg have strict memory/performance requirements. C made the right tradeoff IMO. You can protect yourself against overflow if you need to, but if it always signals errors you can’t turn that off.
cesarb 5y ago
> because any arithmetic just silently does the wrong thing The same thing, but worse, can happen with signed integers. (-3)-INT_MAX is either some huge positive number, or something crazy because it's undefined behavior and the compiler is allowed to do anything it wants.
- dureuill 5y ago
  I guess what the parent is saying is that, while the same kind of bugs can happen with signed integers, they are less common than with unsigned integers because they typically involve very big or very small values such as INT_MAX or INT_MIN. This is due to the fact that the "most common values" (close to 0) are at the beginning of the range of unsigned integers, but at the middle of the range of signed integers, so the risk of underflow is lower for signed integers in practice. I can understand where they come from, especially regarding error handling: for integers whose value must not be negative, it is easier to check for underflow by checking if the result is not negative rather than by checking that the result is not eg smaller than the previous value (eg for addition). That being said, "number must not be negative" really ought to be encoded in the type system, so that the user of the type knows that they must actually look for underflow. I guess that part of the problem is that we don't want to check for underflow after each operation, so checking the negativity is a way to "coalesce" several checks after multiple operations. However this is fragile, because the multiple operations could end up producing a positive value, even if some intermediate values where negative. For full safety, I don't see how we could do better than checking after each operation right now. If we're doing this, I feel like checked_add and friends from rust is a better fit than cramming an unsigned int into a signed one. I wonder if we could design an integer type with 63 bits of value, plus one bit of "overflow/underflow poison", such that any operation that would under/overflow would saturate that bit to one, but otherwise still perform the operation on the value part. That would allow to coalesce multiple checks while keeping safety even in the presence of multiple faulty operations. I wonder how it could be implemented efficiently though
  - cesarb 5y ago
    > I wonder if we could design an integer type with 63 bits of value, plus one bit of "overflow/underflow poison", such that any operation that would under/overflow would saturate that bit to one, but otherwise still perform the operation on the value part. That would allow to coalesce multiple checks while keeping safety even in the presence of multiple faulty operations. I wonder how it could be implemented efficiently though There is something like that already for floating point. Whenever an overflow or underflow happens, it sets a sticky bit in a separate flags register. You can clear these flags, do a sequence of operations, and at the end, see if any of these flags are set. See https://man7.org/linux/man-pages/man3/fenv.3.html https://man7.org/linux/man-pages/man3/fenv.3.html for the standard C API for it.

by joppy 5y ago

tromp 5y ago

In Rust, 3u64 - 4u64 is "error", specifically thread 'main' panicked at 'attempt to subtract with overflow'

Macha 5y ago
It panics _in debug builds_ but wraps in release builds because including the checks in every arithmetic operation is not zero cost.
- masklinn 5y ago
  Though you can configure your release builds to panic as well. That can be quite useful if you don't expect trivial computations to be a large cost center (and you can always bench to check that assumption). Even more so now that custom profiles have been added to Cargo (yay), you can enable overflow checking in release, and add a `release-unchecked` or whatever.
  - tialaramex 5y ago
    Or, better in many cases, you can write what you meant explicitly in Rust. 3u64.checked_sub(4u64).expect("Overflow"); This allows you to spell out that you don't expect this to overflow, and Rust should panic if it does, regardless of your compiler flags.
    - masklinn 5y ago
      Of course you can do that, but I would say that doing so defensively for every operation is way more verbose and unwieldy than you'd want. Much simpler to just toggle on the relevant flag.
      - eyelidlessness 5y ago
        I think GP was saying you can do that when you expect that the general case doesn’t need to be checked but a specific case does.
        masklinn 5y ago
        > when you expect that the general case doesn’t need to be checked but a specific case does. But surely that's almost never the case? Few programs actually desire or care for base 2 modular arithmetics, and when they do it tends to be for very specific tasks (usually cryptographic or cryptography-adjacent e.g. hashing, checksumming, ...).
        eyelidlessness 5y ago
        > But surely that's almost never the case? Which is why it’s not so impractical to make an occasional exception.
    - deepsun 5y ago
      I wish they had "unchecked_sub()" instead, and default "-" would work like checked_sub().
      - tialaramex 5y ago
        unchecked_sub() exists in Nightly and so may some day come to stable However, changing - to perform checked_sub() leaves you with an Option, and this makes ordinary arithmetic look pretty clumsy: let x = (a-b).unwrap()-c; The intent is to land more types with intrinsic behaviour. Today only the Wrapping type is provided, so Wrapping<i32> is an i32 that definitely has Wrapping arithmetic and won't panic on overflow, but eventually Saturating<i16> will be possible (e.g. for CD audio PCM samples, saturating arithmetic is correct, if you try to make the loudest possible noise louder it just stays the same) and so will Unchecked<i8> if you really sure that you're doing 8-bit arithmetic that can't overflow and doesn't need Debug checks.

willis936 5y ago

This sounds like an issue with the language. 3-4 is not a large number and should not be by default. It should be a case handled by the developer. Maybe you want to do some weird bit banging trickery, and that's fine too, but it should require an out-of-the-way function call.

dragonwriter 5y ago
> This sounds like an issue with the language. 3-4 is not a large number and should not be by default Many languages optimize numeric operations and DX around numeric types for efficiency rather than correctness, and make the latter high-friction. (Wrap-around subtraction, decimal literals being treated as IEEE floats, etc.) The exceptions (or cases where it is merely less true) tend to be very high level, dynamic languages.

raverbashing 5y ago

But then your function should check if the value is within reasonable parameters. If it was an int at the function signature and you got -1 what would you expect to happen? It's the same thing.

marginalia_nu 5y ago
> But then your function should check if the value is within reasonable parameters How could you possibly check this using types that cannot represent what you are checking for?
- chromatin 5y ago
  > How could you possibly check this using types that cannot represent what you are checking for? `f(u32 a, u32 b) { assert a > b ... ` (if the hypothetical following operation is `a-b` as discussed higher in the thread)
  - pjmlp 5y ago
    Better ensure that assert is not turned off in release mode then.
    - raverbashing 5y ago
      One of the reasons I don't like asserts. Log the error and act accordingly, instead of believing this kind of issue only happens during development
      - jstimpfle 5y ago
        How do you "act" if you detected your logic is faulty? In many situations aborting is the best choice. If input data is faulty and the reason for that is not seen as part of your logic, then sure, log an error and skip. I used to define my own assert macro to "ensure" my asserts aren't disabled, but I don't bother anymore. There's nothing wrong with assert, and you needn't define NDEBUG. It's important to be aware that there are different situations, and not all warrant aborting, as described above. Another differentation is that there can be asserts that must be disabled in release builds for performance reasons, and others that won't affect performance and can stay enabled.

cle 5y ago

An unsigned integer will never be negative, that intent is expressed clearly and correctly. What the operational semantics should be is unclear, because of the tradeoffs involved in signaling an error. In terms of the C standard, "3-4" for unsigned ints is modular arithmetic, and the "wrong thing" is assuming that it will do anything other than wrap around. This is very clearly defined, and implied whenever you see an arithmetic expression on unsigned integers.

kazinator 5y ago
Conversion from signed to unsigned is implicit and silent. The expression being converted to your "can never be negative" type isn't nonnegative!!! Its value drastically changes; e.g. -3 becomes a huge number.
- snovv_crash 5y ago
  Not if you pass -Wconversion to GCC
dagss 5y ago
Clearly defined, but very inconvenient and makes it too easy to write buggy code that looks correct on surface.
- cle 5y ago
  Inconvenient to whom? It’s pretty inconvenient not to do that if you eg have strict memory/performance requirements. C made the right tradeoff IMO. You can protect yourself against overflow if you need to, but if it always signals errors you can’t turn that off.

cesarb 5y ago

> because any arithmetic just silently does the wrong thing The same thing, but worse, can happen with signed integers. (-3)-INT_MAX is either some huge positive number, or something crazy because it's undefined behavior and the compiler is allowed to do anything it wants.

dureuill 5y ago
I guess what the parent is saying is that, while the same kind of bugs can happen with signed integers, they are less common than with unsigned integers because they typically involve very big or very small values such as INT_MAX or INT_MIN. This is due to the fact that the "most common values" (close to 0) are at the beginning of the range of unsigned integers, but at the middle of the range of signed integers, so the risk of underflow is lower for signed integers in practice. I can understand where they come from, especially regarding error handling: for integers whose value must not be negative, it is easier to check for underflow by checking if the result is not negative rather than by checking that the result is not eg smaller than the previous value (eg for addition). That being said, "number must not be negative" really ought to be encoded in the type system, so that the user of the type knows that they must actually look for underflow. I guess that part of the problem is that we don't want to check for underflow after each operation, so checking the negativity is a way to "coalesce" several checks after multiple operations. However this is fragile, because the multiple operations could end up producing a positive value, even if some intermediate values where negative. For full safety, I don't see how we could do better than checking after each operation right now. If we're doing this, I feel like checked_add and friends from rust is a better fit than cramming an unsigned int into a signed one. I wonder if we could design an integer type with 63 bits of value, plus one bit of "overflow/underflow poison", such that any operation that would under/overflow would saturate that bit to one, but otherwise still perform the operation on the value part. That would allow to coalesce multiple checks while keeping safety even in the presence of multiple faulty operations. I wonder how it could be implemented efficiently though
- cesarb 5y ago
  > I wonder if we could design an integer type with 63 bits of value, plus one bit of "overflow/underflow poison", such that any operation that would under/overflow would saturate that bit to one, but otherwise still perform the operation on the value part. That would allow to coalesce multiple checks while keeping safety even in the presence of multiple faulty operations. I wonder how it could be implemented efficiently though There is something like that already for floating point. Whenever an overflow or underflow happens, it sets a sticky bit in a separate flags register. You can clear these flags, do a sequence of operations, and at the end, see if any of these flags are set. See https://man7.org/linux/man-pages/man3/fenv.3.html https://man7.org/linux/man-pages/man3/fenv.3.html for the standard C API for it.