7 ms·

This is the main reason I don't trust the mob of people working on Rust to get their act together and make it a good language. You can see some awareness of th

This is the main reason I don't trust the mob of people working on Rust to get their act together and make it a good language. You can see some awareness of the value of integer overflow detection among some individuals that work on Rust but despite that, here we are without that feature. Swift, on the other hand, is one language that has put some thought to the matter.

doublec 12y ago
There was a discussion in the Rust mailing list about it a while back. I used that as a basis for looking at overflow detection at compile time in ATS: http://bluishcoder.co.nz/2013/05/07/ranged-integer-types-and-bounds-checking.html http://bluishcoder.co.nz/2013/05/07/ranged-integer-types-and...
- codygman 12y ago
  Thanks for the great read. As someone who's been eyeing ATS for a while and checking it out more lately, this was very inspirational.
TazeTSchnitzel 12y ago
Huh, Swift actually does overflow checking and traps by default. That's quite refreshing.
pcwalton 12y ago
> You can see some awareness of the value of integer overflow detection among some individuals that work on Rust but despite that, here we are without that feature. Swift, on the other hand, is one language that has put some thought to the matter. We have put a lot of thought into the matter, and the conclusion was that (a) the performance overhead, as measured by benchmarks, could be significant; (b) the relative benefits of doing the checking in Rust would be much less than in C and C++, since overflow in (safe) Rust cannot result in memory safety problems. However, we're looking into ways that we could allow folks to opt into the feature globally in the future should the performance story change or if they want to panic rather than overflow. You can already use the special checked types for arithmetic on a case-by-case basis if you'd like overflow checking.
- SamReidHughes 12y ago
  Not to re-relitigate old discussion, but (as I'm sure you know) it's now more probable that you might overflow your integers and then pass them to unsafe code or external APIs, and there are many other bugs or undesirable behavior that could be killed dead instead of allowing them to behave badly. For a not contrived example, I don't want weird stuff to happen when my semaphore counter overflows, or underflows, I want it to just crash, right there. (My perspective and desires come from a project where I didn't care about "raw" performance that much -- the project being a storage engine where C++ was chosen with the performance concerns in mind being lack of GC and resultant traffic shockwaves, so my preferences lie a bit away from CPU integer operation performance and more towards crashing obviously at the spot of the foul. It's the worst kind of bug when you hang mysteriously in production or to write garbage to disk when the error could be caught, and I had many C++ integer underflow worries at that job.) I also think overflow errors should be conveniently handleable without putting a try! or match expression on every operation, and so should other errors, but I'll spare you that rant :-)
  - leoc 12y ago
    It's that old song. "Our language is safe!—for some very specific, rather convenient definition of 'safe' which we just picked ourselves."
    - pcwalton 12y ago
      I don't understand your criticism. Anyone who claims anything is "safe" has to be specific about the definition of "safe", or else they'd be making vague unsubstantiated claims. Rust is "memory safe" per the commonly used Saraswat definition [1] (not proven yet, but the proof is in progress). This is not something we just made up. [1]: http://www.cis.upenn.edu/~bcpierce/courses/629/papers/Saraswat-javabug.html http://www.cis.upenn.edu/~bcpierce/courses/629/papers/Sarasw...
      - leoc 12y ago
        I think I misread you. It's certainly reasonable to say that memory safety will reduce the harm, on average, of integer overflows and that this should affect any cost/benefit calculation for implementing overflow checking. What would have been unreasonable is to imply that the safety implications of integer overflow can't matter much or are somehow out of scope because of Rust memory-safety. No matter how nice and precisely-defined the memory-safety criteria are, they're only part (an important one ofc) of what people reasonably expect in terms of safety improvements from a language which aims to be a better, saner successor to C/C++ and is evangelised as such. "Not a deathtrap like C" may be a much fuzzier standard than "memory safe per the Saraswat definition", but that doesn't change the fact that the latter is just a partial means to the former. > This is not something we just made up. I deliberately did not say 'made up' or 'created', but rather 'picked' meaning 'selected'. The fact that the map is pre-existing and commonly-used (and rightly so) does not alter the fact that it is not the terrain.
derefr 12y ago
It seems to me that, despite all the flexibility people demand, they only really want (and use) exactly three integer data-types in a language: 1. machine bit-string of size 8(2^n) bits (e.g. b/w/d/q fields) with no concept of being a signed/unsigned integer, but where you can apply both signed and unsigned integer operations to it in an optimized manner. Such code will be shimmed with sets of clever shifting ops if the target architecture doesn't actually have a storage type of that width, but you have to be explicit about what you'll allow (e.g. if an integer is of type "d|2w", then it will compile fine on a 16-bit architecture (and operate using shimmed ops), but fail to compile on an 8-bit architecture. 2. unsigned integer of fixed exactly-specified bitsize, which exactly matches the semantics of a having a value on a modular-arithmetic ring. You go up, it wraps around at 2^bitsize; you go down, it wraps back. This stays true even if the code is compiled on an architecture where the exactly-specified bitsize isn't a clean processor storage-width: a 27-bit ring on a 32-bit processor is A-OK, and shims will be inserted to enforce the semantics. Shims will also be inserted if you're targeting an ISA that doesn't have an integer type with wrapping semantics (e.g. the JVM.) 3. signed arbitrary-precision integer (i.e. optimal-machine-word-size-minus-a-tag-bit with bignum promotion checks). The optimizer might convert one of these to something of type #1 if it can be very, very sure of the value range you're operating within. #1 is for doing pointer math in unsafe regions; #2 is for implementing cryptographic primitives; #3 is for everyone else. (There's also a variant on #1—let's call it #1A—which is a fixed-size array of #1s you can repeat signed/unsigned integer transformations over, where this will generate optimized SIMD code. This would be the "buffer" type for wire protocol packing/unpacking, and also the backing store for non-sparse matrix ADTs, bloom filters, etc.) The only place integer overflow raising an exception would make sense, to me, are if you want some sort of hybrid type between #1 and #3: a value that pretends to be arbitrary-precision, but in fact only has a constant-size bitstring to operate within. I could see the use of this in something like Cap'n Proto, where you're keeping wire-encoded integer bitstrings (#1s from a #1A) around and pretending they're fully-functional integers (#3) for the sake of zero-copy—but do you really gain that much by not letting a data structure be recreated resized on the stack, if you're not also throwing down optimizations like intrusive lists in fixed arenas?
- SamReidHughes 12y ago
  I don't want 3, because I don't want tag bits or space wastage, but also because I don't want to have to link a bigint library.
lifthrasiir 12y ago
Asides from pcwalton's argument, I think any successful (or to-be-successful) language is trying to solve some problems very well and other problems reasonably well. For Rust, the former includes the memory safety and the latter includes the integer overflow detection (but you can always name a lot more). It would be very hard to solve all problems at once (especially when known solutions to some problems worsen other problems, as seen in Rust). That can be said to Swift as well, it just picked different problems to solve very well.
- SamReidHughes 12y ago
  I don't think it's hard to solve the problem of memory safety and integer overflow detection at the same time.
  - lifthrasiir 12y ago
    Both problems are undergoing active research. They are indeed orthogonal (albeit one can somehow reduce the severity of another and vice versa), but with a finite resource and time you sometimes have to choose.
raverbashing 12y ago
What is missing is a good and easy way of doing the check: - Without need checking after every operation - Without impeding easy use of overflow (and/or allowing for saturation) when it's needed (in a lot of places) So, I see a narrow use for it, IF you validated your inputs AND overflow is unexpected (or, it is expected and you do something different, but it's still on narrow cases)

doublec 12y ago

There was a discussion in the Rust mailing list about it a while back. I used that as a basis for looking at overflow detection at compile time in ATS: http://bluishcoder.co.nz/2013/05/07/ranged-integer-types-and-bounds-checking.html http://bluishcoder.co.nz/2013/05/07/ranged-integer-types-and...

codygman 12y ago
Thanks for the great read. As someone who's been eyeing ATS for a while and checking it out more lately, this was very inspirational.

TazeTSchnitzel 12y ago

Huh, Swift actually does overflow checking and traps by default. That's quite refreshing.

pcwalton 12y ago

> You can see some awareness of the value of integer overflow detection among some individuals that work on Rust but despite that, here we are without that feature. Swift, on the other hand, is one language that has put some thought to the matter. We have put a lot of thought into the matter, and the conclusion was that (a) the performance overhead, as measured by benchmarks, could be significant; (b) the relative benefits of doing the checking in Rust would be much less than in C and C++, since overflow in (safe) Rust cannot result in memory safety problems. However, we're looking into ways that we could allow folks to opt into the feature globally in the future should the performance story change or if they want to panic rather than overflow. You can already use the special checked types for arithmetic on a case-by-case basis if you'd like overflow checking.

SamReidHughes 12y ago
Not to re-relitigate old discussion, but (as I'm sure you know) it's now more probable that you might overflow your integers and then pass them to unsafe code or external APIs, and there are many other bugs or undesirable behavior that could be killed dead instead of allowing them to behave badly. For a not contrived example, I don't want weird stuff to happen when my semaphore counter overflows, or underflows, I want it to just crash, right there. (My perspective and desires come from a project where I didn't care about "raw" performance that much -- the project being a storage engine where C++ was chosen with the performance concerns in mind being lack of GC and resultant traffic shockwaves, so my preferences lie a bit away from CPU integer operation performance and more towards crashing obviously at the spot of the foul. It's the worst kind of bug when you hang mysteriously in production or to write garbage to disk when the error could be caught, and I had many C++ integer underflow worries at that job.) I also think overflow errors should be conveniently handleable without putting a try! or match expression on every operation, and so should other errors, but I'll spare you that rant :-)
- leoc 12y ago
  It's that old song. "Our language is safe!—for some very specific, rather convenient definition of 'safe' which we just picked ourselves."
  - pcwalton 12y ago
    I don't understand your criticism. Anyone who claims anything is "safe" has to be specific about the definition of "safe", or else they'd be making vague unsubstantiated claims. Rust is "memory safe" per the commonly used Saraswat definition [1] (not proven yet, but the proof is in progress). This is not something we just made up. [1]: http://www.cis.upenn.edu/~bcpierce/courses/629/papers/Saraswat-javabug.html http://www.cis.upenn.edu/~bcpierce/courses/629/papers/Sarasw...
    - leoc 12y ago
      I think I misread you. It's certainly reasonable to say that memory safety will reduce the harm, on average, of integer overflows and that this should affect any cost/benefit calculation for implementing overflow checking. What would have been unreasonable is to imply that the safety implications of integer overflow can't matter much or are somehow out of scope because of Rust memory-safety. No matter how nice and precisely-defined the memory-safety criteria are, they're only part (an important one ofc) of what people reasonably expect in terms of safety improvements from a language which aims to be a better, saner successor to C/C++ and is evangelised as such. "Not a deathtrap like C" may be a much fuzzier standard than "memory safe per the Saraswat definition", but that doesn't change the fact that the latter is just a partial means to the former. > This is not something we just made up. I deliberately did not say 'made up' or 'created', but rather 'picked' meaning 'selected'. The fact that the map is pre-existing and commonly-used (and rightly so) does not alter the fact that it is not the terrain.

derefr 12y ago

It seems to me that, despite all the flexibility people demand, they only really want (and use) exactly three integer data-types in a language: 1. machine bit-string of size 8(2^n) bits (e.g. b/w/d/q fields) with no concept of being a signed/unsigned integer, but where you can apply both signed and unsigned integer operations to it in an optimized manner. Such code will be shimmed with sets of clever shifting ops if the target architecture doesn't actually have a storage type of that width, but you have to be explicit about what you'll allow (e.g. if an integer is of type "d|2w", then it will compile fine on a 16-bit architecture (and operate using shimmed ops), but fail to compile on an 8-bit architecture. 2. unsigned integer of fixed exactly-specified bitsize, which exactly matches the semantics of a having a value on a modular-arithmetic ring. You go up, it wraps around at 2^bitsize; you go down, it wraps back. This stays true even if the code is compiled on an architecture where the exactly-specified bitsize isn't a clean processor storage-width: a 27-bit ring on a 32-bit processor is A-OK, and shims will be inserted to enforce the semantics. Shims will also be inserted if you're targeting an ISA that doesn't have an integer type with wrapping semantics (e.g. the JVM.) 3. signed arbitrary-precision integer (i.e. optimal-machine-word-size-minus-a-tag-bit with bignum promotion checks). The optimizer might convert one of these to something of type #1 if it can be very, very sure of the value range you're operating within. #1 is for doing pointer math in unsafe regions; #2 is for implementing cryptographic primitives; #3 is for everyone else. (There's also a variant on #1—let's call it #1A—which is a fixed-size array of #1s you can repeat signed/unsigned integer transformations over, where this will generate optimized SIMD code. This would be the "buffer" type for wire protocol packing/unpacking, and also the backing store for non-sparse matrix ADTs, bloom filters, etc.) The only place integer overflow raising an exception would make sense, to me, are if you want some sort of hybrid type between #1 and #3: a value that pretends to be arbitrary-precision, but in fact only has a constant-size bitstring to operate within. I could see the use of this in something like Cap'n Proto, where you're keeping wire-encoded integer bitstrings (#1s from a #1A) around and pretending they're fully-functional integers (#3) for the sake of zero-copy—but do you really gain that much by not letting a data structure be recreated resized on the stack, if you're not also throwing down optimizations like intrusive lists in fixed arenas?

SamReidHughes 12y ago
I don't want 3, because I don't want tag bits or space wastage, but also because I don't want to have to link a bigint library.

lifthrasiir 12y ago

Asides from pcwalton's argument, I think any successful (or to-be-successful) language is trying to solve some problems very well and other problems reasonably well. For Rust, the former includes the memory safety and the latter includes the integer overflow detection (but you can always name a lot more). It would be very hard to solve all problems at once (especially when known solutions to some problems worsen other problems, as seen in Rust). That can be said to Swift as well, it just picked different problems to solve very well.

SamReidHughes 12y ago
I don't think it's hard to solve the problem of memory safety and integer overflow detection at the same time.
- lifthrasiir 12y ago
  Both problems are undergoing active research. They are indeed orthogonal (albeit one can somehow reduce the severity of another and vice versa), but with a finite resource and time you sometimes have to choose.

raverbashing 12y ago

What is missing is a good and easy way of doing the check: - Without need checking after every operation - Without impeding easy use of overflow (and/or allowing for saturation) when it's needed (in a lot of places) So, I see a narrow use for it, IF you validated your inputs AND overflow is unexpected (or, it is expected and you do something different, but it's still on narrow cases)