11 ms·

I've vascillated between one camp and the other over the years, but in 25 years of professional development, other than pointer arithmetic I can't think of a si

I've vascillated between one camp and the other over the years, but in 25 years of professional development, other than pointer arithmetic I can't think of a single time where I've dealt with numbers that exceed a trillion in either direction. Integer overflows are so rare that I can't even remember one happening with 64-bit values. Underflow on the other hand... I've encountered that plenty. I'm now back in the signed camp. Signed integers make it a lot easier to sanity check integer subtraction when negative values are disallowed (a common situation), and it's easy to reason about. Also, not converting between signed and unsigned avoids accidental conversion overflows due to different expectations, so keeping it signed bypasses that whole can of worms while maintaining maximum flexibility. Unsigned gives you 2x the positive numeric range, which 99.999% of the time you'll never use (and if you did, underflow detection would become a nightmare). I can't think of a single time I've heard someone complain "I wish Java's long type supported up to 16 quintillion instead of a paltry 8 quintillion" At the end of the day, code is for people, so keep it easy for people.

cma 5y ago
> I can't think of a single time where I've dealt with numbers that exceed a trillion in either direction. A big one is dealing with currencies like Japanese yen in financial software.
- kstenerud 5y ago
  Absolutely! But how about a quintillion? Signed 64-bit integers give you 8 quintillion in the positive integer space, and unsigned gives 16 quintillion. The question is: Will the difference between 8 and 16 quintillion matter enough to say "unsigned by default"? But currency types need to be signed anyway, so I guess that's that ;-)
virtualritz 5y ago
I think types should express intend. This adds more 'value' to the code for people than allowing them to switch their brains off because something is 'unlikely' to happen. A large class of bugs exists because people assume stuff is unlikely. I also think one has to distinguish between API and actual code. If you write a lib that takes inputs that should never be negative you can express this without writing a single line of documentation about it (or if you do, requiring the user to read it) by using unsigned types in these places. You can still use signed types in your actual library code if you want/must/believe in that but do not pollute the outside API with them if unnecessary.
- kstenerud 5y ago
  In a perfect world, this would be so. And if there were types that were performant AND could enforce such invariants without the downsides, I'd welcome them gladly. But primitive types come with baggage and limitations, so you have to make compromises. And the compromises when using unsigned ints are just too costly in my opinion. They're a good tool for specific use cases, but not as good as signed for general purpose code.
  - virtualritz 5y ago
    Could you list some of the downsides & compromises in the case I made?
    - kstenerud 5y ago
      - silent unintended underflow behavior - dangers of mixed signed/unsigned code (signed values MUST exist, but unsigned values don't have to) - more cognitive load to read and understand unsigned underflow detection code + issues once you exceed the halfway point This is why Java offered signed integer types only (I think they should have allowed unsigned as well, but for different reasons)
      - virtualritz 5y ago
        > silent unintended underflow behavior Please show me an example in the wild that could not be trivially caught/alleviated. With 'trivially' I mean the code that needs to be written to prevent it, not the thinking you need to do to consider it. As for the latter: this is what I get paid for. Writing the code out is necessary but not what I really get paid for (unless I were a very slow typist, that is). I find that most of the time I was simply too lazy. Or rather: I wrote C/C++ code for 20+ years. I used int and didn't think (unintended rhyme). Now I write (mostly) Rust I find myself considering the edge cases regularly and also covering them because the compiler forces me to. See also this comment [1]. I.e. a - 1 + b where a is always positive (incl. 0) and b is always > 1 'just' works with int (and unsigned [overflow] too!) in C/C++. In Rust you will get a panic when a = 0 which will then make you think what you're doing and simply reorder this to a + b - 1. I find that this is the 'cognitive load' that needs to be applied commonly and I'm pretty fine with that. > dangers of mixed signed/unsigned code Alleviated by simply not mixing. There is a reason languages like Rust simply do not allow this without explicit typecasts. And those, in cases where one type is signed and the other is not and they have equal width translate to "all bets are off". > more cognitive load to read and understand unsigned underflow detection code [...] See first point. IMHO this is simply the same point expressed differently. Underflow happens when you subtract. Ensure you do not subtract a larger value from a smaller. This is as easy as writing max(a, b) - min(a, b) or what author of [1] does. Same as abs(a - b) but works with unsigned. Cognitive load? I do not see any and I agree with the author of the article that this is easier to read. [1] https://news.ycombinator.com/item?id=29767877 https://news.ycombinator.com/item?id=29767877
berkut 5y ago
I've hit the 2147483647 limit of indices occasionally in VFX/Graphics with large datasets: memory efficiency was also important, so 32-bit was a necessity (random access is required, so can't dynamically pack), so I tend to use unsigned these days (uint32_t) within arrays, but native 64-bit lengths (normally size_t) as standard local variables, and I normally go with unsigned there as well. Several APIs in the VFX arena (Pixar OpenSubD, Pixar USD, and Foundry's Katana GeoLib API) all use signed ints for indices which I've occasionally argued against, with the response being "Google's coding standard says don't use them".
- kstenerud 5y ago
  Yup, and that's a case of an API not aging well. But even in that case, switching to unsigned is only kicking the can down the road, not solving the problem. You'd get another year or two and then hit the 4294967295 limit. As for 64-bit values, unless you can foresee requiring > 8 quintillion, unsigned isn't buying you anything.
  - mjevans 5y ago
    Resource consumption does matter though. For example CPU caches or the quantity of RAM within a system. If their workset does fit within 32 bits of index the intentional design choice very likely correlates to a measurable improvement in speed due to cache locality and system resource utilization.
    - kstenerud 5y ago
      Indeed it does matter! As I said this is an example of an API that hasn't aged well (when they devised it, they probably didn't envision this happening). There are a number of ways to handle this: - Use unsigned 32-bit (gives you a little bit more runway, but you'll still hit the end hard) - Use 64-bit values (ends your problem once and for all, but costs double the memory so now you have a new problem) - Use 40-bit values (Gives you a lot more runway, but costs CPU)
- virtualritz 5y ago
StefanKarpinski 5y ago
Fully agree with this, although one case where Java’s lack of unsigned integers is a quite bad is in representing binary data because the byte type is signed. I recently had cause to write some Java code that did a lot of modular arithmetic with arrays of bytes and oof, what a mess. There were four or five subtle bugs caused by the unexpected signedness of the byte type. I also wrote the same thing in Julia using UInt8 vectors and had no such bugs.
- jstimpfle 5y ago
  I don't write Java, but a good way to deal with this might be to cast to int (a larger type) immediately after loading a byte and before working with it. So, complexity-wise, the overhead of signed byte vs unsigned byte should be at most two casts, and often casting to a larger type would be required anyway even for operations with unsigned byte values.
- xxs 5y ago
  Java technically doesn't even have non-array bytes (or short), all types are just that integers: either 4 or 8 bytes. So if you see a declaration of byte/short (and often times char) is likely a mistake. The correct way is using ints, e.g. "int x = b[i] & 0xff;". Pretty much you should do math with int and long only. java.util.Integer/Long nowadays have static methods to work with unsigned types if you need them. Once you remember that you need (the annoying) "0xff", you'll be free from errors. It has been there for 24 or so years. Other than that wrap your arrays into ByteBuffer and use provided utility.
  - charleslmunger 5y ago
    Citation needed? Java primitive fields are padded for alignment, but that's not unique to java. https://www.baeldung.com/java-memory-layout#layout https://www.baeldung.com/java-memory-layout#layout
    - xxs 5y ago
      It's pretty much a part of the bytecode spec [0]. The stack model has only 4 or 8 bytes long operands. Yes, it can convert int to byte, as method signitures actually need the cast. However a operation involving two bytes (say add, sub, shift) would result into an int, and requires a downcast back to byte. "b+=anotherByte" sums b and anotherByte and then downcasts it automatically. The memory layout of the classes is quite immaterial here as the operation do happen within CPU registers. [0]: https://docs.oracle.com/javase/specs/jvms/se12/html/index.html https://docs.oracle.com/javase/specs/jvms/se12/html/index.ht... [1]: https://en.wikipedia.org/wiki/List_of_Java_bytecode_instructions https://en.wikipedia.org/wiki/List_of_Java_bytecode_instruct...
xondono 5y ago
A lot of processors around you are still 16 and 32 bits. I’ve hit the overflow a lot too. > At the end of the day, code is for people, so keep it easy for people. This is a dangerous heuristic, because people here means exclusively developers. It’s easy for us to trade annoying but mostly harmless bugs for rare but harmful ones.
- kstenerud 5y ago
  Yes, I hit it a lot during my Z80 days, too. I also had to do manual memory management using a memory-mapped page register. We all have our constraints we must operate under, and a bag of tools to help deal with those constraints. But "unsigned by default" won't save you here; all it does is give you a little more breathing room for your special case code (and hey, maybe that doubled positive integer space is all you needed, in which case congrats). But all of the other problems are still there. That's why I'm saying that "unsigned by default" is backwards. It should be "signed by default", and unsigned for special situations that warrant it.
tored 5y ago
Actually I had a colleague that always complained to me that Java didn't have unsigned types. He was of course a C++ programmer.
- xxs 5y ago
  c'mon java does have an unsigned type, just that it's 16bit only (char)
masklinn 5y ago
> I can't think of a single time I've heard someone complain "I wish Java's long type supported up to 16 quintillion instead of a paltry 8 quintillion" That Java has no support for unsigned integers is a very common complaint.
- marginalia_nu 5y ago
  It's one of my biggest complaints attempting to do systems programming in the language, that and the lack of 64bit mmap.
  - xxs 5y ago
    I have no issue with the lack of unsigned - that's ok. It's a trivial workaround, as it's rare enough. However the lack of 64bit indices over memory mapped files blows indeed, with no easy (performant) replacement. It's doable but it'd require a shim over an array of ByteBuffer with the double indirection and pretty much inability for the JIT to optimize away all the bound checks.
  - hashmash 5y ago
    I've had to resort to rolling my own mapping layer using JNA. Fortunately, 64-bit mmap support is being added as part of the Panama foreign-memory project.
spacechild1 5y ago
> Integer overflows are so rare that I can't even remember one happening with 64-bit values. Underflow on the other hand... I've encountered that plenty. I totally agree! After some horrible bugs caused by unsigned integers in innocent looking C/C++ code, I'm 100% in the signed camp. The only time I use unsigned is when I need well-defined overflow behavior (which is rare). As a bonus, using signed loop counters in C/C++ code enables certain kinds of compiler optimizations (based on the fact that signed integer overflow is undefined behavior): http://blog.llvm.org/2011/05/what-every-c-programmer-should-know.html#signed_overflow http://blog.llvm.org/2011/05/what-every-c-programmer-should-...
- cwzwarich 5y ago
  The loop optimization you point to can be done for unsigned loop counters even without exploiting undefined behavior. It’s part not why the standard C practice was to use unsigned loop counters.
colanderman 5y ago
A bonus is that you can enable UBSAN to automatically and efficiently catch signed integer overflow, without changing language semantics (since, unlike unsigned overflow, signed overflow is undefined in C/C++). Here's an earlier post of mine showing how: https://news.ycombinator.com/item?id=24578534 https://news.ycombinator.com/item?id=24578534
kevin_thibedeau 5y ago
It isn't about squeezing out extra range. A silent overflow mechanism with predictable behavior is useful and can't be cheaply replicated with signed numbers. Timestamp arithmetic is a common case where unsigned integers with silent overflow is superior. You can avoid bounds checks and special cases by always computing a delta between a newer time and an earlier time. As long as you can guarantee that the sample period is less than the clock rollover period there will never be any ambiguity. Ada tried imposing the purity of signed integers and had to add modular types to get these semantics because of their utility.
dc-programmer 5y ago
I’ve seen projects that use a mix of unsigned int, size_t, and off_t variable declarations with implicit conversions between them. This seems like a security hazard if you are handling user input because just checking the original input may be insufficient. size_t in general seems extra ridiculous because max(size_t) is a common sentinel value returned by methods and if you aren’t careful and add to max(size_t) it wraps around to -1.
pjmlp 5y ago
> For me as a language designer, which I don't really count myself as these days, what "simple" really ended up meaning was could I expect J. Random Developer to hold the spec in his head. That definition says that, for instance, Java isn't -- and in fact a lot of these languages end up with a lot of corner cases, things that nobody really understands. Quiz any C developer about unsigned, and pretty soon you discover that almost no C developers actually understand what goes on with unsigned, what unsigned arithmetic is. Things like that made C complex. The language part of Java is, I think, pretty simple. The libraries you have to look up. http://www.gotw.ca/publications/c_family_interview.htm http://www.gotw.ca/publications/c_family_interview.htm Gosling on why Java only has signed arithmetic.

cma 5y ago

> I can't think of a single time where I've dealt with numbers that exceed a trillion in either direction. A big one is dealing with currencies like Japanese yen in financial software.

kstenerud 5y ago
Absolutely! But how about a quintillion? Signed 64-bit integers give you 8 quintillion in the positive integer space, and unsigned gives 16 quintillion. The question is: Will the difference between 8 and 16 quintillion matter enough to say "unsigned by default"? But currency types need to be signed anyway, so I guess that's that ;-)

virtualritz 5y ago

I think types should express intend. This adds more 'value' to the code for people than allowing them to switch their brains off because something is 'unlikely' to happen. A large class of bugs exists because people assume stuff is unlikely. I also think one has to distinguish between API and actual code. If you write a lib that takes inputs that should never be negative you can express this without writing a single line of documentation about it (or if you do, requiring the user to read it) by using unsigned types in these places. You can still use signed types in your actual library code if you want/must/believe in that but do not pollute the outside API with them if unnecessary.

kstenerud 5y ago
In a perfect world, this would be so. And if there were types that were performant AND could enforce such invariants without the downsides, I'd welcome them gladly. But primitive types come with baggage and limitations, so you have to make compromises. And the compromises when using unsigned ints are just too costly in my opinion. They're a good tool for specific use cases, but not as good as signed for general purpose code.
- virtualritz 5y ago
  Could you list some of the downsides & compromises in the case I made?
  - kstenerud 5y ago
    - silent unintended underflow behavior - dangers of mixed signed/unsigned code (signed values MUST exist, but unsigned values don't have to) - more cognitive load to read and understand unsigned underflow detection code + issues once you exceed the halfway point This is why Java offered signed integer types only (I think they should have allowed unsigned as well, but for different reasons)
    - virtualritz 5y ago
      > silent unintended underflow behavior Please show me an example in the wild that could not be trivially caught/alleviated. With 'trivially' I mean the code that needs to be written to prevent it, not the thinking you need to do to consider it. As for the latter: this is what I get paid for. Writing the code out is necessary but not what I really get paid for (unless I were a very slow typist, that is). I find that most of the time I was simply too lazy. Or rather: I wrote C/C++ code for 20+ years. I used int and didn't think (unintended rhyme). Now I write (mostly) Rust I find myself considering the edge cases regularly and also covering them because the compiler forces me to. See also this comment [1]. I.e. a - 1 + b where a is always positive (incl. 0) and b is always > 1 'just' works with int (and unsigned [overflow] too!) in C/C++. In Rust you will get a panic when a = 0 which will then make you think what you're doing and simply reorder this to a + b - 1. I find that this is the 'cognitive load' that needs to be applied commonly and I'm pretty fine with that. > dangers of mixed signed/unsigned code Alleviated by simply not mixing. There is a reason languages like Rust simply do not allow this without explicit typecasts. And those, in cases where one type is signed and the other is not and they have equal width translate to "all bets are off". > more cognitive load to read and understand unsigned underflow detection code [...] See first point. IMHO this is simply the same point expressed differently. Underflow happens when you subtract. Ensure you do not subtract a larger value from a smaller. This is as easy as writing max(a, b) - min(a, b) or what author of [1] does. Same as abs(a - b) but works with unsigned. Cognitive load? I do not see any and I agree with the author of the article that this is easier to read. [1] https://news.ycombinator.com/item?id=29767877 https://news.ycombinator.com/item?id=29767877

berkut 5y ago

I've hit the 2147483647 limit of indices occasionally in VFX/Graphics with large datasets: memory efficiency was also important, so 32-bit was a necessity (random access is required, so can't dynamically pack), so I tend to use unsigned these days (uint32_t) within arrays, but native 64-bit lengths (normally size_t) as standard local variables, and I normally go with unsigned there as well. Several APIs in the VFX arena (Pixar OpenSubD, Pixar USD, and Foundry's Katana GeoLib API) all use signed ints for indices which I've occasionally argued against, with the response being "Google's coding standard says don't use them".

kstenerud 5y ago
Yup, and that's a case of an API not aging well. But even in that case, switching to unsigned is only kicking the can down the road, not solving the problem. You'd get another year or two and then hit the 4294967295 limit. As for 64-bit values, unless you can foresee requiring > 8 quintillion, unsigned isn't buying you anything.
- mjevans 5y ago
  Resource consumption does matter though. For example CPU caches or the quantity of RAM within a system. If their workset does fit within 32 bits of index the intentional design choice very likely correlates to a measurable improvement in speed due to cache locality and system resource utilization.
  - kstenerud 5y ago
    Indeed it does matter! As I said this is an example of an API that hasn't aged well (when they devised it, they probably didn't envision this happening). There are a number of ways to handle this: - Use unsigned 32-bit (gives you a little bit more runway, but you'll still hit the end hard) - Use 64-bit values (ends your problem once and for all, but costs double the memory so now you have a new problem) - Use 40-bit values (Gives you a lot more runway, but costs CPU)
virtualritz 5y ago

StefanKarpinski 5y ago

Fully agree with this, although one case where Java’s lack of unsigned integers is a quite bad is in representing binary data because the byte type is signed. I recently had cause to write some Java code that did a lot of modular arithmetic with arrays of bytes and oof, what a mess. There were four or five subtle bugs caused by the unexpected signedness of the byte type. I also wrote the same thing in Julia using UInt8 vectors and had no such bugs.

jstimpfle 5y ago
I don't write Java, but a good way to deal with this might be to cast to int (a larger type) immediately after loading a byte and before working with it. So, complexity-wise, the overhead of signed byte vs unsigned byte should be at most two casts, and often casting to a larger type would be required anyway even for operations with unsigned byte values.
xxs 5y ago
Java technically doesn't even have non-array bytes (or short), all types are just that integers: either 4 or 8 bytes. So if you see a declaration of byte/short (and often times char) is likely a mistake. The correct way is using ints, e.g. "int x = b[i] & 0xff;". Pretty much you should do math with int and long only. java.util.Integer/Long nowadays have static methods to work with unsigned types if you need them. Once you remember that you need (the annoying) "0xff", you'll be free from errors. It has been there for 24 or so years. Other than that wrap your arrays into ByteBuffer and use provided utility.
- charleslmunger 5y ago
  Citation needed? Java primitive fields are padded for alignment, but that's not unique to java. https://www.baeldung.com/java-memory-layout#layout https://www.baeldung.com/java-memory-layout#layout
  - xxs 5y ago
    It's pretty much a part of the bytecode spec [0]. The stack model has only 4 or 8 bytes long operands. Yes, it can convert int to byte, as method signitures actually need the cast. However a operation involving two bytes (say add, sub, shift) would result into an int, and requires a downcast back to byte. "b+=anotherByte" sums b and anotherByte and then downcasts it automatically. The memory layout of the classes is quite immaterial here as the operation do happen within CPU registers. [0]: https://docs.oracle.com/javase/specs/jvms/se12/html/index.html https://docs.oracle.com/javase/specs/jvms/se12/html/index.ht... [1]: https://en.wikipedia.org/wiki/List_of_Java_bytecode_instructions https://en.wikipedia.org/wiki/List_of_Java_bytecode_instruct...

xondono 5y ago

A lot of processors around you are still 16 and 32 bits. I’ve hit the overflow a lot too. > At the end of the day, code is for people, so keep it easy for people. This is a dangerous heuristic, because people here means exclusively developers. It’s easy for us to trade annoying but mostly harmless bugs for rare but harmful ones.

kstenerud 5y ago
Yes, I hit it a lot during my Z80 days, too. I also had to do manual memory management using a memory-mapped page register. We all have our constraints we must operate under, and a bag of tools to help deal with those constraints. But "unsigned by default" won't save you here; all it does is give you a little more breathing room for your special case code (and hey, maybe that doubled positive integer space is all you needed, in which case congrats). But all of the other problems are still there. That's why I'm saying that "unsigned by default" is backwards. It should be "signed by default", and unsigned for special situations that warrant it.

tored 5y ago

Actually I had a colleague that always complained to me that Java didn't have unsigned types. He was of course a C++ programmer.

xxs 5y ago
c'mon java does have an unsigned type, just that it's 16bit only (char)

masklinn 5y ago

> I can't think of a single time I've heard someone complain "I wish Java's long type supported up to 16 quintillion instead of a paltry 8 quintillion" That Java has no support for unsigned integers is a very common complaint.

marginalia_nu 5y ago
It's one of my biggest complaints attempting to do systems programming in the language, that and the lack of 64bit mmap.
- xxs 5y ago
  I have no issue with the lack of unsigned - that's ok. It's a trivial workaround, as it's rare enough. However the lack of 64bit indices over memory mapped files blows indeed, with no easy (performant) replacement. It's doable but it'd require a shim over an array of ByteBuffer with the double indirection and pretty much inability for the JIT to optimize away all the bound checks.
- hashmash 5y ago
  I've had to resort to rolling my own mapping layer using JNA. Fortunately, 64-bit mmap support is being added as part of the Panama foreign-memory project.

spacechild1 5y ago

> Integer overflows are so rare that I can't even remember one happening with 64-bit values. Underflow on the other hand... I've encountered that plenty. I totally agree! After some horrible bugs caused by unsigned integers in innocent looking C/C++ code, I'm 100% in the signed camp. The only time I use unsigned is when I need well-defined overflow behavior (which is rare). As a bonus, using signed loop counters in C/C++ code enables certain kinds of compiler optimizations (based on the fact that signed integer overflow is undefined behavior): http://blog.llvm.org/2011/05/what-every-c-programmer-should-know.html#signed_overflow http://blog.llvm.org/2011/05/what-every-c-programmer-should-...

cwzwarich 5y ago
The loop optimization you point to can be done for unsigned loop counters even without exploiting undefined behavior. It’s part not why the standard C practice was to use unsigned loop counters.

colanderman 5y ago

A bonus is that you can enable UBSAN to automatically and efficiently catch signed integer overflow, without changing language semantics (since, unlike unsigned overflow, signed overflow is undefined in C/C++). Here's an earlier post of mine showing how: https://news.ycombinator.com/item?id=24578534 https://news.ycombinator.com/item?id=24578534

kevin_thibedeau 5y ago

It isn't about squeezing out extra range. A silent overflow mechanism with predictable behavior is useful and can't be cheaply replicated with signed numbers. Timestamp arithmetic is a common case where unsigned integers with silent overflow is superior. You can avoid bounds checks and special cases by always computing a delta between a newer time and an earlier time. As long as you can guarantee that the sample period is less than the clock rollover period there will never be any ambiguity. Ada tried imposing the purity of signed integers and had to add modular types to get these semantics because of their utility.

dc-programmer 5y ago

I’ve seen projects that use a mix of unsigned int, size_t, and off_t variable declarations with implicit conversions between them. This seems like a security hazard if you are handling user input because just checking the original input may be insufficient. size_t in general seems extra ridiculous because max(size_t) is a common sentinel value returned by methods and if you aren’t careful and add to max(size_t) it wraps around to -1.

pjmlp 5y ago

> For me as a language designer, which I don't really count myself as these days, what "simple" really ended up meaning was could I expect J. Random Developer to hold the spec in his head. That definition says that, for instance, Java isn't -- and in fact a lot of these languages end up with a lot of corner cases, things that nobody really understands. Quiz any C developer about unsigned, and pretty soon you discover that almost no C developers actually understand what goes on with unsigned, what unsigned arithmetic is. Things like that made C complex. The language part of Java is, I think, pretty simple. The libraries you have to look up. http://www.gotw.ca/publications/c_family_interview.htm http://www.gotw.ca/publications/c_family_interview.htm Gosling on why Java only has signed arithmetic.