8 ms·

Can't quite tell if this is a joke, but here's a related "story about a bug" from Doug Crockford [0]: I made a bug once, and I need to tell you about it.

by gavinpc 12y ago

Can't quite tell if this is a joke, but here's a related "story about a bug" from Doug Crockford [0]: I made a bug once, and I need to tell you about it. So, in 2001, I wrote a reference library for JSON, in Java, and in it, I had this line private int index that created a variable called "index" which counted the number of characters in the JSON text that we were parsing, and it was used to produce an error message. Last year, I got a bug report from somebody. It turns out that they had a JSON text which was several gigabytes in size, and they had a syntax error past two gigabytes, and my JSON library did not properly report where the error was — it was off by two gigabytes, which, that's kind of a big error, isn't it? And the reason was, I used an int. Now, I can justify my choice in doing that. At the time that I did it, two gigabytes was a really big disk drive, and my use of JSON still is very small messages. My JSON messages are rarely bigger than a couple of K. And — a couple gigs, yeah that's about a thousand times bigger than I need, I should be all right. No, turns out it wasn't enough. You might think well, one bug in 12 years you're doing pretty good. And I'm saying no, that's not good enough. I want my programs to be perfect. I don't want anything to go wrong. And in this case it went wrong simply because *Java gave me a choice that I didn't need, and I made the wrong choice*. [0] https://www.youtube.com/watch?v=bo36MrBfTk4&t=38m https://www.youtube.com/watch?v=bo36MrBfTk4&t=38m EDIT: is there a reference for formatting comments? I've never been able to find one.

talles 12y ago
I loved how he ended *Java gave me a choice that I didn't need, and I made the wrong choice*
- lerchmo 12y ago
  What if someone else needs that choice? like anyone doing compression?
  - phillmv 12y ago
    Let those people eat cake, and in the meantime protect me from my own misbehaviour.
  - lukeschlather 12y ago
    When you're storing an integer, ideally "MAX_INT" is not something you have to worry about. If you're doing compression you should use some sort of raw bytes type. At least I think that's the platonic ideal.
  - aidenn0 12y ago
    Make the default numeric type effectively unbounded, and allow those who need it to choose more compact types when needed. This is what many languages do, and it is possible to both generate efficient code when needed, and correct code is more likely to happen.
batuhanicoz 12y ago
Reference for comment formatting can be found here: https://news.ycombinator.com/formatdoc https://news.ycombinator.com/formatdoc
danbruc 12y ago
He did not need the choice but others do. And he is wrong when he says it makes no difference whether you use a byte or eight of them. Yes, it will take the same amount of time to add two of them but it will also cost eight times more cache space and memory bandwidth to move them around. It may not be an issue if you have a single number or ten of them but it certainly becomes one if you have an array with millions or billions of them.
- philosophus 12y ago
  I don't believe he said it makes no difference.
  - danbruc 12y ago
    »But in todays CPUs there is no advantage using the short thing. You can add 64 bits or 8 bits, takes the same amount of time. And you look up what is the cash value of having saved seven bytes on a number. When you add that up it is zero. So there is no benefit.« [1] As long as you are concerned with adding a bit of eye candy and interactivity to a web page this may be true enough to get away with the JavaScript way of making every number a double precision floating point number but there a other domains where this will not fly. And even in the world of JavaScript asm.js is trying hard to overcome this limitation. [1] http://www.youtube.com/watch?v=bo36MrBfTk4&t=42m17s http://www.youtube.com/watch?v=bo36MrBfTk4&t=42m17s
    - coldtea 12y ago
      >but there a other domains where this will not fly. Sure, and I believe everybody, including Doug, knows this not so subtle distinction. He wasn't talking about programming HPC for NASA.
      - danbruc 12y ago
        He is comparing Java and JavaScript and implies that it is a bad choice of Java to offer several options. And given the broad range of applications Java is used for I don't think this is a justifiable opinion.
cstavish 12y ago
>> Java gave me a choice that I didn't need, and I made the wrong choice What if Java gave an arguably more useful choice--whether to use a signed or unsigned integer?
- deleted 12y ago
  [deleted]
  - danbruc 12y ago
    But only because asm.js works hard to make JavaScript differentiate between integers and floating point numbers.
    - deleted 12y ago
      [deleted]
      - danbruc 12y ago
        This is a decision a compiler can never make in a reliable way because it entirely depends on the actual input and is not known until runtime. You may get away with dynamic recompilation when you realize that the input is not what you assumed when you compiled the code but I really doubt that this is a smart and efficient way to go about it. And asm.js is no evidence for the no side - the information is in the original source code, it does matter and asm.js works around the JavaScript limitation and makes this information available to the JavaScript compiler.
- whyever 12y ago
  If you use an unsigned 32 bit integer instead of a signed one then you run into problems at 4 instead 2 gigabytes.
  - takeda 12y ago
    But who would have JSON files bigger than 4GB? ;)
tdsamardzhiev 12y ago
If you are working with files bigger than 2GB, hoping they're smaller than 4GB is NOT a good habit. And I certainly don't believe there are programmers making only 1 mistake for 12 years. I believe he's just making a joke, or using the example as a means to an end.
gioele 12y ago
Interesting, this is similar to the discussion going on for "int" in Rust (or the exact opposite, depending on how you view it). [1, 2] One the one hand "implicit int" are being phased out in favour of explicit int size. Your variables cannot be `int` anymore, you have to sit down, think and choose: u8? u16? u32? i32? u64? i64? This avoids all the pains of programs behaving differently or crashing when compiled on different architectures. On the other hand, a new "native integer for sizes that do not matter, minimum 32 bits" is being brewed, for example for pointer offsets or collection sizes. The idea is that you will not be able to have a collection with more of 2^32 elements in a 32-bit architecture nor more than 2^64 in a 64-bit architecture. After this discussion, my hope is to see the introduction of a fast-ish dynamic bigint (that starts native and grow up to 256 or 512 bits) that can be used in all the cases where you do not care about the exact size type, yet you want to be future-proof (this `private int index` fits this case, IMO). [1] http://discuss.rust-lang.org/t/if-int-has-the-wrong-size/454/23 http://discuss.rust-lang.org/t/if-int-has-the-wrong-size/454... [2] https://github.com/rust-lang/rfcs/pull/464 https://github.com/rust-lang/rfcs/pull/464
- Narishma 12y ago
  Wouldn't that limit rust to 32-bit or more architectures?
  - tinco 12y ago
    It would affect the standard library of rust, not rust itself I think. Second, the word-size matching the architecture has only a slight performance impact, i.e. operations on a 64-bit word on a 32-bit architecture take more cycles/instructions than 64-bit words on a 64-bit architecture. It's only important for things like pointers that they match the size of the addressing space, and not even that is a very hard constraint, just a very convenient one.
  - maxlybbert 12y ago
    16 bit architectures are weird enough that a lot of tools don't support them. And 8 bit architectures are basically only used for things like Arduino nowadays. However, using a 32 bit number to store values that will never be larger than 16 bits isn't that bad, it's just very slow.
Tepix 12y ago
He's pushing for DEC64 now (http://DEC64.com http://DEC64.com)
lmm 12y ago
So for that kind of software you use Python. Doesn't everyone know that? Java gives you that choice because it's for the kind of software where you need that choice.
robert_tweed 12y ago
I'm guessing (since this is Doug Crockford talking about JSON) that this was in reference to how JavaScript does things differently, in that it just stores everything as floats, which are quite capable of representing integers within the 32-bit range anyway. However, an overflow to floating point isn't necessarily an improvement because, while a float will hold bigger numbers, it does so with limited precision and sometimes that lack of precision will cause bugs too. Probably more often, in fact. In the example given it wouldn't be so bad, but you'd only get an approximate indication of where the error occurred rather than a specific line/character. So of course, whatever is reporting the error would now need to understand and handle the much more complex scenario of "fuzzy" location information instead of a simple unique index to a specific character. Depending on what it then needs to do with that information, the complexity could spiral from there. If you want to just have things work no matter what, you have no choice but to use bignums. I was wondering about this recently, so did some benchmarks in Clojure. The performance was horrible, so frankly this is still not a viable alternative. Maybe in 10 years time, if every CPU has a bignum coprocessor by then. Also, there are times, particularly in low-level graphics programming or cryptography, where you actually want integer modulo arithmetic, or to be able to do bitwise booleans predictably. In those cases, JavaScript-style loose typing can be a huge pain. BTW, I've been a big advocate of JavaScript for about as long as Doug Crockford, so my point isn't that JavaScript-style type handling is bad: just that it's very far from a silver bullet.
ColinWright 12y ago
This is a FAQ. Oddly enough, the "FAQ" link at the bottom of the page takes you to the FAQ, in which it says: ======== What kind of formatting can you use in comments? http://news.ycombinator.com/formatdoc http://news.ycombinator.com/formatdoc ======== Was that what you were looking for?
peterashford 12y ago
I'm constantly amused how Java is supposedly stupid for protecting programmers from things they shouldn't do and yet also stupid for not protecting programmers from things they shouldn't do. IMHO Java makes some choices about safety. If you don't agree with those choices, use a different tool. It doesn't make Java wrong for having a different opinion. Likewise I wouldn't berate C for being too low level or Ruby for favouring readability over performance.

by gavinpc 12y ago

talles 12y ago

I loved how he ended *Java gave me a choice that I didn't need, and I made the wrong choice*

lerchmo 12y ago
What if someone else needs that choice? like anyone doing compression?
- phillmv 12y ago
  Let those people eat cake, and in the meantime protect me from my own misbehaviour.
- lukeschlather 12y ago
  When you're storing an integer, ideally "MAX_INT" is not something you have to worry about. If you're doing compression you should use some sort of raw bytes type. At least I think that's the platonic ideal.
- aidenn0 12y ago
  Make the default numeric type effectively unbounded, and allow those who need it to choose more compact types when needed. This is what many languages do, and it is possible to both generate efficient code when needed, and correct code is more likely to happen.

batuhanicoz 12y ago

Reference for comment formatting can be found here: https://news.ycombinator.com/formatdoc https://news.ycombinator.com/formatdoc

danbruc 12y ago

He did not need the choice but others do. And he is wrong when he says it makes no difference whether you use a byte or eight of them. Yes, it will take the same amount of time to add two of them but it will also cost eight times more cache space and memory bandwidth to move them around. It may not be an issue if you have a single number or ten of them but it certainly becomes one if you have an array with millions or billions of them.

philosophus 12y ago
I don't believe he said it makes no difference.
- danbruc 12y ago
  »But in todays CPUs there is no advantage using the short thing. You can add 64 bits or 8 bits, takes the same amount of time. And you look up what is the cash value of having saved seven bytes on a number. When you add that up it is zero. So there is no benefit.« [1] As long as you are concerned with adding a bit of eye candy and interactivity to a web page this may be true enough to get away with the JavaScript way of making every number a double precision floating point number but there a other domains where this will not fly. And even in the world of JavaScript asm.js is trying hard to overcome this limitation. [1] http://www.youtube.com/watch?v=bo36MrBfTk4&t=42m17s http://www.youtube.com/watch?v=bo36MrBfTk4&t=42m17s
  - coldtea 12y ago
    >but there a other domains where this will not fly. Sure, and I believe everybody, including Doug, knows this not so subtle distinction. He wasn't talking about programming HPC for NASA.
    - danbruc 12y ago
      He is comparing Java and JavaScript and implies that it is a bad choice of Java to offer several options. And given the broad range of applications Java is used for I don't think this is a justifiable opinion.

cstavish 12y ago

>> Java gave me a choice that I didn't need, and I made the wrong choice What if Java gave an arguably more useful choice--whether to use a signed or unsigned integer?

deleted 12y ago
[deleted]
- danbruc 12y ago
  But only because asm.js works hard to make JavaScript differentiate between integers and floating point numbers.
  - deleted 12y ago
    [deleted]
    - danbruc 12y ago
      This is a decision a compiler can never make in a reliable way because it entirely depends on the actual input and is not known until runtime. You may get away with dynamic recompilation when you realize that the input is not what you assumed when you compiled the code but I really doubt that this is a smart and efficient way to go about it. And asm.js is no evidence for the no side - the information is in the original source code, it does matter and asm.js works around the JavaScript limitation and makes this information available to the JavaScript compiler.
whyever 12y ago
If you use an unsigned 32 bit integer instead of a signed one then you run into problems at 4 instead 2 gigabytes.
- takeda 12y ago
  But who would have JSON files bigger than 4GB? ;)

tdsamardzhiev 12y ago

If you are working with files bigger than 2GB, hoping they're smaller than 4GB is NOT a good habit. And I certainly don't believe there are programmers making only 1 mistake for 12 years. I believe he's just making a joke, or using the example as a means to an end.

gioele 12y ago

Interesting, this is similar to the discussion going on for "int" in Rust (or the exact opposite, depending on how you view it). [1, 2] One the one hand "implicit int" are being phased out in favour of explicit int size. Your variables cannot be `int` anymore, you have to sit down, think and choose: u8? u16? u32? i32? u64? i64? This avoids all the pains of programs behaving differently or crashing when compiled on different architectures. On the other hand, a new "native integer for sizes that do not matter, minimum 32 bits" is being brewed, for example for pointer offsets or collection sizes. The idea is that you will not be able to have a collection with more of 2^32 elements in a 32-bit architecture nor more than 2^64 in a 64-bit architecture. After this discussion, my hope is to see the introduction of a fast-ish dynamic bigint (that starts native and grow up to 256 or 512 bits) that can be used in all the cases where you do not care about the exact size type, yet you want to be future-proof (this `private int index` fits this case, IMO). [1] http://discuss.rust-lang.org/t/if-int-has-the-wrong-size/454/23 http://discuss.rust-lang.org/t/if-int-has-the-wrong-size/454... [2] https://github.com/rust-lang/rfcs/pull/464 https://github.com/rust-lang/rfcs/pull/464

Narishma 12y ago
Wouldn't that limit rust to 32-bit or more architectures?
- tinco 12y ago
  It would affect the standard library of rust, not rust itself I think. Second, the word-size matching the architecture has only a slight performance impact, i.e. operations on a 64-bit word on a 32-bit architecture take more cycles/instructions than 64-bit words on a 64-bit architecture. It's only important for things like pointers that they match the size of the addressing space, and not even that is a very hard constraint, just a very convenient one.
- maxlybbert 12y ago
  16 bit architectures are weird enough that a lot of tools don't support them. And 8 bit architectures are basically only used for things like Arduino nowadays. However, using a 32 bit number to store values that will never be larger than 16 bits isn't that bad, it's just very slow.

Tepix 12y ago

He's pushing for DEC64 now (http://DEC64.com http://DEC64.com)

lmm 12y ago

So for that kind of software you use Python. Doesn't everyone know that? Java gives you that choice because it's for the kind of software where you need that choice.

robert_tweed 12y ago

I'm guessing (since this is Doug Crockford talking about JSON) that this was in reference to how JavaScript does things differently, in that it just stores everything as floats, which are quite capable of representing integers within the 32-bit range anyway. However, an overflow to floating point isn't necessarily an improvement because, while a float will hold bigger numbers, it does so with limited precision and sometimes that lack of precision will cause bugs too. Probably more often, in fact. In the example given it wouldn't be so bad, but you'd only get an approximate indication of where the error occurred rather than a specific line/character. So of course, whatever is reporting the error would now need to understand and handle the much more complex scenario of "fuzzy" location information instead of a simple unique index to a specific character. Depending on what it then needs to do with that information, the complexity could spiral from there. If you want to just have things work no matter what, you have no choice but to use bignums. I was wondering about this recently, so did some benchmarks in Clojure. The performance was horrible, so frankly this is still not a viable alternative. Maybe in 10 years time, if every CPU has a bignum coprocessor by then. Also, there are times, particularly in low-level graphics programming or cryptography, where you actually want integer modulo arithmetic, or to be able to do bitwise booleans predictably. In those cases, JavaScript-style loose typing can be a huge pain. BTW, I've been a big advocate of JavaScript for about as long as Doug Crockford, so my point isn't that JavaScript-style type handling is bad: just that it's very far from a silver bullet.

ColinWright 12y ago

This is a FAQ. Oddly enough, the "FAQ" link at the bottom of the page takes you to the FAQ, in which it says: ======== What kind of formatting can you use in comments? http://news.ycombinator.com/formatdoc http://news.ycombinator.com/formatdoc ======== Was that what you were looking for?

peterashford 12y ago

I'm constantly amused how Java is supposedly stupid for protecting programmers from things they shouldn't do and yet also stupid for not protecting programmers from things they shouldn't do. IMHO Java makes some choices about safety. If you don't agree with those choices, use a different tool. It doesn't make Java wrong for having a different opinion. Likewise I wouldn't berate C for being too low level or Ruby for favouring readability over performance.