7 ms·

I have to question the value of CS educations when a post of this nature pops up every couple of weeks, as if it is news that floating point arithmetic as imple

by heresy 16y ago

I have to question the value of CS educations when a post of this nature pops up every couple of weeks, as if it is news that floating point arithmetic as implemented today, is by its nature an approximation, and that this is more noticeable the less bits you have to play with, when working with values that can't be represented neatly. Financial arithmetic? Convert to smallest unit and use integers or the currency data type du jour in your language, and don't act surprised when operations on 32-bit floating point don't yield the intuitively correct values. If you understood the representation format, you'd understand why.

rlpb 16y ago
It would be nice if more languages supported fractional rational number arithmetic as a built-in type. Converting into fractional units and back again shouldn't be as much of a pain as it is in most languages at the moment.
- 3pt14159 16y ago
  It is easy in Ruby and Python.
- teamonkey 16y ago
  One of the things I love about Scheme is not having to worry about number formats. It's a number, it is what it is. You never run out of digits, you never have to worry about precision errors. It removed a layer of stress I never knew existed.
  - eru 16y ago
    Actually you are talking about a specific implementation of Scheme. Probably PLT Scheme? An standard compliant implementation can just stick to floating point for everything as the bare miminum. PLT Scheme is the Scheme to go for, though.
    - teamonkey 16y ago
      Any fully-compliant R5RS really. I'm talking about the concepts of exactness and the numerical tower in particular. The notion that an integer is a subset of a real number instead of a whole separate type is conceptually very nice. http://www.schemers.org/Documents/Standards/R5RS/HTML/r5rs-Z-H-9.html#%_sec_6.2.1 http://www.schemers.org/Documents/Standards/R5RS/HTML/r5rs-Z...
      - eru 16y ago
        Yes, and: "Implementations of Scheme are not required to implement the whole tower of subtypes given in section 6.2.1, but they must implement a coherent subset consistent with both the purposes of the implementation and the spirit of the Scheme language. For example, an implementation in which all numbers are real may still be quite useful." (From your URL)
        teamonkey 16y ago
        True, but I think you're picking at hairs here. The suggestion in the R5RS spec is that you shouldn't need to worry about the segregation of integer and real types (unless you need to). And indeed, why should you? It's a conceptual shift away from char, int, long, float and double, which exist only because they're easier for the computer to deal with. Scheme was the first language I used that attempted to treat numbers like numbers. Besides, Exactness is required in R5RS and the full numerical stack is no longer optional in R6RS.
        eru 16y ago
        Yes, even Guile seems to have acquired a proper tower nowadays.
Cushman 16y ago
I don't think it's just that it's an approximation. We're used to doing math with approximations (0.33333...). What's unintuitive about it is that numbers with very simple finite expressions in decimal (0.3) can only be expressed by repeating binary floating points. Even competent coders who usually work in higher-level languages can sometimes forget that the decimal representations they work with are actually approximations of binary numbers, which is responsible for other seemingly weird behavior (why does ~2 = -3?), and I doubt even most skilled computer scientists are used to converting decimals to arbitrary-precision binary in their head. It is unintuitive, and it's not like it's hard to explain why. "You'd understand if you were smarter" is a cop out.
- heresy 16y ago
  My point is, this is elementary stuff that you need to know as a computer scientist or self-taught hacker. Whether you can convert a decimal number into its binary IEEE-754 equivalent, or not, is besides the point. You just need to know that it is lossy for certain classes of decimal numbers, and how to avoid or mitigate those effects.
  - Cushman 16y ago
    Sure, if you want to have computer hacking be a big part of your life you should know that numbers are binary on computers, and that that has side effects. But I don't think it betrays a serious lack of education that someone who does know that theoretically has never realized that practically means there's no such thing as 0.1 on a computer.
    - jerf 16y ago
      You theoretically shouldn't even have to be a computer scientist. A standard mathematical education ought to teach you that for any given base, the only numbers that can be represented in that base exactly are numbers where the prime factorization of the denominator contains only prime numbers that are also contained in the prime factorization of the base itself, or is simply 1 for integers. Base 10 has 2 and 5. .2 is 1/5, so it's precise. .5 is 1/2, so it's precise. 1/12 has 2 2s in it, which works, and a 3, which doesn't, so 1/12 is recurring. But prime factorization seems to be taught for almost superstitious reasons, as some sort of math trick rather than a fundamental aspect of understanding numbers at even the most basic of levels, with neither students nor teachers nor the curriculum writers really understanding why this is in the lesson plan, so, yeah, I suppose you have to be some sort of super expert genius to swiftly realize that 1/3 can't be represented in a base-2 number. But it shouldn't be that way.
dansingerman 16y ago
I bet many many people who "do a bit of programming" have no formal CS training, and only begin to understand the complexities of floating point arithmetic when they come across a problem like this (and hence ask the question) Oh, this applies to me btw.
- billswift 16y ago
  And they only really remember the issue after it bites them in the ass. Most CS majors, for that matter, probably don't really think about or remember the issue until after it bites them. I would expect a really good programming course (if one exists), to present exercises that actively cause the students to run into these types of problems, but from what I have seen there has been a progressive de-emphasis on "trick" questions and really confusing problems, at least in some schools, in favor of self-esteem and "empowerment".
frobozz 16y ago
I would't necessarily call this a failing of CS education specifically - some, or even most of these questions may have come from self-taught individuals. What does worry me is how many times this question is asked on The Web. This is a sad indictment on the quality of education in general. Given this resource, where information is far easier to find than in any library, there are still so many people who can't be bothered to look things up for themselves.
Roboprog 16y ago
Well, some of us (before uqiquitous home computers) actually pulled out the little manual that came with our calculators to learn about "binary coded decimal", and some didn't. OK, I'm probably giving away too much about my age :-) And some had better things to do anyway... That said, yes, this does get misprogrammed in financial applications very frequently. FWIW, I wrote a little tuturial about this very issue a while back: http://roboprogs.com/devel/2010.02.html http://roboprogs.com/devel/2010.02.html
jimbokun 16y ago
"I have to question the value of CS educations when a post of this nature pops up every couple of weeks..." How do we know it's not freshmen, high school students, and self taught developers asking the question? From what is currently the top hit for that search: "I am writing a program for class project..."
silentbicycle 16y ago
There are a LOT of programmers who don't understand how the fundamentals are implemented, though. I'm not saying you need to use C (or machine language) for everything, but having an idea how the whole stack fits together will show which operations are inherently more expensive, error prone, etc. Two books I'd recommend: _Write Great Code vol. 1: Understanding the Machine_ by Randall Hyde, and _The Elements of Computing Systems_ by Nisan & Schocken (http://www.idc.ac.il/tecs/ http://www.idc.ac.il/tecs/). WGC1 is about things typically learned about the machine while learning assembly (but leaves assembly itself for later). TECS guides you through building a (virtual) computer, starting with simulated NAND gates and building up. (Admittedly, I just got that one and I'm only a few chapters in, but excellent so far.) I think every developer would benefit from knowing at least one low-level language (such as C), and one very high level language (such as Prolog).
- happy4crazy 16y ago
  I'm also a few chapters into The Elements of Computing Systems, and I also think it's excellent so far :) I'm posting my project solutions to my github account, fwiw: http://github.com/happy4crazy/elements_of_computing_systems http://github.com/happy4crazy/elements_of_computing_systems
eru 16y ago
> Financial arithmetic? Convert to smallest unit and use integers or the currency data type du jour in your language, and don't act surprised when operations on 32-bit floating point don't yield the intuitively correct values. Yes, though you may want more precision then that. For accuracy you could use, say, exact rational numbers, and for speed a fixed precision below e.g. Cents might be useful. (Though you probably meant something like nano-Cent when you said, convert to the smallest unit.)
redstripe 16y ago
Look at the answers people give in the search. It doesn't help that every time someone asks this they get a link to something like this: http://docs.sun.com/source/806-3568/ncg_goldberg.html http://docs.sun.com/source/806-3568/ncg_goldberg.html I'm surprised someone hasn't translated it into Klingon or Sanskrit yet. It's pure snobbery that makes people leave responses like this. This document falls into the further reading for enthusiasts category. Nobody should be linking to it as an introductory answer to the question. At least not if they want people to learn and stop asking the question eventually.
anigbrowl 16y ago
I have to question the value of CS educations Many people have no interest in being computer scientists. They want to learn enough software engineering to get something done using a computer. Outside of CS, programming is only a means to an end. Higher level languages ought not to expose the user/programmer to this sort of thing, even at the expense of performance; for every 'why is it that...' question, there are probably several more buggy programs being used in production environments. Also, it's a confusing distraction for kids trying to learn programming, who may not be able to wrap their heads around the theory of FP implementation, although they may be quite competent to handle the task they're trying to implement in software. I used to work with a Motorola 56k series DSP, so named because it offers up to 56 bits of integer precision. Dealing with data that's usually represented as gloating point involves a normalization conversion as you describe for the currency unit which soon becomes second nature. But for newcomers to the platform, that step at first seems like an inefficient limitation, adding complexity to an otherwise simple algorithm. Not everybody wants or needs to know how the underlying hardware works, often they just want to implement some fairly trivial math or logic without worrying about accuracy. I can't bring myself to correct that spelling error :)
- JanezStupar 16y ago
  To me its only an indicator. There's a ton of stuff on similar level that does matter and it adds up. Phenomena like knowing O(n) of basic algorithms and data structures. Understanding basic file formats and how they are implemented. Basic knowledge of network protocols and their building blocks. Basic knowledge of assembler. Understanding how IO happens and how your use case is bound (IO or CPU). Knowledge of character code pages, etc. Nothing of the above is necessary to program in high level language. But if you want to do it well they can't possibly harm you. Or you will get involved with a vendor product that was written by someone who didn't know the above and you will be forced to reverse engineer the goddamn thing just to get your job done. :)
  - anigbrowl 16y ago
    I totally agree where full-time development is concerned. And it's the sort of thing macro languages and so on tend to build in. There are lot of people that might need to do a bit more programming - more elaborate than what a spreadsheet easily allows, but which doesn't justify becoming/hiring a pro. I was surprised to find last year that there are programming languages designed specifically for knitting, for example.
scrame 16y ago
Not so much that this is a question, but that this particular question gets 100+ points on "hacker news", when its not even a question or explanation, just a link to google result of adding numbers in java. If it was at least a homework question on stackoverflow there would at least be a good description, and possibly a reasonable explanation.

by heresy 16y ago

rlpb 16y ago

It would be nice if more languages supported fractional rational number arithmetic as a built-in type. Converting into fractional units and back again shouldn't be as much of a pain as it is in most languages at the moment.

3pt14159 16y ago
It is easy in Ruby and Python.
teamonkey 16y ago
One of the things I love about Scheme is not having to worry about number formats. It's a number, it is what it is. You never run out of digits, you never have to worry about precision errors. It removed a layer of stress I never knew existed.
- eru 16y ago
  Actually you are talking about a specific implementation of Scheme. Probably PLT Scheme? An standard compliant implementation can just stick to floating point for everything as the bare miminum. PLT Scheme is the Scheme to go for, though.
  - teamonkey 16y ago
    Any fully-compliant R5RS really. I'm talking about the concepts of exactness and the numerical tower in particular. The notion that an integer is a subset of a real number instead of a whole separate type is conceptually very nice. http://www.schemers.org/Documents/Standards/R5RS/HTML/r5rs-Z-H-9.html#%_sec_6.2.1 http://www.schemers.org/Documents/Standards/R5RS/HTML/r5rs-Z...
    - eru 16y ago
      Yes, and: "Implementations of Scheme are not required to implement the whole tower of subtypes given in section 6.2.1, but they must implement a coherent subset consistent with both the purposes of the implementation and the spirit of the Scheme language. For example, an implementation in which all numbers are real may still be quite useful." (From your URL)
      - teamonkey 16y ago
        True, but I think you're picking at hairs here. The suggestion in the R5RS spec is that you shouldn't need to worry about the segregation of integer and real types (unless you need to). And indeed, why should you? It's a conceptual shift away from char, int, long, float and double, which exist only because they're easier for the computer to deal with. Scheme was the first language I used that attempted to treat numbers like numbers. Besides, Exactness is required in R5RS and the full numerical stack is no longer optional in R6RS.
        eru 16y ago
        Yes, even Guile seems to have acquired a proper tower nowadays.

Cushman 16y ago

I don't think it's just that it's an approximation. We're used to doing math with approximations (0.33333...). What's unintuitive about it is that numbers with very simple finite expressions in decimal (0.3) can only be expressed by repeating binary floating points. Even competent coders who usually work in higher-level languages can sometimes forget that the decimal representations they work with are actually approximations of binary numbers, which is responsible for other seemingly weird behavior (why does ~2 = -3?), and I doubt even most skilled computer scientists are used to converting decimals to arbitrary-precision binary in their head. It is unintuitive, and it's not like it's hard to explain why. "You'd understand if you were smarter" is a cop out.

heresy 16y ago
My point is, this is elementary stuff that you need to know as a computer scientist or self-taught hacker. Whether you can convert a decimal number into its binary IEEE-754 equivalent, or not, is besides the point. You just need to know that it is lossy for certain classes of decimal numbers, and how to avoid or mitigate those effects.
- Cushman 16y ago
  Sure, if you want to have computer hacking be a big part of your life you should know that numbers are binary on computers, and that that has side effects. But I don't think it betrays a serious lack of education that someone who does know that theoretically has never realized that practically means there's no such thing as 0.1 on a computer.
  - jerf 16y ago
    You theoretically shouldn't even have to be a computer scientist. A standard mathematical education ought to teach you that for any given base, the only numbers that can be represented in that base exactly are numbers where the prime factorization of the denominator contains only prime numbers that are also contained in the prime factorization of the base itself, or is simply 1 for integers. Base 10 has 2 and 5. .2 is 1/5, so it's precise. .5 is 1/2, so it's precise. 1/12 has 2 2s in it, which works, and a 3, which doesn't, so 1/12 is recurring. But prime factorization seems to be taught for almost superstitious reasons, as some sort of math trick rather than a fundamental aspect of understanding numbers at even the most basic of levels, with neither students nor teachers nor the curriculum writers really understanding why this is in the lesson plan, so, yeah, I suppose you have to be some sort of super expert genius to swiftly realize that 1/3 can't be represented in a base-2 number. But it shouldn't be that way.

dansingerman 16y ago

I bet many many people who "do a bit of programming" have no formal CS training, and only begin to understand the complexities of floating point arithmetic when they come across a problem like this (and hence ask the question) Oh, this applies to me btw.

billswift 16y ago
And they only really remember the issue after it bites them in the ass. Most CS majors, for that matter, probably don't really think about or remember the issue until after it bites them. I would expect a really good programming course (if one exists), to present exercises that actively cause the students to run into these types of problems, but from what I have seen there has been a progressive de-emphasis on "trick" questions and really confusing problems, at least in some schools, in favor of self-esteem and "empowerment".

frobozz 16y ago

I would't necessarily call this a failing of CS education specifically - some, or even most of these questions may have come from self-taught individuals. What does worry me is how many times this question is asked on The Web. This is a sad indictment on the quality of education in general. Given this resource, where information is far easier to find than in any library, there are still so many people who can't be bothered to look things up for themselves.

Roboprog 16y ago

Well, some of us (before uqiquitous home computers) actually pulled out the little manual that came with our calculators to learn about "binary coded decimal", and some didn't. OK, I'm probably giving away too much about my age :-) And some had better things to do anyway... That said, yes, this does get misprogrammed in financial applications very frequently. FWIW, I wrote a little tuturial about this very issue a while back: http://roboprogs.com/devel/2010.02.html http://roboprogs.com/devel/2010.02.html

jimbokun 16y ago

"I have to question the value of CS educations when a post of this nature pops up every couple of weeks..." How do we know it's not freshmen, high school students, and self taught developers asking the question? From what is currently the top hit for that search: "I am writing a program for class project..."

silentbicycle 16y ago

There are a LOT of programmers who don't understand how the fundamentals are implemented, though. I'm not saying you need to use C (or machine language) for everything, but having an idea how the whole stack fits together will show which operations are inherently more expensive, error prone, etc. Two books I'd recommend: _Write Great Code vol. 1: Understanding the Machine_ by Randall Hyde, and _The Elements of Computing Systems_ by Nisan & Schocken (http://www.idc.ac.il/tecs/ http://www.idc.ac.il/tecs/). WGC1 is about things typically learned about the machine while learning assembly (but leaves assembly itself for later). TECS guides you through building a (virtual) computer, starting with simulated NAND gates and building up. (Admittedly, I just got that one and I'm only a few chapters in, but excellent so far.) I think every developer would benefit from knowing at least one low-level language (such as C), and one very high level language (such as Prolog).

happy4crazy 16y ago
I'm also a few chapters into The Elements of Computing Systems, and I also think it's excellent so far :) I'm posting my project solutions to my github account, fwiw: http://github.com/happy4crazy/elements_of_computing_systems http://github.com/happy4crazy/elements_of_computing_systems

eru 16y ago

> Financial arithmetic? Convert to smallest unit and use integers or the currency data type du jour in your language, and don't act surprised when operations on 32-bit floating point don't yield the intuitively correct values. Yes, though you may want more precision then that. For accuracy you could use, say, exact rational numbers, and for speed a fixed precision below e.g. Cents might be useful. (Though you probably meant something like nano-Cent when you said, convert to the smallest unit.)

redstripe 16y ago

Look at the answers people give in the search. It doesn't help that every time someone asks this they get a link to something like this: http://docs.sun.com/source/806-3568/ncg_goldberg.html http://docs.sun.com/source/806-3568/ncg_goldberg.html I'm surprised someone hasn't translated it into Klingon or Sanskrit yet. It's pure snobbery that makes people leave responses like this. This document falls into the further reading for enthusiasts category. Nobody should be linking to it as an introductory answer to the question. At least not if they want people to learn and stop asking the question eventually.

anigbrowl 16y ago

I have to question the value of CS educations Many people have no interest in being computer scientists. They want to learn enough software engineering to get something done using a computer. Outside of CS, programming is only a means to an end. Higher level languages ought not to expose the user/programmer to this sort of thing, even at the expense of performance; for every 'why is it that...' question, there are probably several more buggy programs being used in production environments. Also, it's a confusing distraction for kids trying to learn programming, who may not be able to wrap their heads around the theory of FP implementation, although they may be quite competent to handle the task they're trying to implement in software. I used to work with a Motorola 56k series DSP, so named because it offers up to 56 bits of integer precision. Dealing with data that's usually represented as gloating point involves a normalization conversion as you describe for the currency unit which soon becomes second nature. But for newcomers to the platform, that step at first seems like an inefficient limitation, adding complexity to an otherwise simple algorithm. Not everybody wants or needs to know how the underlying hardware works, often they just want to implement some fairly trivial math or logic without worrying about accuracy. I can't bring myself to correct that spelling error :)

JanezStupar 16y ago
To me its only an indicator. There's a ton of stuff on similar level that does matter and it adds up. Phenomena like knowing O(n) of basic algorithms and data structures. Understanding basic file formats and how they are implemented. Basic knowledge of network protocols and their building blocks. Basic knowledge of assembler. Understanding how IO happens and how your use case is bound (IO or CPU). Knowledge of character code pages, etc. Nothing of the above is necessary to program in high level language. But if you want to do it well they can't possibly harm you. Or you will get involved with a vendor product that was written by someone who didn't know the above and you will be forced to reverse engineer the goddamn thing just to get your job done. :)
- anigbrowl 16y ago
  I totally agree where full-time development is concerned. And it's the sort of thing macro languages and so on tend to build in. There are lot of people that might need to do a bit more programming - more elaborate than what a spreadsheet easily allows, but which doesn't justify becoming/hiring a pro. I was surprised to find last year that there are programming languages designed specifically for knitting, for example.

scrame 16y ago

Not so much that this is a question, but that this particular question gets 100+ points on "hacker news", when its not even a question or explanation, just a link to google result of adding numbers in java. If it was at least a homework question on stackoverflow there would at least be a good description, and possibly a reasonable explanation.