15 ms·

Hacker's Guide to Numerical Analysis

356 points by bollu 6y ago

SavantIdiot 6y ago
"It is not transparent to me why this definition is sensible." Because if I have a six digit meter and I measure 5.0000 volts I know that it is 5.0000 volts. I don't see why that is confusing at all. "Here, yyy has correct one and three significant digits relative to xxx, but incorrect 2 significant digits, since the truncation at x2x_2x2 and y2y_2y2 do not agree even to the first significant digit. " This doesn't make sense. Y has the correct one? I think you might be confused about the purpose of significant digits. SD are intended to be paired with a tolerance. If there is no tolerence quote with the number, it is meaningless to talk about significant digits. You arbitrarily defined the tolerance as +/0.5 (half a unit). That is not always true, almost never true. Consider test equipment, not every multimeter is going to be +/0.5mV, some might be +/-.3uV. It varies.
- leephillips 6y ago
  Yes, I think all we've learned from this article is that the author never learned about significant digits.
- PhaseLockk 6y ago
  Exactly this. It may be useful for OP to consider what the hypothetical opposite of significant digits would be: insignificant digits, ie. digits that tell you nothing. A leading zero can be omitted from a number without any loss of information. A trailing zero, when correctly used in a position that is not below the tolerance, gives you significant information. On the other hand, if you start combine numbers with different tolerances, but do not truncate the result, you will have a bunch of trailing digits that are meaningless in practice because they are below the combined tolerance.
- wsh 6y ago
  I agree that it’s better to state the uncertainty [1] together with the measurement, but when this isn’t done, there is a standard convention for significant digits, explained in ISO 80000-1:2009 [2], subclause 7.3.4: When a number is given without any further information, it is generally interpreted so that the last digit is rounded with a rounding range equal to 1 in the last digit (see Annex B). Thus, for example, the number 401 008 is generally assumed to represent a value between 401 007,5 and 401 008,5. In this case, the maximum magnitude of the error in the number 401 008 is 0,5. This is consistent with the author’s “less than half a unit.” [1] JCGM 100:2008, Evaluation of measurement data — Guide to the expression of uncertainty in measurement, https://www.iso.org/sites/JCGM/GUM-JCGM100.htm https://www.iso.org/sites/JCGM/GUM-JCGM100.htm [2] ISO 80000-1:2009, Quantities and units — Part 1: General
  - deleted 6y ago
    [deleted]
- ticklemyelmo 6y ago
  I'm not entirely clear on the author's thinking. But consider the case of watching a real world voltmeter read 1.02v, 1.01v, 1.00v, 0.99v. Did we really lose a digit of precision there? The leading digit transition between 1 and 9 is a boundary condition that messes up that rule.
  - taeric 6y ago
    I think the catch is that you aren't keeping the number normalized. Significant digits somewhat assume scientific notation. So that last number is 9.9e-1. Two digits.
  - ska 6y ago
    You didn't lose a digit, you wrote it that way. If your instrument has 3 digits, you should have written 0.990v
    - alisonkisk 6y ago
      You can't make your meter add an extra digit (unless you can turn the dial to a different scale). The digit is lost. But of course it isn't a big deal to lose that dgiti because a trailing digit is 0.1 as much error fraction when the leading digit is 9 vs 1.
      - ska 6y ago
        I'm not sure what you mean here, probably because I worded it poorly. Of course your instrument will have different scales, and you should use the appropriate one. But the measurement was done with a number of significant figures, this is independent of how you write it. Nothing is "lost", if measurement had 2 sig figs it is 0.99, if it had 3 it was 0.990. Or 0.991 or whatever. Am I missing something?
        taeric 6y ago
        You skipped the example. They said if your device measures 1.00, it is three significant digits. If it measures 0.01, it is one significant digit. From the same measuring device. So it is counter intuitive that the measurement just going down below 1 somehow causes it to "lose" significant digits. It didn't lost precision, though. Just the number of digits that are significant.
- yudlejoza 6y ago
  When it comes to measurement, 1.7320 is not the same as 1.732 The trailing zero was part of the measurement and, hence, is included in the significant digit count. That also means you can't just measure 1.7320, and report 1.73200 instead (or 1.732). You have to report only, and exactly, what you measured. (I'm ignoring rounding in what I said).
- taeric 6y ago
  I think you missed what they found weird. The subject for that was that 0.0491 only has three significant digits. It is not obvious why, if you don't normalize the number.
  - Trombone12 6y ago
    No, what is weird is that the original author seems to think you can look at a number and know how many significant digits it has. This is almost completely wrong: the number of significant digits is something given by metadata, you cannot look at a number and know that it has been properly rounded to the correct number of significant digits. You always need to be told this special circumstance because most numbers you encounter have in fact not been properly rounded.
    - jhallenworld 6y ago
      In the world of general chemistry, the significant figures rule is assumed. It's not a great rule because in this case: .000300 you have 3 significant figures. But in this case: 300000 you could have 1 to 6 significant figures. The convention to deal with this is always to use scientific notation: 3.00x10^5 has three significant figures.
    - alisonkisk 6y ago
      It's assumed that you shouldn't write down insignificant digits.
      - Trombone12 6y ago
        Not sure how you're supposed to do floating point math while following that rule... you'll have better luck with writing C code with all semicolons in place.
    - taeric 6y ago
      You lost me on this one. :( There is the odd case of trailing zeroes in non decimal numbers, but otherwise if a number is written, when is that not assumed to be significant? That is, there are fairly standard rules for conveying significant digits, I thought?
- sengstrom 6y ago
  I think it was more a matter of attitude. Never a great way to actually explain something.
- mkl 6y ago
  > "Here, y has correct one and three significant digits relative to x, but incorrect 2 significant digits, since the truncation at x_2 and y_2 do not agree even to the first significant digit. " > This doesn't make sense. Y has the correct one? I think that's a language issue, like "that x̂ agress to x upto p" in the next section. I think what they're trying to say is the following. If you round x = 0.9949 to 1, 2, or 3 significant digits, you get 1, 0.99, or 0.995 respectively. If you round y = 0.9951 to 1, 2, or 3 significant digits, you get 1, 1.0, or 0.995 respectively. So x and y look equal if you round to 1 or 3 significant digits, but not if you round to 2 significant digits.
jphoward 6y ago
This is such a great example of how people that know math(s) really struggle to explain things to people who don't, because they struggle to realise how much lingo they are using. The first point explains something as basic as absolute and relative errors. Great. 1 page down (s)he's casually thrown in function notation with R representing the real numbers with no explanation. Anyone who isn't familiar with this notation is immediately intimidated. But I'd wager almost anyone who is familiar with it didn't need to read the page before it. But also, I think it's just unnecessary to be this specific in the first place, if you're trying to quickly explain concepts. It's actually a huge problem in maths and even Wikipedia (I feel) is pretty guilty of this. If you take a (relatively) key mathematical concept, such as the Taylor series, then the opening gambit is completely incomprehensible to anyone who isn't already knowledgeable in maths: https://en.wikipedia.org/wiki/Taylor_series https://en.wikipedia.org/wiki/Taylor_series This, to me, seems in huge contrast with other scientific fields. Medicine wikipedia, for example, caters towards the "average reader" first, and doctors later. This is entirely appropriate for an encyclopedia. For example: https://en.wikipedia.org/wiki/AV_nodal_reentrant_tachycardia#:~:text=AV%2Dnodal%20reentrant%20tachycardia%20(AVNRT,most%20common%20regular%20supraventricular%20tachycardia https://en.wikipedia.org/wiki/AV_nodal_reentrant_tachycardia...
- howenterprisey 6y ago
  I run into that same problem a lot while trying to find tutorials that target beginners. Absolute beginners, the kind showing up to my organization's Google Summer of Code chat rooms. It's rare to find tutorials that just keep it simple enough. I wanted a regex one recently. First off, they all presumed mastery of the concept of a string, but more seriously, none of them had a really good definition of a regex! I still can't think of one myself, but I know it's probably not "pattern to match specific strings", because it uses an uncommon meaning of "pattern" and a jargon meaning of "match". And of course they all immediately used terms like "formal languages". The perldoc page 'perlretut' did pretty well with "a template that is used to determine if a string has certain characteristics" - ok, a bit abstract for my taste, but let's keep going - then a bit further down they use "modifier" (flag) with no definition. But I really had to look hard for flaws in that one. I guess the only way to actually nail this is to supervise successive groups of students as they try to learn using your tutorial-in-progress...
apbytes 6y ago
In the derivation for conditioning what is `O(dx^2)` and why is it there? Looks something like error margin?
- sampo 6y ago
  https://en.wikipedia.org/wiki/Big_O_notation#Infinitesimal_asymptotics https://en.wikipedia.org/wiki/Big_O_notation#Infinitesimal_a...
  - apbytes 6y ago
    Thanks!
ubavic 6y ago
Digression: for all authors that want minimalistic website, please don’t push it too hard, as author of this website did. Color scheme is nice, as is font selection, but it lacks navigation links. Putting simple link to website home would do so much for explorers of site.
- leephillips 6y ago
  Since I dumped on the author in another comment, let me defend him here: going up one level in the URL is really not a problem. It’s even easier if you use something like Vimperator, where you just need to type “gu”. And there is a nice table of contents.
rarefied_tomato 6y ago
For a deeper look, Trefethen's Numerical Linear Algebra starts with these ideas and applies them to matrix algorithms.
- bollu 6y ago
  Thank you for the reference!
singhrac 6y ago
Since it was linked, Herbie will help you find numerically stable versions of your expression: https://herbie.uwplse.org/ https://herbie.uwplse.org/
- bloaf 6y ago
  I saw something like that years ago (I think someone had done it in matlab?) and was recently struggling to find it again. Thanks so much for posting!
cambalache 6y ago
My rule of thumb is that anything title "...for Hackers" is crap.This link was not an exception. > Since the relative error is invariant under scaling (x↦αx)(x \mapsto \alpha x)(x↦αx), we will mostly be interested in relative error. No, No, No, No. This is very much dependent on what you are measuring/modelling, what is the tolerance you are willing to accept, the consequences and so on. A 1% of error in the change the cashier gave you back maybe is not a big deal. That same error in the national budget is equivalent to 2X the NASA's budget.
- enchiridion 6y ago
  I know this is besides the point, but that is a damn shame.
- domnomnom 6y ago
  > My rule of thumb is that anything title "...for Hackers" is crap.This link was not an exception. Well then you're missing the whole point completely.
- vulcan01 6y ago
  I get your point in the rest of your comment, but: > My rule of thumb is that anything title "...for Hackers" is crap Remind me what site you're on again?
  - nexuist 6y ago
    HackerNews, I guess, not NewsForHackers (:P) Although I do think that kind of rule of thumb is dumb. I've read plenty of materials with titles like that that have been informative and well written.
chrisjharris 6y ago
Are there any plans for a physical book? My attention span seems to be calibrated to focus for a prolonged period on paper but less so for information on screen.
deleted 6y ago
[deleted]
omginternets 6y ago
I really wanted to be interested in this, but I wish there were more context. When does this kind of stuff come up?
- macksd 6y ago
  Anytime you're doing a lot of floating-point math, the imprecision can bite you in non-intuitive ways. But you can mitigate that by fixing the order of operations to minimize error. I suppose one other specific example of a similar concept would be that when training deep learning models models you're doing a ton of matrix multiplication and tweaking the weights constantly. Especially when using lower-precision numbers, you can hit issues where your coefficient gets too close to 0, and the next thing you know it IS 0, and everything it gets multiplied by also comes out to 0, which messes everything up. So you have to avoid getting too close to 0: https://pytorch.org/docs/stable/amp.html#gradient-scaling https://pytorch.org/docs/stable/amp.html#gradient-scaling.
  - omginternets 6y ago
    Gotcha, thanks. This is actually exactly the piece of information that was missing!
    - fuzzfactor 6y ago
      Careful numerical analysis can also be very worthwhile with integer math as well.
      - omginternets 6y ago
        When does that come up?
        fuzzfactor 6y ago
        Can happen more seriously than with floating point since the dynamic range of integer math is relatively much more limited for the same bit depth. This can be especially bad with division followed by inherent truncation. The inability to represent very big values as well as very small values might need to be worked around. I needed to overcome this back in 1982 (before 16-bit portable PC's, and PC's were not yet popular at all) to do equivalent calculations on an 8-bit TRS-80 in 0.5K of BASIC program memory, to what was officially developed by 1980 in many pages of 32-bit FORTRAN. The algorithms had been well established for decades but always implemented using the equivalent of trig tables until the mainframe was capable of substituting (or even automatically printing the official tables!) after 1980. One volume of the published tables was bigger and weighed more than the TRS-80 portable. Ended up using something like the 21st century Gustafson approach for my little problem back then, now he is a mathematician pursuing a brilliant more universal solution. By 1982 it didn't bother me that my integer algorithms were dramatically different than the official floating-point FORTRAN. I had gotten accustomed to morphing fundamental equations by computerizing much less onerous tables where the simple near-linear equation was useless because the published tables contained official rounding errors from the original slide-rule calculations. To match the official calculations that time, the convergence of manual and automatic rounding errors took major precedence over the underlying equation which actually was defined by physical properties.
smlckz 6y ago
I am unable to forgive the author for the following: #include <cmath> #include <stdio.h> The code looks almost like pure C, but that little `c`...
dennis714 6y ago
Shameless plug about IEEE 754 round-off errors: https://yurichev.com/news/20210131_IEEE_754/ https://yurichev.com/news/20210131_IEEE_754/

356 points by bollu 6y ago

"It is not transparent to me why this definition is sensible." Because if I have a six digit meter and I measure 5.0000 volts I know that it is 5.0000 volts. I don't see why that is confusing at all. "Here, yyy has correct one and three significant digits relative to xxx, but incorrect 2 significant digits, since the truncation at x2x_2x2 and y2y_2y2 do not agree even to the first significant digit. " This doesn't make sense. Y has the correct one? I think you might be confused about the purpose of significant digits. SD are intended to be paired with a tolerance. If there is no tolerence quote with the number, it is meaningless to talk about significant digits. You arbitrarily defined the tolerance as +/0.5 (half a unit). That is not always true, almost never true. Consider test equipment, not every multimeter is going to be +/0.5mV, some might be +/-.3uV. It varies.

leephillips 6y ago
Yes, I think all we've learned from this article is that the author never learned about significant digits.
PhaseLockk 6y ago
Exactly this. It may be useful for OP to consider what the hypothetical opposite of significant digits would be: insignificant digits, ie. digits that tell you nothing. A leading zero can be omitted from a number without any loss of information. A trailing zero, when correctly used in a position that is not below the tolerance, gives you significant information. On the other hand, if you start combine numbers with different tolerances, but do not truncate the result, you will have a bunch of trailing digits that are meaningless in practice because they are below the combined tolerance.
wsh 6y ago
I agree that it’s better to state the uncertainty [1] together with the measurement, but when this isn’t done, there is a standard convention for significant digits, explained in ISO 80000-1:2009 [2], subclause 7.3.4: When a number is given without any further information, it is generally interpreted so that the last digit is rounded with a rounding range equal to 1 in the last digit (see Annex B). Thus, for example, the number 401 008 is generally assumed to represent a value between 401 007,5 and 401 008,5. In this case, the maximum magnitude of the error in the number 401 008 is 0,5. This is consistent with the author’s “less than half a unit.” [1] JCGM 100:2008, Evaluation of measurement data — Guide to the expression of uncertainty in measurement, https://www.iso.org/sites/JCGM/GUM-JCGM100.htm https://www.iso.org/sites/JCGM/GUM-JCGM100.htm [2] ISO 80000-1:2009, Quantities and units — Part 1: General
- deleted 6y ago
  [deleted]
ticklemyelmo 6y ago
I'm not entirely clear on the author's thinking. But consider the case of watching a real world voltmeter read 1.02v, 1.01v, 1.00v, 0.99v. Did we really lose a digit of precision there? The leading digit transition between 1 and 9 is a boundary condition that messes up that rule.
- taeric 6y ago
  I think the catch is that you aren't keeping the number normalized. Significant digits somewhat assume scientific notation. So that last number is 9.9e-1. Two digits.
- ska 6y ago
  You didn't lose a digit, you wrote it that way. If your instrument has 3 digits, you should have written 0.990v
  - alisonkisk 6y ago
    You can't make your meter add an extra digit (unless you can turn the dial to a different scale). The digit is lost. But of course it isn't a big deal to lose that dgiti because a trailing digit is 0.1 as much error fraction when the leading digit is 9 vs 1.
    - ska 6y ago
      I'm not sure what you mean here, probably because I worded it poorly. Of course your instrument will have different scales, and you should use the appropriate one. But the measurement was done with a number of significant figures, this is independent of how you write it. Nothing is "lost", if measurement had 2 sig figs it is 0.99, if it had 3 it was 0.990. Or 0.991 or whatever. Am I missing something?
      - taeric 6y ago
        You skipped the example. They said if your device measures 1.00, it is three significant digits. If it measures 0.01, it is one significant digit. From the same measuring device. So it is counter intuitive that the measurement just going down below 1 somehow causes it to "lose" significant digits. It didn't lost precision, though. Just the number of digits that are significant.
yudlejoza 6y ago
When it comes to measurement, 1.7320 is not the same as 1.732 The trailing zero was part of the measurement and, hence, is included in the significant digit count. That also means you can't just measure 1.7320, and report 1.73200 instead (or 1.732). You have to report only, and exactly, what you measured. (I'm ignoring rounding in what I said).
taeric 6y ago
I think you missed what they found weird. The subject for that was that 0.0491 only has three significant digits. It is not obvious why, if you don't normalize the number.
- Trombone12 6y ago
  No, what is weird is that the original author seems to think you can look at a number and know how many significant digits it has. This is almost completely wrong: the number of significant digits is something given by metadata, you cannot look at a number and know that it has been properly rounded to the correct number of significant digits. You always need to be told this special circumstance because most numbers you encounter have in fact not been properly rounded.
  - jhallenworld 6y ago
    In the world of general chemistry, the significant figures rule is assumed. It's not a great rule because in this case: .000300 you have 3 significant figures. But in this case: 300000 you could have 1 to 6 significant figures. The convention to deal with this is always to use scientific notation: 3.00x10^5 has three significant figures.
  - alisonkisk 6y ago
    It's assumed that you shouldn't write down insignificant digits.
    - Trombone12 6y ago
      Not sure how you're supposed to do floating point math while following that rule... you'll have better luck with writing C code with all semicolons in place.
  - taeric 6y ago
    You lost me on this one. :( There is the odd case of trailing zeroes in non decimal numbers, but otherwise if a number is written, when is that not assumed to be significant? That is, there are fairly standard rules for conveying significant digits, I thought?
sengstrom 6y ago
I think it was more a matter of attitude. Never a great way to actually explain something.
mkl 6y ago
> "Here, y has correct one and three significant digits relative to x, but incorrect 2 significant digits, since the truncation at x_2 and y_2 do not agree even to the first significant digit. " > This doesn't make sense. Y has the correct one? I think that's a language issue, like "that x̂ agress to x upto p" in the next section. I think what they're trying to say is the following. If you round x = 0.9949 to 1, 2, or 3 significant digits, you get 1, 0.99, or 0.995 respectively. If you round y = 0.9951 to 1, 2, or 3 significant digits, you get 1, 1.0, or 0.995 respectively. So x and y look equal if you round to 1 or 3 significant digits, but not if you round to 2 significant digits.

jphoward 6y ago

This is such a great example of how people that know math(s) really struggle to explain things to people who don't, because they struggle to realise how much lingo they are using. The first point explains something as basic as absolute and relative errors. Great. 1 page down (s)he's casually thrown in function notation with R representing the real numbers with no explanation. Anyone who isn't familiar with this notation is immediately intimidated. But I'd wager almost anyone who is familiar with it didn't need to read the page before it. But also, I think it's just unnecessary to be this specific in the first place, if you're trying to quickly explain concepts. It's actually a huge problem in maths and even Wikipedia (I feel) is pretty guilty of this. If you take a (relatively) key mathematical concept, such as the Taylor series, then the opening gambit is completely incomprehensible to anyone who isn't already knowledgeable in maths: https://en.wikipedia.org/wiki/Taylor_series https://en.wikipedia.org/wiki/Taylor_series This, to me, seems in huge contrast with other scientific fields. Medicine wikipedia, for example, caters towards the "average reader" first, and doctors later. This is entirely appropriate for an encyclopedia. For example: https://en.wikipedia.org/wiki/AV_nodal_reentrant_tachycardia#:~:text=AV%2Dnodal%20reentrant%20tachycardia%20(AVNRT,most%20common%20regular%20supraventricular%20tachycardia https://en.wikipedia.org/wiki/AV_nodal_reentrant_tachycardia...

howenterprisey 6y ago
I run into that same problem a lot while trying to find tutorials that target beginners. Absolute beginners, the kind showing up to my organization's Google Summer of Code chat rooms. It's rare to find tutorials that just keep it simple enough. I wanted a regex one recently. First off, they all presumed mastery of the concept of a string, but more seriously, none of them had a really good definition of a regex! I still can't think of one myself, but I know it's probably not "pattern to match specific strings", because it uses an uncommon meaning of "pattern" and a jargon meaning of "match". And of course they all immediately used terms like "formal languages". The perldoc page 'perlretut' did pretty well with "a template that is used to determine if a string has certain characteristics" - ok, a bit abstract for my taste, but let's keep going - then a bit further down they use "modifier" (flag) with no definition. But I really had to look hard for flaws in that one. I guess the only way to actually nail this is to supervise successive groups of students as they try to learn using your tutorial-in-progress...

apbytes 6y ago

In the derivation for conditioning what is `O(dx^2)` and why is it there? Looks something like error margin?

sampo 6y ago
https://en.wikipedia.org/wiki/Big_O_notation#Infinitesimal_asymptotics https://en.wikipedia.org/wiki/Big_O_notation#Infinitesimal_a...
- apbytes 6y ago
  Thanks!

ubavic 6y ago

Digression: for all authors that want minimalistic website, please don’t push it too hard, as author of this website did. Color scheme is nice, as is font selection, but it lacks navigation links. Putting simple link to website home would do so much for explorers of site.

leephillips 6y ago
Since I dumped on the author in another comment, let me defend him here: going up one level in the URL is really not a problem. It’s even easier if you use something like Vimperator, where you just need to type “gu”. And there is a nice table of contents.

rarefied_tomato 6y ago

For a deeper look, Trefethen's Numerical Linear Algebra starts with these ideas and applies them to matrix algorithms.

bollu 6y ago
Thank you for the reference!

singhrac 6y ago

Since it was linked, Herbie will help you find numerically stable versions of your expression: https://herbie.uwplse.org/ https://herbie.uwplse.org/

bloaf 6y ago
I saw something like that years ago (I think someone had done it in matlab?) and was recently struggling to find it again. Thanks so much for posting!

cambalache 6y ago

My rule of thumb is that anything title "...for Hackers" is crap.This link was not an exception. > Since the relative error is invariant under scaling (x↦αx)(x \mapsto \alpha x)(x↦αx), we will mostly be interested in relative error. No, No, No, No. This is very much dependent on what you are measuring/modelling, what is the tolerance you are willing to accept, the consequences and so on. A 1% of error in the change the cashier gave you back maybe is not a big deal. That same error in the national budget is equivalent to 2X the NASA's budget.

enchiridion 6y ago
I know this is besides the point, but that is a damn shame.
domnomnom 6y ago
> My rule of thumb is that anything title "...for Hackers" is crap.This link was not an exception. Well then you're missing the whole point completely.
vulcan01 6y ago
I get your point in the rest of your comment, but: > My rule of thumb is that anything title "...for Hackers" is crap Remind me what site you're on again?
- nexuist 6y ago
  HackerNews, I guess, not NewsForHackers (:P) Although I do think that kind of rule of thumb is dumb. I've read plenty of materials with titles like that that have been informative and well written.

chrisjharris 6y ago

Are there any plans for a physical book? My attention span seems to be calibrated to focus for a prolonged period on paper but less so for information on screen.

deleted 6y ago

[deleted]

omginternets 6y ago

I really wanted to be interested in this, but I wish there were more context. When does this kind of stuff come up?

macksd 6y ago
Anytime you're doing a lot of floating-point math, the imprecision can bite you in non-intuitive ways. But you can mitigate that by fixing the order of operations to minimize error. I suppose one other specific example of a similar concept would be that when training deep learning models models you're doing a ton of matrix multiplication and tweaking the weights constantly. Especially when using lower-precision numbers, you can hit issues where your coefficient gets too close to 0, and the next thing you know it IS 0, and everything it gets multiplied by also comes out to 0, which messes everything up. So you have to avoid getting too close to 0: https://pytorch.org/docs/stable/amp.html#gradient-scaling https://pytorch.org/docs/stable/amp.html#gradient-scaling.
- omginternets 6y ago
  Gotcha, thanks. This is actually exactly the piece of information that was missing!
  - fuzzfactor 6y ago
    Careful numerical analysis can also be very worthwhile with integer math as well.
    - omginternets 6y ago
      When does that come up?
      - fuzzfactor 6y ago
        Can happen more seriously than with floating point since the dynamic range of integer math is relatively much more limited for the same bit depth. This can be especially bad with division followed by inherent truncation. The inability to represent very big values as well as very small values might need to be worked around. I needed to overcome this back in 1982 (before 16-bit portable PC's, and PC's were not yet popular at all) to do equivalent calculations on an 8-bit TRS-80 in 0.5K of BASIC program memory, to what was officially developed by 1980 in many pages of 32-bit FORTRAN. The algorithms had been well established for decades but always implemented using the equivalent of trig tables until the mainframe was capable of substituting (or even automatically printing the official tables!) after 1980. One volume of the published tables was bigger and weighed more than the TRS-80 portable. Ended up using something like the 21st century Gustafson approach for my little problem back then, now he is a mathematician pursuing a brilliant more universal solution. By 1982 it didn't bother me that my integer algorithms were dramatically different than the official floating-point FORTRAN. I had gotten accustomed to morphing fundamental equations by computerizing much less onerous tables where the simple near-linear equation was useless because the published tables contained official rounding errors from the original slide-rule calculations. To match the official calculations that time, the convergence of manual and automatic rounding errors took major precedence over the underlying equation which actually was defined by physical properties.

smlckz 6y ago

I am unable to forgive the author for the following: #include <cmath> #include <stdio.h> The code looks almost like pure C, but that little `c`...

dennis714 6y ago

Shameless plug about IEEE 754 round-off errors: https://yurichev.com/news/20210131_IEEE_754/ https://yurichev.com/news/20210131_IEEE_754/