3 ms·

Is that how scientific modelling (ie., modelling of non-computable functions) works though? It's not necessarily that squaring precision leads to a "better mod

Is that how scientific modelling (ie., modelling of non-computable functions) works though? It's not necessarily that squaring precision leads to a "better model", its that it fails to be a model at all without that precision. Eg., consider a climate model with effects at the first-decimal place, and 10th decimal-place of some measurable quantity vs. time. The effects at the 10th dp. could aggregate to overwhelm the effects at the 1 dp. It isnt that the model is better if it can "compute at the 10th dp" -- its just wildly inaccurate otherwise.

ben_w 5y ago
10 decimal points reminds me of a conversation I had with my father. Last century my dad was working on radar/IFF simulations, and he had an anecdote (if you can call it that) about when they switched from normal to double precision, and needing to get more digits of π (I think from 10 to 20, the way he talked implied some very strange things about the programming language(s) he was using). The third time he told this story, late-teenage-me pointed out that the curvature of spacetime under GR had more impact than the extra digits. While the broader point is valid, IIRC climate (being about average and broad trends) is fine with the imperfections, and handles this by running multiple simulations with slightly different parameters and seeing what usually happens. Weather forecast likewise.
- mjburgess 5y ago
  I'm not sure this is as true as it seems. When creating simulations we do so by defining a computable function (ie., over discrete binary strings) which approximates some non-computable function (ie., over the reals). It is important to note that those non-computable functions are derived, in part, analytically based on theorems which require infinite precision, ie., real numbers. When we define a computable approximation we can only do so by narrowing the simulation to a scale where "the numbers have only a few significant digits". This is really the only thing we can simulate, and even then, we are using mathematical analysis to derive much of it -- which cannot be simulated. If you really wanted to simulate across scales, ie., run a simulation "from atoms to tables", i think we'd be requiring many many many decimal places. It would likely, i think, be physically impossible to do. (Or if not, then atoms to planets would be) -- as the discrete precision required would require a theoretical minimum of mass that would create a black hole. eg., Consider that a table has 10^verylargenumber of "spherical" atoms, and so its aggregate shape would be extremely sensitive to a miscalculation of each atoms shape ... and we would require a precision in PI probably something like 10^-verylargenumber.
  - wcoenen 5y ago
    > Consider that a table has 10^verylargenumber of "spherical" atoms That "verylargenumber" exponent is not actually a large number. It's probably close to 28. E.g. for a rough estimate, pretend that the table has a mass of 120kg and is made of only carbon atoms. 120000 gram / (12 gram/mol) = 10000 mol = 6x10^27 < 10^28. On a related note, the estimated number of atoms in the visible universe is around 10^80.
    - mjburgess 5y ago
      So the volume of one atom can be calculated using, c. 30 dp. of pi -- +3 so that errors do not compound. So we need 30 bits/atom to store the volume, for 10^30 atoms -- this is 10^15 petabytes. Suppose we're trying to estimate the shape of the table using the relevant state vector for each atom (ie., radius, coordinate, EM-shell, etc.). Ie., how these atoms interact to produce a structure. Then we need perhaps 10 numbers/atom, so 10^16 petabytes. And consider the computation. The issue is that macroscopic structuring effects, ie., that the table has its tabular shape also exert a top-down effect constraining the structure of the atoms. We'd have to solve that problem through iteration. Ie., we compute a shape from local effects only, then iteratively recompute until increasing orders of scale are included. Suppose we perform 1 operation/atom to compute an interaction, that's 10^30 operations. At 1ns/op, we have 10^21 seconds -- optimistically. That's one scale, now we need to iterate; given the orders of effects are 1nm...1m that's c. 10 scales, say. We're now at 10^22 seconds with 10^16 petabytes of data. And also, since we're running a 10-fold iteration we need more than 10^30 precision, as errors will compound. This is for a table!. As I said, choose your scale, eg., atom to star, and the storage requirments have an essentially physically impossible mass.
- TeMPOraL 5y ago
  > The third time he told this story, late-teenage-me pointed out that the curvature of spacetime under GR had more impact than the extra digits. That may be true, but in simulations using floating points (which are, as 'mjburgess points out, discrete), there are discrete approximation errors that accumulate with almost any operation, on top of any measurement or modelling approximation errors you may have. On some problems, numerical integration on floats bleeds precision really fast. I recall seeing some problems on numerics labs during university years, where using doubles instead of floats was what made the algorithm converge on a solution instead of endlessly spinning or diverging into infinity. I imagine that ideally, you'd use a more sophisticated algorithm instead[0] - perhaps one that simulates real numbers with greater precision. But computer power isn't infinite (and it certainly wasn't infinite in the last century) - floats and doubles are fast, supported by hardware, and you may need that speed to get results in a reasonable timeframe. Using doubles instead of floats may give you just enough precision buffer to make the faster algorithm work. -- [0] - A trivial example that I'm sure is known to anyone doing physical simulations (including for videogames) is using increasingly sophisticated integrators for kinematics. You may start with naive Euler calculating v += aΔt, s += vΔt, but this quickly becomes inaccurate in quite spectacular ways (try simulating springs with naive Euler). So you may then progress through backwards / symplectic Euler, and end up with Verlet integration or even reaching for 4th Order Runge-Kutta, to keep the energy in the system more-less conserved as it evolves. And yes, the difference between floats and doubles is also very apparent, especially with Euler methods.
  - gugagore 5y ago
    It sort of sounds like you're confusing two different kinds of errors. Euler vs RK4 is an issue of truncation error. It's called that because when you approximate a derivative with a finite difference, you can see that you've truncated the infinite Taylor series. Truncation error is present even if there were zero approximation in the elementary floating point operations. I'm not sure how you mean that the difference between floats and doubles shows up with Euler methods especially. Under no floating point error, you can drive truncation error toward zero by driving the step size toward zero. But the floating point precision sets a floor on how low you can go. Perhaps this is what you meant.
- actually_a_dog 5y ago
  That doesn't surprise me a lot. Apparently, you'd only need 40 digits of pi to compute the area of a circle the size of the visible universe[0]. Pi tends to show up in equations where it's being multiplied by relatively small constants like 1/2, 2, or 8. Maybe sometimes you might see a pi^2 times a smallish constant, too. So, not needing a ton of digits in order to be good enough for numerical models doesn't seem crazy at all. --- [0]: https://www.jpl.nasa.gov/edu/news/2016/3/16/how-many-decimals-of-pi-do-we-really-need/ https://www.jpl.nasa.gov/edu/news/2016/3/16/how-many-decimal...
  - morpheos137 5y ago
    >only need 40 digits...to calculate area. This comment makes no sense to without the number of significant digits.

ben_w 5y ago

10 decimal points reminds me of a conversation I had with my father. Last century my dad was working on radar/IFF simulations, and he had an anecdote (if you can call it that) about when they switched from normal to double precision, and needing to get more digits of π (I think from 10 to 20, the way he talked implied some very strange things about the programming language(s) he was using). The third time he told this story, late-teenage-me pointed out that the curvature of spacetime under GR had more impact than the extra digits. While the broader point is valid, IIRC climate (being about average and broad trends) is fine with the imperfections, and handles this by running multiple simulations with slightly different parameters and seeing what usually happens. Weather forecast likewise.

mjburgess 5y ago
I'm not sure this is as true as it seems. When creating simulations we do so by defining a computable function (ie., over discrete binary strings) which approximates some non-computable function (ie., over the reals). It is important to note that those non-computable functions are derived, in part, analytically based on theorems which require infinite precision, ie., real numbers. When we define a computable approximation we can only do so by narrowing the simulation to a scale where "the numbers have only a few significant digits". This is really the only thing we can simulate, and even then, we are using mathematical analysis to derive much of it -- which cannot be simulated. If you really wanted to simulate across scales, ie., run a simulation "from atoms to tables", i think we'd be requiring many many many decimal places. It would likely, i think, be physically impossible to do. (Or if not, then atoms to planets would be) -- as the discrete precision required would require a theoretical minimum of mass that would create a black hole. eg., Consider that a table has 10^verylargenumber of "spherical" atoms, and so its aggregate shape would be extremely sensitive to a miscalculation of each atoms shape ... and we would require a precision in PI probably something like 10^-verylargenumber.
- wcoenen 5y ago
  > Consider that a table has 10^verylargenumber of "spherical" atoms That "verylargenumber" exponent is not actually a large number. It's probably close to 28. E.g. for a rough estimate, pretend that the table has a mass of 120kg and is made of only carbon atoms. 120000 gram / (12 gram/mol) = 10000 mol = 6x10^27 < 10^28. On a related note, the estimated number of atoms in the visible universe is around 10^80.
  - mjburgess 5y ago
    So the volume of one atom can be calculated using, c. 30 dp. of pi -- +3 so that errors do not compound. So we need 30 bits/atom to store the volume, for 10^30 atoms -- this is 10^15 petabytes. Suppose we're trying to estimate the shape of the table using the relevant state vector for each atom (ie., radius, coordinate, EM-shell, etc.). Ie., how these atoms interact to produce a structure. Then we need perhaps 10 numbers/atom, so 10^16 petabytes. And consider the computation. The issue is that macroscopic structuring effects, ie., that the table has its tabular shape also exert a top-down effect constraining the structure of the atoms. We'd have to solve that problem through iteration. Ie., we compute a shape from local effects only, then iteratively recompute until increasing orders of scale are included. Suppose we perform 1 operation/atom to compute an interaction, that's 10^30 operations. At 1ns/op, we have 10^21 seconds -- optimistically. That's one scale, now we need to iterate; given the orders of effects are 1nm...1m that's c. 10 scales, say. We're now at 10^22 seconds with 10^16 petabytes of data. And also, since we're running a 10-fold iteration we need more than 10^30 precision, as errors will compound. This is for a table!. As I said, choose your scale, eg., atom to star, and the storage requirments have an essentially physically impossible mass.
TeMPOraL 5y ago
> The third time he told this story, late-teenage-me pointed out that the curvature of spacetime under GR had more impact than the extra digits. That may be true, but in simulations using floating points (which are, as 'mjburgess points out, discrete), there are discrete approximation errors that accumulate with almost any operation, on top of any measurement or modelling approximation errors you may have. On some problems, numerical integration on floats bleeds precision really fast. I recall seeing some problems on numerics labs during university years, where using doubles instead of floats was what made the algorithm converge on a solution instead of endlessly spinning or diverging into infinity. I imagine that ideally, you'd use a more sophisticated algorithm instead[0] - perhaps one that simulates real numbers with greater precision. But computer power isn't infinite (and it certainly wasn't infinite in the last century) - floats and doubles are fast, supported by hardware, and you may need that speed to get results in a reasonable timeframe. Using doubles instead of floats may give you just enough precision buffer to make the faster algorithm work. -- [0] - A trivial example that I'm sure is known to anyone doing physical simulations (including for videogames) is using increasingly sophisticated integrators for kinematics. You may start with naive Euler calculating v += aΔt, s += vΔt, but this quickly becomes inaccurate in quite spectacular ways (try simulating springs with naive Euler). So you may then progress through backwards / symplectic Euler, and end up with Verlet integration or even reaching for 4th Order Runge-Kutta, to keep the energy in the system more-less conserved as it evolves. And yes, the difference between floats and doubles is also very apparent, especially with Euler methods.
- gugagore 5y ago
  It sort of sounds like you're confusing two different kinds of errors. Euler vs RK4 is an issue of truncation error. It's called that because when you approximate a derivative with a finite difference, you can see that you've truncated the infinite Taylor series. Truncation error is present even if there were zero approximation in the elementary floating point operations. I'm not sure how you mean that the difference between floats and doubles shows up with Euler methods especially. Under no floating point error, you can drive truncation error toward zero by driving the step size toward zero. But the floating point precision sets a floor on how low you can go. Perhaps this is what you meant.
actually_a_dog 5y ago
That doesn't surprise me a lot. Apparently, you'd only need 40 digits of pi to compute the area of a circle the size of the visible universe[0]. Pi tends to show up in equations where it's being multiplied by relatively small constants like 1/2, 2, or 8. Maybe sometimes you might see a pi^2 times a smallish constant, too. So, not needing a ton of digits in order to be good enough for numerical models doesn't seem crazy at all. --- [0]: https://www.jpl.nasa.gov/edu/news/2016/3/16/how-many-decimals-of-pi-do-we-really-need/ https://www.jpl.nasa.gov/edu/news/2016/3/16/how-many-decimal...
- morpheos137 5y ago
  >only need 40 digits...to calculate area. This comment makes no sense to without the number of significant digits.