3 ms·
> There's a ton relating to precision - we had a huge ordeal converting our 32-bit space weather model to 64-bit because the change in precision changed our res
by MaxBarraclough 6y ago
> There's a ton relating to precision - we had a huge ordeal converting our 32-bit space weather model to 64-bit because the change in precision changed our results, so we couldn't publish (in good conscious) without making sure the results weren't within a good margin of error.
That sounds like a question of numerical stability of the algorithm, rather than anything to do with the programming language, no?
- codezero 6y agoYep. Fair point. The folks I worked with were very pragmatic so didn’t get into the internals and just treated the code as the instructions. Many of them came from an earlier era where the code was a lot more specific and the concept of an R8 is not quite as specific on x86_84 as it was on a VAX because the modern CPU does a lot of tricks under your nose.