5 ms·

While I don't disagree with you that beyond 64 bit precision is often useful, be careful with overgeneralizing that point. Many applications are just fine with

While I don't disagree with you that beyond 64 bit precision is often useful, be careful with overgeneralizing that point. Many applications are just fine with 32 bits or even less and benefit greatly from the associated speedups, hence the proliferation of GPGPU computing (where single precision has a huge advantage). Neural networks are the popular examples, but plenty of physics/engineering simulations (molecular dynamics (particularly with symplectic methods), lattice Boltzmann hydrodynamics, etc.) are on that list as well.

graycat 9y ago
I agree, but I specifically restricted my remarks to numerical linear algebra. Double precision inner product accumulation is a standard idea in numerical linear algebra, e.g., as in George E. Forsythe and Cleve B. Moler, Computer Solution of Linear Algebraic Systems. People have long struggled with accuracy in numerical linear algebra. At one time at the US National Bureau of Standards, the struggles were so severe that M. Newman wrote software for numerically exact solution of systems of linear equations using only machine precision integer arithmetic. How? In the given system, he multiplied by powers of 10 to get all the decimal points to the right so that he had only whole numbers. Then for each prime number in a list of some dozens of primes, he solved the system in the finite field of the integers modulo that prime. Multiplicative inverse is from the Euclidean greatest common divisor algorithm. Then using the Chinese remainder theorem, he constructed the multiprecision integer numerators and denominators of the exact rational solution. It was nice work and somewhat popular. Careful work on condition number estimation and exploitation has also been used. Condition number can be the basis of error estimates. Sure, for, say, the fast Fourier transform, mostly in practice 32 bits can be fine.

graycat 9y ago

I agree, but I specifically restricted my remarks to numerical linear algebra. Double precision inner product accumulation is a standard idea in numerical linear algebra, e.g., as in George E. Forsythe and Cleve B. Moler, Computer Solution of Linear Algebraic Systems. People have long struggled with accuracy in numerical linear algebra. At one time at the US National Bureau of Standards, the struggles were so severe that M. Newman wrote software for numerically exact solution of systems of linear equations using only machine precision integer arithmetic. How? In the given system, he multiplied by powers of 10 to get all the decimal points to the right so that he had only whole numbers. Then for each prime number in a list of some dozens of primes, he solved the system in the finite field of the integers modulo that prime. Multiplicative inverse is from the Euclidean greatest common divisor algorithm. Then using the Chinese remainder theorem, he constructed the multiprecision integer numerators and denominators of the exact rational solution. It was nice work and somewhat popular. Careful work on condition number estimation and exploitation has also been used. Condition number can be the basis of error estimates. Sure, for, say, the fast Fourier transform, mostly in practice 32 bits can be fine.