3 ms·

> Almost no problems can be represented exactly by rationals. Even the sqrt(2) you started with is not rational. Almost all numerical problems software develop

> Almost no problems can be represented exactly by rationals. Even the sqrt(2) you started with is not rational. Almost all numerical problems software developers deal with can be represented exactly using rationals. Only a small fraction of all problems involve transcendentals. And those can't be represented exactly by floats either so I don't know what point you're making? > For example, if you tried to compute a simple mandelbrot set with rationals, you'd need more bytes than there are subatomic particles in the universe (~10^80). Well, yes. You'd run into the same problem if you tried to compute all the decimals of pi too. Using rationals does not mean that you can't round results. > So you see rationals are computationally unusable for even simple tasks. I see that rationals cannot represent all real numbers. > These can be arbitrarily slow and use arbitrarily large memory for simple problems - which I stated above when I said "Such things are either incredibly inefficient or mathematically unsound." For example simple converging infinite sums that don't have closed forms known to your tools would never evaluate - the only solution is some fixed, finite representation if you want to use the value in later computations. I wrote that in mainstream languages should be able to evaluate (2^(1/2))^2 == 2. How do you go from there to requiring them to be able to evaluate infinite sums?! > Care to show me where deep learning libraries do perfect symbolic computation in the process of deep learning? I didn't claim that. > Most compilers would reduce sqrt(2) * sqrt(2) to 2? List them. (Heck - list one!) It's simply not true (as can be checked trivially for all common C++ compilers on godbolt). I didn't claim that. Compilers are stuck with IEEE 754 fp semantics and therefore won't simplify (2^(1/2))^2. But there is no technical reason why they couldn't. To paraphrase your reply: "Rational arithmetic and symbolic computation can't solve every problem. Therefore they are useless/no better than floating point arithmetic." From a theoretical perspective that is perhaps true but in practice both are very useful tools.

ChrisLomont 6y ago
>Almost all numerical problems software developers deal with can be represented exactly using rationals. Only a small fraction of all problems involve transcendentals. There's more classes of real numbers than rationals and transcendentals. Hint: your example of sqrt(2) is neither. As to doing things exactly with rationals, anytime you take a sin, cos, exp, log of any non-zero rational number the result is not rational. Any time you take a root of a non-perfect power rational you don't get a rational number (you get an algebraic number, which is neither rational nor transcendental). So all you can do is simple arithmetic: + * / - (actually, that's from the definition of the field Q of rational numbers). And you still run into the problem that rational number arithmetic gets arbitrarily slow and runs out of memory. Each multiply on average results in requiring the sum of the storage sizes of the multiplicands, making more than about 40 multiplications infeasible before you're out of RAM. So your view of "almost all numerical problems software developers deal with" must be limited to only those using simple arithmetic and no more than around 40 operations. That's amazingly constraining. What problems do you call numerical problems that fit your constraints? Things that cannot be done exactly with rational numbers includes pretty much all of gaming (3D or otherwise), deep learning (probably every model has an exp in it), scientific computing, audio and video processing (cos and sin all over the place), and on and on. >Using rationals does not mean that you can't round results. Wait, you just claimed "...can be represented exactly using rationals. Now you want to round, throwing out the exactness? Which is it? And congrats - you just re-invented floating-point math which is simply approximating values using rational numbers in a clever manner to handle larger ranges than fixed point. But they're still always rational approximations to values. >I wrote that in mainstream languages should be able to evaluate (2^(1/2))^2 == 2. How do you go from there to requiring them to be able to evaluate infinite sums?! Ah, so you mean languages should be able to simply solve this one specific problem, not all relatively simple symbolic problems? That seems like an even bigger mess than using floating point, which is well-defined and covers a large class of problems. Does your mythical language know (A^(1/B))^B==A for all rationals A and B also representable in your language? Or are you limited to the value A=B=2 only? (Hint: if you claim it should hold, you're gonna hit problems again :)
- bjourne 6y ago
  > As to doing things exactly with rationals, anytime you take a sin, cos, exp, log of any non-zero rational number the result is not rational. I fail to see your point. I never claimed that you can do real arithmetic with rationals. > So all you can do is simple arithmetic: + * / - (actually, that's from the definition of the field Q of rational numbers). Same as with floats. Except with floats you don't even get division. > And you still run into the problem that rational number arithmetic gets arbitrarily slow and runs out of memory. Each multiply on average results in requiring the sum of the storage sizes of the multiplicands, making more than about 40 multiplications infeasible before you're out of RAM. What? >>> from fractions import Fraction >>> Fraction(7, 8)**40 Fraction(6366805760909027985741435139224001, 1329227995784915872903807060280344576) In other words, no, you don't run out of memory after 40 multiplications... > Things that cannot be done exactly with rational numbers includes pretty much all of gaming (3D or otherwise), deep learning (probably every model has an exp in it), scientific computing, audio and video processing (cos and sin all over the place), and on and on. You sure can do all of those things with rational arithmetic. What you can't do is do them exactly but I never claimed you could. > > Using rationals does not mean that you can't round results. > Wait, you just claimed "...can be represented exactly using rationals. Now you want to round, throwing out the exactness? Which is it? I wrote "ALMOST ALL numerical problems software developers DEAL WITH can be represented exactly using rationals." The key word in that sentence is "ALMOST." > And congrats - you just re-invented floating-point math which is simply approximating values using rational numbers in a clever manner to handle larger ranges than fixed point. But they're still always rational approximations to values. Yes, but I have also solved the freakishly annoying issues detailed in the article. Which one do you prefer: >>> print(sum(0.1 for _ in range(10))) 0.9999999999999999 or >>> print(sum(Fraction(1, 10) for _ in range(10))) 1 ? I know which one users prefer. > >I wrote that in mainstream languages should be able to evaluate (2^(1/2))^2 == 2. How do you go from there to requiring them to be able to evaluate infinite sums?! > Ah, so you mean languages should be able to simply solve this one specific problem, not all relatively simple symbolic problems? That seems like an even bigger mess than using floating point, which is well-defined and covers a large class of problems. I don't think Mathematica or the Python symbolic math packages I've worked with are particularily messy. > Does your mythical language know (A^(1/B))^B==A for all rationals A and B also representable in your language? Or are you limited to the value A=B=2 only? Yes, assuming A and B are positive.

ChrisLomont 6y ago

>Almost all numerical problems software developers deal with can be represented exactly using rationals. Only a small fraction of all problems involve transcendentals. There's more classes of real numbers than rationals and transcendentals. Hint: your example of sqrt(2) is neither. As to doing things exactly with rationals, anytime you take a sin, cos, exp, log of any non-zero rational number the result is not rational. Any time you take a root of a non-perfect power rational you don't get a rational number (you get an algebraic number, which is neither rational nor transcendental). So all you can do is simple arithmetic: + * / - (actually, that's from the definition of the field Q of rational numbers). And you still run into the problem that rational number arithmetic gets arbitrarily slow and runs out of memory. Each multiply on average results in requiring the sum of the storage sizes of the multiplicands, making more than about 40 multiplications infeasible before you're out of RAM. So your view of "almost all numerical problems software developers deal with" must be limited to only those using simple arithmetic and no more than around 40 operations. That's amazingly constraining. What problems do you call numerical problems that fit your constraints? Things that cannot be done exactly with rational numbers includes pretty much all of gaming (3D or otherwise), deep learning (probably every model has an exp in it), scientific computing, audio and video processing (cos and sin all over the place), and on and on. >Using rationals does not mean that you can't round results. Wait, you just claimed "...can be represented exactly using rationals. Now you want to round, throwing out the exactness? Which is it? And congrats - you just re-invented floating-point math which is simply approximating values using rational numbers in a clever manner to handle larger ranges than fixed point. But they're still always rational approximations to values. >I wrote that in mainstream languages should be able to evaluate (2^(1/2))^2 == 2. How do you go from there to requiring them to be able to evaluate infinite sums?! Ah, so you mean languages should be able to simply solve this one specific problem, not all relatively simple symbolic problems? That seems like an even bigger mess than using floating point, which is well-defined and covers a large class of problems. Does your mythical language know (A^(1/B))^B==A for all rationals A and B also representable in your language? Or are you limited to the value A=B=2 only? (Hint: if you claim it should hold, you're gonna hit problems again :)

bjourne 6y ago
> As to doing things exactly with rationals, anytime you take a sin, cos, exp, log of any non-zero rational number the result is not rational. I fail to see your point. I never claimed that you can do real arithmetic with rationals. > So all you can do is simple arithmetic: + * / - (actually, that's from the definition of the field Q of rational numbers). Same as with floats. Except with floats you don't even get division. > And you still run into the problem that rational number arithmetic gets arbitrarily slow and runs out of memory. Each multiply on average results in requiring the sum of the storage sizes of the multiplicands, making more than about 40 multiplications infeasible before you're out of RAM. What? >>> from fractions import Fraction >>> Fraction(7, 8)**40 Fraction(6366805760909027985741435139224001, 1329227995784915872903807060280344576) In other words, no, you don't run out of memory after 40 multiplications... > Things that cannot be done exactly with rational numbers includes pretty much all of gaming (3D or otherwise), deep learning (probably every model has an exp in it), scientific computing, audio and video processing (cos and sin all over the place), and on and on. You sure can do all of those things with rational arithmetic. What you can't do is do them exactly but I never claimed you could. > > Using rationals does not mean that you can't round results. > Wait, you just claimed "...can be represented exactly using rationals. Now you want to round, throwing out the exactness? Which is it? I wrote "ALMOST ALL numerical problems software developers DEAL WITH can be represented exactly using rationals." The key word in that sentence is "ALMOST." > And congrats - you just re-invented floating-point math which is simply approximating values using rational numbers in a clever manner to handle larger ranges than fixed point. But they're still always rational approximations to values. Yes, but I have also solved the freakishly annoying issues detailed in the article. Which one do you prefer: >>> print(sum(0.1 for _ in range(10))) 0.9999999999999999 or >>> print(sum(Fraction(1, 10) for _ in range(10))) 1 ? I know which one users prefer. > >I wrote that in mainstream languages should be able to evaluate (2^(1/2))^2 == 2. How do you go from there to requiring them to be able to evaluate infinite sums?! > Ah, so you mean languages should be able to simply solve this one specific problem, not all relatively simple symbolic problems? That seems like an even bigger mess than using floating point, which is well-defined and covers a large class of problems. I don't think Mathematica or the Python symbolic math packages I've worked with are particularily messy. > Does your mythical language know (A^(1/B))^B==A for all rationals A and B also representable in your language? Or are you limited to the value A=B=2 only? Yes, assuming A and B are positive.