4 ms·
Can this be true? Does the calculator have any irrational number? What if I enter PI * 2? Infinite precision, wouldn't that mean running until your RAM is full?
by zelphirkalt 2y ago
Can this be true? Does the calculator have any irrational number? What if I enter PI * 2? Infinite precision, wouldn't that mean running until your RAM is full? Or would it simply work, because no irrational number is calculated to infinite (as many as the device could) digits and is therefore finite and can easily be multiplied?
Or another even simpler case: If division is infinite precision, and I enter (2/3) * X, does the calculator internally work with fractions? Otherwise it would have infinite digits to compute.
- Khoth 2y agoFor 2/3, it does indeed work with fractions internally. For pi, it uses some floating point representation I think, you get about 50 digits.
- canucker2016 2y agoFor PI, calc.exe, in scientific mode, returns 3.1415926535897932384626433832795 which is 32 digits of precision (33 chars if you include the decimal pt).