3 ms·
For an RPN calculator I discovered similar problems with a number hierarchy as well. The calculator has four number types: - integers (for hexadecimal operatio
by _nalply 2y ago
For an RPN calculator I discovered similar problems with a number hierarchy as well. The calculator has four number types:
- integers (for hexadecimal operations),
- quotients (like some HP calculators),
- IEEE 756 floating point numbers (because that is the most simple thing on computers) and
- complex numbers based on two IEEE 756 components
Mathematically these types are subsets of each other:
ℤ ⊂ ℚ ⊂ ℝ ⊂ ℂ
But number values in a computer aren't subsets. Not all 64 bit integers are representable as 64 bit IEEE 756 numbers, for example odd integers beyond 10^53.
Hand-held calculators need to hide a lot of complexity to be intuitive.
One example. If you have two values in the calculator that approximate e and i * pi as good as the calculator can and request the power operation, then you have the Euler identity and should get the integer 1. The implementation uses complex numbers and because of rounding errors you might get something like 1.0000000003. To be user-friendly, the calculator needs to hide that.
That's what I did:
- Use four distinct types without any object-oriented features
- Operations aren't methods but functions with a given signature, for example as a Rust function:
fn sin(c: Complex) -> Complex
- Coerce when an operation expects a different type
- Don't use the whole precision of floating point (truncate to a lower precision and hide one decimal precision digit after truncation)
- The default display omits both the imaginary part and the fraction if zero