The key though is probably to have a strong Number interface, where the overhead of it being an object is complied away, so you can easily switch out different

The key though is probably to have a strong Number interface, where the overhead of it being an object is complied away, so you can easily switch out different implementations, optimize to a more concrete time at AOT/JIT time and have clear semantics for conversion when different parts of the system want different concrete numeric types. You can then have any sort of default you want, such as an arbitrary precision library, or decimal or whatever, but easily change the declaration and get all the benefits, without needing to modify downstream code that respects the interface and doesn't rely on a more specific type (which would be enforced by the type system and thus not silent if incompatible).