3 ms·
How big can my world be in meters if I require at least 0.01mm precision and I model things in millimeters? Does that number change if I model in meters?
by dahart 9y ago
How big can my world be in meters if I require at least 0.01mm precision and I model things in millimeters? Does that number change if I model in meters?
- fyi1183 9y agoBecause of how floating point works, it doesn't matter. The difference between modelling in meters or millimeters disappears in the exponents. What matters for precision is the mantissa, which is 23 bits in single precision floats. So you get ~16 million of your smallest unit before you're losing precision, counting the implied leading 1. You can double that to ~32 million by using the sign bit. Edit: this translates to 320m if you want 0.01mm precision.
- Const-me 9y agoAbout 0.008 km^3, each coordinate between -100 and +100 meters. The number's the roughly the same even if you'll model in km or imperial units.
- dahart 9y agoYeah, you're right. Brain fart. I'm mis-remembering production stories instead of thinking clearly about floats, despite having just linked the precision chart. :P