4 ms·
I have found that among software engineers, it is surprisingly not common knowledge that floating point operations have all these sharp edges and gotchas. The
by opportune 4y ago
I have found that among software engineers, it is surprisingly not common knowledge that floating point operations have all these sharp edges and gotchas.
The most common situation in which it crops up is when dealing with quantities that require fractional units/arithmetic of some commonly discrete unit of measure. For example, you implement some complex logic to do request sampling, and in your binary you convert the total number of active requests to a float, add some stuff, divide some stuff, add some more stuff, multiply it again, then convert back to an int something like “number of requests that should be sampled.” Because floating point operations are non-associative, non-distributive, and commonly introduce remainder artifacts, you can end up with results like sampling 1 more request than there are total requests active, even when the arithmetic itself seems like that should be impossible.
This is also common when dealing with time, although typically the outcome is not that bad. Despite time having a simple workaround of just changing the unit of measure (eg using milliseconds instead of seconds) and using int operations on that, because people don’t know why they shouldn’t use floating point operations in this case, they don’t always reach for it.
The worst is when some complicated operation is done to report a float (or int converted from a float) as a metric. In the request sampling example, that would likely be noticed quickly and fixed. But when the float value looks reasonable enough and doesn’t violate some kind of system invariant, it can feed you bad data for a very long time before someone catches it.
- mooooooooooooo 4y agoWould you happen to have any resources on how to treat floating point values? I noticed some odd behaviour recently when using ruby to save to Postgres where the handoff between the two systems introduced imprecision in the saved value. Didn’t get to dig into it because it wasn’t a priority but it’s definitely an annoying unanswered question.
- cbhl 4y agoIf you want a thorough understanding: you'll want to look up "numerical computation", "numerical methods" or "computational methods"; in particular computing error bounds and error propagation. Typically covered in bachelor's level university-level Math, CS, or Engineering departments. If you just want to fix the odd behavior: adjust the schema so that you only work with whole numbers. For example, in the database schema, you can use DECIMAL instead of REAL/DOUBLE columns, or use two columns to specify the ratio of two integers (for example num/denom in frame rates in various video containers/codecs). In the application code: work only in whole numbers (e.g. cents, satoshis) instead of fractionals, using bigint or string types as applicable instead of e.g. double.
- gowld 4y agoOr Fixed Point numbers .
- flopriore 4y agoDon't Fixed Point numbers add more issues? Let's assume you have a machine that accepts 8-bits fixed-point numbers, with 6 bits for integers and 2 for decimals. For simplicity, let's use decimal digits. If you must represent the number 2.013, the resulting number would be +00002.01 So you cut the third decimal digit and wasted 4 bits with useless zeros. At the same time, if you represented the same number in 8 bits floating point, you would have all the digits.
- brianwawok 4y agoDepends what you track. Fixed point is great for money. Floating point is not. Using an integer cents is easier than both.
- mfashby 4y agoNot necessarily explaining how to do it right, but lots of examples of pitfalls https://jvns.ca/blog/2023/01/13/examples-of-floating-point-problems/ https://jvns.ca/blog/2023/01/13/examples-of-floating-point-p...
- aDfbrtVt 4y agoI have had this page bookmarked for years, hopefully you find it as useful as I do. It is a formal yet digestible treatment of floating point numbers. https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.h...
- opportune 4y agoIn addition to what others mentioned, to start from the basics I would try to learn how floating point values are implemented and how processors evaluate floating point expressions. For example the float is separated into two parts, it’s not able to efficiently represent many decimal numbers, etc. A simple rule of thumb is to try to avoid using floating point values at all outside of contexts like scientific simulations. For basic situations, you can almost always use either a library (for big numbers, decimals, fractions, etc.) or express your logic with ints by using established patterns (make the unit of measurement smaller/bigger, explicitly round up or down, etc.). Any time you take something that is an int 99% of the time, convert it to a float for something, then convert it back to an int, you are doing something wrong.