8 ms·
I found the following note for -ffinite-math-only and -fno-signed-zeros quite worrying: The program may behave in strange ways (such as not evaluating either t
by rocqua 5y ago
I found the following note for -ffinite-math-only and -fno-signed-zeros quite worrying:
The program may behave in strange ways (such as not evaluating either the true or false part of an if-statement) if calculations produce Inf, NaN, or -0.0 when these flags are used.
I always thought that -ffast-math was telling the compiler. "I do not care about floating point standards compliance, and I do not rely on it. So optimize things that break the standard".
But instead it seems like this also implies a promise to the compiler. A promise that you will not produce Inf, NaN, or -0.0.
Whilst especially Inf and NaN can be hard to exclude.
This changes the flag from saying "don't care about standards, make it fast" to "I hereby guarantee this code meets a stricter standard" where also it becomes quite hard to actually deduce you will meet this standard. Especially if you want to actually keep your performance. Because if you need to start putting all divisions in if statements to prevent getting Inf or NaN, that is a huge performance penalty.
- praptak 5y agoNot only math optimizations are like this. Strict aliasing is a promise to the compiler to not make two pointers of different types point to the same address. You technically make that promise by writing in C (except a narrow set of allowed conversions) but most compilers on standard settings do let you get away with it.
- okl 5y ago> A promise that you will not produce Inf, NaN, or -0.0. Whilst especially Inf and NaN can be hard to exclude. Cumbersome but not that difficult: Range-check all input values.
- orangepanda 5y agoAs the compiler assumes you wont produce such values, wouldnt it also optimise those range checks away?
- nly 5y agoHe means simple stuff like ensuring a function like float unit_price (float volume, int count) { return volume / count; } is only called where count > 0
- colejohnson66 5y agoBut couldn’t the compiler optimize out the check because you’re guaranteeing to never divide by 0, so logically count would never be 0. Akin to undefined behavior almost.
- throwawaylinux 5y agoThis test can not be optimized to always true: if (count > 0) total += unit_price(volume, count); Are you confusing it with the opposite scenario: total += unit_price(volume, count); if (count == 0) printf("oops\n"); That test might be optimized out because divide by zero is undefined behavior.
- colejohnson66 5y agoI probably am confusing them. I don’t program in C or C++, so I’m not aware of all the “gotchas”, and can easily confuse them (as shown here). Not to mention that compilers have been getting more aggressive in optimizing undefined behavior.
- adrian_b 5y agoWhat the compiler assumes depends on the compilation options. Unfortunately most compilers have too permissive default options. C and C++ programs should always be compiled, at least during development, with strict options, e.g.: -fsanitize=undefined -fsanitize-undefined-trap-on-error or equivalent options, which eliminate all the horror stories with unexpected behavior caused by integer divide-by-zero or overflow, pointer overflow, out-of-bounds access and many others. Because unlike integer operations the FP operations are standardized, there is no need for compiler options, the behavior for exceptions can be controlled from the program with the functions defined in <fenv.h> in C or <cfenv> in C++.
- contravariant 5y agoAnd don't do any division. Easy.
- scheme271 5y agoOr multiplications or additions to avoid overflows or denormals
- okl 5y agoThere's nothing wrong with divisions if you exclude problematic combinations of input values.
- contravariant 5y agoYou're going to have to do some intermediate checks if you want stuff like x/sin(x) to behave.
- fho 5y agoThat's where more advanced type systems shine. You get to encode things like "this number is not null" and a safe division function will only take "not nulls" as the input. The workflow would then be: 1. prove that the input is not null (if input == 0) yielding a "not null number" 2. running your calculation (no check needed for multiplication or division, if you add something you have to reprove that the number is not null)
- benttoothpaste 5y agoThe checks could be expensive though, more expensive than the gains from this optimization. Often it is better to let the calculation proceed and use isnan to check the final result.
- adrian_b 5y agoThe whole rationale for the standard for floating-point operations is to specify the FP operations with such properties that a naive programmer will be able to write programs which will behave as expected. If you choose any option that is not compliant with the standard, that means, exactly as you have noticed, that you claim that you are an expert in FP computations and you know how to write FP programs that will give the desired results even with FP operations that can behave in strange ways. So yes, that means that you become responsible to either guarantee that erroneous results do not matter or that you will take care to always check the ranges of input operands, as "okl" has already posted, to ensure that no overflows, underflows or undefined operations will happen.
- celrod 5y ago> So yes, that means that you become responsible to either guarantee that erroneous results do not matter or that you will take care to always check the ranges of input operands, as "okl" has already posted, to ensure that no overflows, underflows or undefined operations will happen. This is why I dislike the fact that it removes simple methods for checking if values are NaN, for example. I do not find it ergonomic to disable these checks.
- kazinator 5y agoIf NaN and Inf values can still happen, but you don't have reliable ways of working with them, that is poor. I think what people really want is a sane floating point mode that throws exceptions instead of these Inf and NaN things, so if their program isn't debugged, they get a clear signal by way of its abnormal termination.
- billfruit 5y agoIs it adviced to detect overflow and do something about it after an FP computation rather than adding pre-checks to try to avert the overflow?
- owlbite 5y agoSo it depends on how you want to detect overflow. Trapping on math exceptions or otherwise pulling an emergency break doesn't really fit with modern hardware design if you want performance (for one thing, what do you do with the rest of your vector? do you just live with a pipeline flush?). Adding targeted checks used sparingly can work, but probably requires a deeper understanding of the underlying numerical analysis of your problem. Generally just checking for NaN or Inf at the end is a better solution as these are in most cases absorbing states.
- gpderetta 5y agoI think the documentation is fairly clear: "Allow optimizations for floating-point arithmetic that assume that arguments and results are not NaNs or +-Infs." The obvious implication is that is that arguments and results are always finite and as a programmer you are responsible of guaranteeing the correct preconditions. Certainly do not use finite-math-only when dealing with external data.
- hurrrrrrrr 5y ago>Certainly do not use finite-math-only when dealing with external data. so... never?
- gpderetta 5y agolet me rephrase: data which you cannot guarantee meet the preconditions. Either the input is known good data or you sanitize it through some code path not compiled with fast-math.
- pinteresting 5y agoGenerally if you're seeing a NaN/Inf something has gone wrong, It's very difficult to gracefully recover from and if you tried I think you would lose both sanity and performance! Regarding performance, the cost of a real division is about 3-4 orders worse performance than an if statement that is very consistent, but the usual way is to have fast/safe versions of functions, where you need performance and can deduce if something is always/never true, create/use the fast function but by default everything uses the slower/safer function.
- nemetroid 5y ago> Generally if you're seeing a NaN/Inf something has gone wrong That’s a bold claim.
- pinteresting 5y agoCan you think of a function where the input is valid, the output is NaN and nothing has gone wrong in the process? I can't think of any, haven't experienced any, not heard of any examples of it, so you're welcome to break my ignorance on the subject.
- ImprobableTruth 5y agoDo you count under/overflow resulting in Inf as 'something has gone wrong'? If so, why would gracefully recovering be hard?
- vanviegen 5y agoBecause you're about 15 stack frames deep into some calculation when the problem gets noticed. So how do you handle this? You could throw an exception (or similar), which would allow the UI to state something semi-helpful like 'Calculation failed'. But recover? Usually not. I think NaN and inf can be really useful in specific cases, but they probably shouldn't be allowed to propagate through your program, and shouldn't occur unless you're specifically making use of them.
- heavenlyblue 5y ago> such as not evaluating either the true or false part of an if-statement Do you mean `if-else if` or `if-else`? Because the second would work and would always short-circuit to else.
- nwallin 5y agoNo. Imagine you have this: if (x < 7) doSomething(); else doSomethingElse(); With -ffast-math, the compiler is allowed to call neither doSomething() nor doSomethingElse() if x is nan. Which IMHO is not great.
- Linosaurus 5y agoI imagine it means that the compiler could turn an if(x<6)-else into two free standing if statement like if(x<6), if(x>=6). Which will logically work under these assumptions.
- dspillett 5y ago> I always thought that -ffast-math was telling the compiler. "I do not care about floating point standards compliance, ... So optimize things that break the standard". But instead it seems like this also implies a promise to the compiler [that] you will not produce Inf, NaN, or -0.0. I see it as both, as with many such flags¹n though as you say it does put the onus on the developer to avoid inputs the optimisation may cause problems for. You are saying “I promise that I'm being careful not to do things that will be affected by these parts of the standard, or won't blame you for undefined behaviour if I do, so you can skip those rules if it helps you speed things up”. [1] like those that enable optimisations which can cause significant breakage if pointer aliasing is happening
- 0xTJ 5y agoThat's the way a number of those special flags work. They are a contract to the compiler saying "act like this can't happen, I promise it won't". The picking and choosing specific flags enable might be better in general. The flag `-funsafe-math-optimizations` seems to be useful as general-purpose, although as this article mentions, some of those optimizations need the more unsafe ones to become active. I would say that `-ffast-math` is a dangerously safe-sounding flag, `-funsafe-fast-math` would be better IMO.
- phkahler 5y agoI thought the idea was to never produce NaN or -0 in the first place, but it seems that's not the case.
- deleted 5y ago[deleted]