4 ms·

> The problem with this argument is that it doesn't match economic reality. Compiler authors wouldn't disagree that it would be nice to not have to do these opt

> The problem with this argument is that it doesn't match economic reality. Compiler authors wouldn't disagree that it would be nice to not have to do these optimizations! I think that the extent to which that's actually true is pretty debatable. Note that both Linux and Postgres use -fwrapv/-fno-strict-overflow and -fno-strict-aliasing. I'm not a compiler engineer, but if I was I might think to ask why that's the policy of these projects, and address the criticism on something closer to its own terms. The author makes a clear distinction between "Optimization*" and "Optimization". That distinction might be a bit squishy, but not so much that it isn't still useful. Even the Fabian Giesen thing that you linked to says "A lot of people (myself included) are against transforms that aggressively exploit undefined behavior", right at the start. It's trendy to blame the C language itself, but surely we can expect C compilers to care about the C language.

jessewmc 8y ago
I think Linux and Postgres are much more exception than norm programming. They’re both relatively low level, critical infrastructure code worked on by domain experts with low level expertise —- not at all what most software looks like, and exactly the sort of people who would override defaults anyway.
brandmeyer 8y ago
The problem is that this group of programmers are operating on a language that is almost-but-not-quite C. They would very much like to believe that there is a set of flags which make the compiler run in the mode of almost-C that the application engineers thought they were working with all those years ago. But the fact of the matter is that the ISO C standard is the only contract which has been written down between application authors and compiler authors. No amount of flags are enough to guarantee that a particular individual's expectation of almost-C will be equal to what their compiler is producing. What the industry needs to move forward are two things. A set of tools that can help identify more cases of almost-C that need to be ported to C. We're getting this, in the form of UBSAN, ASAN, and their kin. The other thing is for the veterans to accept that their particular brand of almost-C doens't actually exist. There's only ISO C. For almost three decades now, that's all we've had. the engineering cost of trying to keep almost-C alive is just too high to keep paying.
- mpweiher 8y ago
  This is a common trope ("language users are just ignorant"), but couldn't be more wrong. There actually is explicit language in the C standard that gives a range of "permissible behaviours": Permissible undefined behavior ranges from ignoring the situation completely with unpredictable results, to behaving during translation or program execution in a documented manner characteristic of the environment (with or without the issuance of a diagnostic message), to terminating a translation or execution (with the issuance of a diagnostic message). This gives a range of permissible behaviours, and what current compilers do with UB clearly does not fall into that range, whereas it closely matches the expectations of those "ignorant" language users. In the first ANSI/ISO C standard, this language was binding, it was only in later version that it was declared non-binding. So the language is in the standard, but implementers are free to ignore it and still call themselves standards-compliant, which they do with abandon. > the ISO C standard is the only contract This is also a common trope, and also not (completely) true. In fact, the C standard very explicitly says that the standard by itself is not sufficient to make a useful/usable implementation! There is a way in which this sentiment is becoming true, which is that we used to have a different contract between user and compiler writer: a sales contract. Users were also customers, and if you produced a compiler that only conformed to the standard, your customers would stop giving you money. Today, compilers are largely free for users, and therefore the standard actually is in many ways the only "contract" between user and writer. So that's a problem, as the goals and incentives aren't aligned.
  - brandmeyer 8y ago
    > This is a common trope ("language users are just ignorant"), but couldn't be more wrong. It isn't a matter of ignorance; using those switches to get a vendor-specific version of not-quite-C is a matter of willful disobedience.
    - mpweiher 8y ago
      Hmmm...actually, those switches turn the no-longer-C back into C.
pcwalton 8y ago
Compiler developers do care about the C language. They just care about it as it's actually used. In real-world C, people write "for (int i = 0; i < array->length; i++)". The Linux kernel might not, but that's a choice the Linux kernel developers have made. Lots of C code uses different idioms from the Linux kernel, and users expect that code to run fast. Slowing down that code out of a notion of language purity (especially when "Friendly C" isn't even defined) does a disservice to users. I can't speak for anyone but me, but I read the opposition to doing these kinds of optimizations in that Gist as preferring compiler warnings to suggest users use size_t when appropriate. This would be great, but I'm skeptical that, for example, Linux distros would be happy about that. Most Linux distributions consist of packages of vast amounts of unmaintained or mostly unmaintained C code. There simply isn't the manpower available to fix all that code to be properly 64-bit optimized (which is in fact why int is 32 bits on x86-64 Linux in the first place).
- petergeoghegan 8y ago
  I'm suggesting that compiler authors don't seem to pay sufficient attention to how C is actually used, though. It may well be true that I don't have the full picture here, but I don't think that these concerns can be waved away so easily. Postgres has code like "for (int i = 0; i < array->length; i++)" in many places. I believe that the same is true of the kernel.
  - pcwalton 8y ago
    > I'm suggesting that compiler authors don't seem to pay sufficient attention to how C is actually used, though. Compiler developers pay more attention to how C is used than virtually anyone else. After all, they are the ones who have to deal with bug reports--often bug reports saying that code is slow. It's not like compiler engineers implement optimizations for no reason, or just to make themselves seem smart. They implement optimizations because somebody files a bug report saying that some code is slow, and they go to fix it. > Postgres has code like "for (int i = 0; i < array->length; i++)" in many places. I believe that the same is true of the kernel. If that's true, then Postgres is giving up performance on 64-bit for that, which can be significant. That is a decision they can opt into. It's not a decision everyone else should have to opt out of.

jessewmc 8y ago

I think Linux and Postgres are much more exception than norm programming. They’re both relatively low level, critical infrastructure code worked on by domain experts with low level expertise —- not at all what most software looks like, and exactly the sort of people who would override defaults anyway.

brandmeyer 8y ago

The problem is that this group of programmers are operating on a language that is almost-but-not-quite C. They would very much like to believe that there is a set of flags which make the compiler run in the mode of almost-C that the application engineers thought they were working with all those years ago. But the fact of the matter is that the ISO C standard is the only contract which has been written down between application authors and compiler authors. No amount of flags are enough to guarantee that a particular individual's expectation of almost-C will be equal to what their compiler is producing. What the industry needs to move forward are two things. A set of tools that can help identify more cases of almost-C that need to be ported to C. We're getting this, in the form of UBSAN, ASAN, and their kin. The other thing is for the veterans to accept that their particular brand of almost-C doens't actually exist. There's only ISO C. For almost three decades now, that's all we've had. the engineering cost of trying to keep almost-C alive is just too high to keep paying.

mpweiher 8y ago
This is a common trope ("language users are just ignorant"), but couldn't be more wrong. There actually is explicit language in the C standard that gives a range of "permissible behaviours": Permissible undefined behavior ranges from ignoring the situation completely with unpredictable results, to behaving during translation or program execution in a documented manner characteristic of the environment (with or without the issuance of a diagnostic message), to terminating a translation or execution (with the issuance of a diagnostic message). This gives a range of permissible behaviours, and what current compilers do with UB clearly does not fall into that range, whereas it closely matches the expectations of those "ignorant" language users. In the first ANSI/ISO C standard, this language was binding, it was only in later version that it was declared non-binding. So the language is in the standard, but implementers are free to ignore it and still call themselves standards-compliant, which they do with abandon. > the ISO C standard is the only contract This is also a common trope, and also not (completely) true. In fact, the C standard very explicitly says that the standard by itself is not sufficient to make a useful/usable implementation! There is a way in which this sentiment is becoming true, which is that we used to have a different contract between user and compiler writer: a sales contract. Users were also customers, and if you produced a compiler that only conformed to the standard, your customers would stop giving you money. Today, compilers are largely free for users, and therefore the standard actually is in many ways the only "contract" between user and writer. So that's a problem, as the goals and incentives aren't aligned.
- brandmeyer 8y ago
  > This is a common trope ("language users are just ignorant"), but couldn't be more wrong. It isn't a matter of ignorance; using those switches to get a vendor-specific version of not-quite-C is a matter of willful disobedience.
  - mpweiher 8y ago
    Hmmm...actually, those switches turn the no-longer-C back into C.

pcwalton 8y ago

Compiler developers do care about the C language. They just care about it as it's actually used. In real-world C, people write "for (int i = 0; i < array->length; i++)". The Linux kernel might not, but that's a choice the Linux kernel developers have made. Lots of C code uses different idioms from the Linux kernel, and users expect that code to run fast. Slowing down that code out of a notion of language purity (especially when "Friendly C" isn't even defined) does a disservice to users. I can't speak for anyone but me, but I read the opposition to doing these kinds of optimizations in that Gist as preferring compiler warnings to suggest users use size_t when appropriate. This would be great, but I'm skeptical that, for example, Linux distros would be happy about that. Most Linux distributions consist of packages of vast amounts of unmaintained or mostly unmaintained C code. There simply isn't the manpower available to fix all that code to be properly 64-bit optimized (which is in fact why int is 32 bits on x86-64 Linux in the first place).

petergeoghegan 8y ago
I'm suggesting that compiler authors don't seem to pay sufficient attention to how C is actually used, though. It may well be true that I don't have the full picture here, but I don't think that these concerns can be waved away so easily. Postgres has code like "for (int i = 0; i < array->length; i++)" in many places. I believe that the same is true of the kernel.
- pcwalton 8y ago
  > I'm suggesting that compiler authors don't seem to pay sufficient attention to how C is actually used, though. Compiler developers pay more attention to how C is used than virtually anyone else. After all, they are the ones who have to deal with bug reports--often bug reports saying that code is slow. It's not like compiler engineers implement optimizations for no reason, or just to make themselves seem smart. They implement optimizations because somebody files a bug report saying that some code is slow, and they go to fix it. > Postgres has code like "for (int i = 0; i < array->length; i++)" in many places. I believe that the same is true of the kernel. If that's true, then Postgres is giving up performance on 64-bit for that, which can be significant. That is a decision they can opt into. It's not a decision everyone else should have to opt out of.