11 ms·

Author here, happy for any feedback. I'll own up to misleading-and-possibly-clickbait title, I just gave up trying to think of anything better without revealin

Author here, happy for any feedback. I'll own up to misleading-and-possibly-clickbait title, I just gave up trying to think of anything better without revealing the conclusion.

acemarke 7y ago
Clickbait, but _good_ clickbait :) I do so love good deep technical blog posts that dig into why something works a certain way, and the title is a legit statement based on the initial cause. Thanks for writing this!
Someone 7y ago
clang format can tank performance” would, IMO, not decrease the number of people who click the link, yet remove the clickbait.
- BeeOnRope 7y ago
  Could be, but I feel it would equally dishonest? This really has nothing to do with clang-format: I only mention that once (twice with an update: where clang format actually helps because it reorders stdlib.h after ctype.h) - it's all about header include order.
  - mipmap04 7y ago
    Maybe someone should make a tool that randomly reorders your header includes until optimal performance is found.
  - emmelaich 7y ago
    I think clang-format is definitely relevant; the take-away here is that no formatter should change the order of includes. I was wondering about this only yesterday - tidying up some C++ and C code. I don't remember any standard that says it's OK to re-order headers, and clearly it can matter a lot.
    - BeeOnRope 7y ago
      That wouldn't be my take away. My takeway would be anything that changes in semantics or performance due to header includes should be considered a bug by the (possibly joint) writers of the involved headers. clang-format is an innocent victim here. Maybe I should make that clearer.
      - flukus 7y ago
        Clang isn't the root cause, but as a (relatively) new tool in an existing ecosystem it needs to work with the existing ecosystem and not some imaginary idealistic world.
      - mypalmike 7y ago
        Changing semantics due to header order is considered a feature in C/C++. If you aren't willing to waste hours working through esoteric header issues, you just aren't tough enough.
BubRoss 7y ago
Why does every title have to be a mystery? Why not tell people what it contains?
- BeeOnRope 7y ago
  In this case it's an excerpt of a larger post that I was writing while I ran into this. I don't think there is a good single-sentence title that would encapsulate the whole thing without being an already well-known fact, like say "Header include order matters". Also, I wrote this one as a narrative, not revealing the conclusion at the start - basically a reconstructed version of how I encountered it myself. I don't write all my posts like that and it's not for everyone: but once I've decided on that, it makes no sense to reveal the conclusion in the title, I think? The HN headline could be changed though if it places a priority on information-full titles.
  - pvg 7y ago
    It places priority on not-clickbait, above most things. You could have a title that isn't super descriptive, asks a question, maintains an air of mystery, whatever. But a title that deliberately elevates some near-irrelevant detail for attention is purposefully deceptive and, well, less than good (by HN's rules, that is).
    - BeeOnRope 7y ago
      Eh, sure - but you are arguing for non-clickbait titles, while I was replying to the OP who complained about not revealing more about the conclusion in the title. It seems we agree there that some mystery is allowed: the point of contention is on "click-baitiness". I don't have a similar principled defense of clickbait titles, either in general or mine specifically! I just thought it was a funny one and didn't expend more effort on coming up with a non-clickait one. That's an independent issue from revealing more info in the title.
      - pvg 7y ago
        Oh, I don't really have any argument about what titles you should put on your own writing. It's your writing, if you feel its narrative purpose is served by an intricate web of title feints, line noise, whatnot, more power to you. I'm just explaining the HN title rules, you can see them here (do a find title): https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html There's nothing in there about titles having to be 'information-full'. Lots of perfectly HN-good titles aren't.
JonathonW 7y ago
Just to make sure I'm understanding this correctly: the performance issue you found actually amounts to include-order dependent behavior in GCC and glibc; clang doesn't really have anything to do with anything here except that you used its code formatter to sort your include statements. If that's the case, the clickbait-y headline's bordering on useless, since the performance differences here have nothing to do with clang itself-- you'd see the same results if you had ordered your includes the same way by hand.
- big_chungus 7y ago
  No, the headline pretty clearly states "Clang Format". I've used clang-format on MSVC codebases, and imagine there are some who use it with GCC (say, for instance, the author). It's still the best C++ formatter, as far as I know, and is compiler-independent.
  - BeeOnRope 7y ago
    It's basically the gorilla in the space, and it's good. There is no gcc equivalent. In any case there no 1:1 relationship between formatters anyways: you might compiler you codebase with N compilers, but you'll only have one formatter unless you are insane. So my codebases are not "gcc" or "clang" codebases, but basically "linux" codebases - but the formatter is always clang-format.
- BeeOnRope 7y ago
  I'm talking about clang-format, a formatting tool for C and C++ code, which has nothing to do with compiling your code with clang. You can use it with any compiler. That said, the title is still a lie in that clang-format is not really at a fault here at all: clang-format triggered the issue on my codebase, by sorting header files, and this in turn triggered a libc performance issue. So as I initially say it, a commit which purely clang-format 'd my codebase caused a massive regression - but the full story is doesn't implicate clang-format at all, really.
  - glandium 7y ago
pydave 7y ago
> Are my standard algorithms letting me down? Does std::transform have some fatal flaw? Not really. Well, not at all. This part lost me because I can't see how you can use std::transform (which is in <algorithm>) without hitting this flaw. Unless you edit the headers? I guess you mean the algorithm isn't flawed. I also didn't understand the title because I don't use clang-format and I skimmed past the single mention of it (my fault). Regardless, interesting post.
- deleted 7y ago
  [deleted]
- BeeOnRope 7y ago
  The title was basically clickbait. It reflects how I encountered the issue, but the investigation doesn't have much of that, because I was starting from a point where I suddenly had a slow and fast algorithm (the actual scenario was more complicated than shown). The investigation doesn't have much to do with clang-format because I'm just trying to see why a raw loop is faster, and ultimately has nothing to do with "raw" vs "std" at all, really. Only when I understood that includes and include order matter was I able to map it back to a header order swap made by clang-format. > This part lost me because I can't see how you can use std::transform (which is in <algorithm>) without hitting this flaw. No, two ways. You can get the fast performance with std::transform if you happen to include <ctype.h> before <algorithm>. The order matters. Similarly, you can get the slow performance with a raw loop if you happen to include <algorithm> before <ctype.h> in the file for the raw loop. In fact, I'd say that the raw loop and std::transform will have the same performance almost all the time. If they are in the same file, they will have the same performance. If they are in a C++ file, you are almost certainly including some C++ header that triggers the issue. Only in special cases, like a coding convention that separates C and C++ headers with whitespace with C first (clang-format doesn't sort headers separated by whitespace) are you likely to come into it "natively". Even if you do, it's absolutely no fault of std::transform or <algorithm> - it's a weird effect of interaction between C, C++ and OS headers, nothing inherent to the C++ algorithms.
  - TwoBit 7y ago
fyp 7y ago
The "three observations that aren’t really part of the story" were really interesting and doesn't seem to be fully resolved. Will you be discussing them more in the future post?
- BeeOnRope 7y ago
  Maybe. This is really a three part answer, since there were three quite-different effects I glossed over. The first one is interesting, but it is also well-covered elsewhere, e.g., [1], [2] and [3]. That said, I have a ton of stuff to say on branch prediction, so I probably will at some point. Those posts are slow going, however. It may never emerge. For the second and third points, I am partly blocked by the fact the effect disappeared or is only intermittently reproducible. I am more interested in the third uninteresting effect than the second. The second thing could be a lot of things, since it shows up when running a python script with results piped to disk. What I've seen in the past is that the "driver" program (driver.py) undergoes some kind of phase change, e.g., suddenly causing a bunch of context switches or allocating/freeing a munch of memory which affects the program under test (despite CPU pinning, since there are shared resources). They are phantoms (but still interesting!). The third one is very interesting to me, because it involves a CPU-bound loop with only L1 hits. It goes to the core of the Skylake uarch and the kind of fine-grained uarch details I'm interested in. However, it disappeared as I was tracking it down. I was getting a mix of 2.07 cycles and 1.57 cycles (slow and fast) running the benchmark repeatedly, trying differnet perf counters and doing the needful to track it down and then suddenly I only got 2.07. A new plot showed "blue" only at ~2.07. So I don't know. I have seen similar things though, and I've tracked down an similar "mode shift" and will post on it soon. Stay tuned. --- [1] https://lemire.me/blog/2019/11/12/unrolling-your-loops-can-improve-branch-prediction/ https://lemire.me/blog/2019/11/12/unrolling-your-loops-can-i... [2] https://lemire.me/blog/2019/10/16/benchmarking-is-hard-processors-learn-to-predict-branches/ https://lemire.me/blog/2019/10/16/benchmarking-is-hard-proce... [3] https://lemire.me/blog/2019/10/15/mispredicted-branches-can-multiply-your-running-times/ https://lemire.me/blog/2019/10/15/mispredicted-branches-can-...
gpm 7y ago
I'm going to take the contrarian view here, and say it's not clickbait. I admit that I'm not a C++ expert, but naively I would never have expected clang-format to have any effect on my code. If you stopped and suggested header re-ordering to me my first thought would have been "I guess it must not do that. Maybe in practice header re-ordering doesn't actually matter with reasonable code?". The title here doesn't describe the contents of the article, but it describes a very important consequence of it that C++ programmers should be aware of. It gives a specific example of the consequence. I'm ok with that.
- BeeOnRope 7y ago
  I agree with most of that, although as the author I think it's still clickbait in the sense that: 1) clang-format is not really at fault here at all - the same effect could have happened by adding a new header, reordering the headers manually, switching to a system with a different libc version or a different transitive header include tree, etc. 2) The title doesn't tell you wtf is up. You have to get at least half way through the post to figure that out. It's in the vein of "This one food could wreck your health" posts. That said, other aspects of clickbait are advertising or monetization to profit from the clicks. Except for the crypto mining bot I load into your browser, and which subsequently writes itself to your MBR or UEFI partition, I have no ads, affiliate links, patreon-alikes or other monetization vectors.
  - fizwhiz 7y ago
    > Except for the crypto mining bot I load into your browser, and which subsequently writes itself to your MBR or UEFI partition Guess I won't be clicking on the link now.
    - skrebbel 7y ago
      You're missing out. It's a great article, definitely worth all that monero.
  - TehCorwiz 7y ago
    At least you’re forthcoming.
iameli 7y ago
I clicked because of the title and laughed when I read the reason, don't sweat about anyone accusing you of clickbait
jimbo1qaz 7y ago
I read the title and thought the article was about (clang-format's performance) on a (program simulating military tanks)... Apparently you meant "tanks" as a verb...
- BeeOnRope 7y ago
  Yes, although a quick search reveals that Oxford considers this a US usage so it was probably not the ideal choice for a title.
deleted 7y ago
[deleted]

Author here, happy for any feedback. I'll own up to misleading-and-possibly-clickbait title, I just gave up trying to think of anything better without revealing the conclusion.

acemarke 7y ago

Clickbait, but _good_ clickbait :) I do so love good deep technical blog posts that dig into why something works a certain way, and the title is a legit statement based on the initial cause. Thanks for writing this!

Someone 7y ago

clang format can tank performance” would, IMO, not decrease the number of people who click the link, yet remove the clickbait.

BeeOnRope 7y ago
Could be, but I feel it would equally dishonest? This really has nothing to do with clang-format: I only mention that once (twice with an update: where clang format actually helps because it reorders stdlib.h after ctype.h) - it's all about header include order.
- mipmap04 7y ago
  Maybe someone should make a tool that randomly reorders your header includes until optimal performance is found.
- emmelaich 7y ago
  I think clang-format is definitely relevant; the take-away here is that no formatter should change the order of includes. I was wondering about this only yesterday - tidying up some C++ and C code. I don't remember any standard that says it's OK to re-order headers, and clearly it can matter a lot.
  - BeeOnRope 7y ago
    That wouldn't be my take away. My takeway would be anything that changes in semantics or performance due to header includes should be considered a bug by the (possibly joint) writers of the involved headers. clang-format is an innocent victim here. Maybe I should make that clearer.
    - flukus 7y ago
      Clang isn't the root cause, but as a (relatively) new tool in an existing ecosystem it needs to work with the existing ecosystem and not some imaginary idealistic world.
    - mypalmike 7y ago
      Changing semantics due to header order is considered a feature in C/C++. If you aren't willing to waste hours working through esoteric header issues, you just aren't tough enough.

BubRoss 7y ago

Why does every title have to be a mystery? Why not tell people what it contains?

BeeOnRope 7y ago
In this case it's an excerpt of a larger post that I was writing while I ran into this. I don't think there is a good single-sentence title that would encapsulate the whole thing without being an already well-known fact, like say "Header include order matters". Also, I wrote this one as a narrative, not revealing the conclusion at the start - basically a reconstructed version of how I encountered it myself. I don't write all my posts like that and it's not for everyone: but once I've decided on that, it makes no sense to reveal the conclusion in the title, I think? The HN headline could be changed though if it places a priority on information-full titles.
- pvg 7y ago
  It places priority on not-clickbait, above most things. You could have a title that isn't super descriptive, asks a question, maintains an air of mystery, whatever. But a title that deliberately elevates some near-irrelevant detail for attention is purposefully deceptive and, well, less than good (by HN's rules, that is).
  - BeeOnRope 7y ago
    Eh, sure - but you are arguing for non-clickbait titles, while I was replying to the OP who complained about not revealing more about the conclusion in the title. It seems we agree there that some mystery is allowed: the point of contention is on "click-baitiness". I don't have a similar principled defense of clickbait titles, either in general or mine specifically! I just thought it was a funny one and didn't expend more effort on coming up with a non-clickait one. That's an independent issue from revealing more info in the title.
    - pvg 7y ago
      Oh, I don't really have any argument about what titles you should put on your own writing. It's your writing, if you feel its narrative purpose is served by an intricate web of title feints, line noise, whatnot, more power to you. I'm just explaining the HN title rules, you can see them here (do a find title): https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html There's nothing in there about titles having to be 'information-full'. Lots of perfectly HN-good titles aren't.

JonathonW 7y ago

Just to make sure I'm understanding this correctly: the performance issue you found actually amounts to include-order dependent behavior in GCC and glibc; clang doesn't really have anything to do with anything here except that you used its code formatter to sort your include statements. If that's the case, the clickbait-y headline's bordering on useless, since the performance differences here have nothing to do with clang itself-- you'd see the same results if you had ordered your includes the same way by hand.

big_chungus 7y ago
No, the headline pretty clearly states "Clang Format". I've used clang-format on MSVC codebases, and imagine there are some who use it with GCC (say, for instance, the author). It's still the best C++ formatter, as far as I know, and is compiler-independent.
- BeeOnRope 7y ago
  It's basically the gorilla in the space, and it's good. There is no gcc equivalent. In any case there no 1:1 relationship between formatters anyways: you might compiler you codebase with N compilers, but you'll only have one formatter unless you are insane. So my codebases are not "gcc" or "clang" codebases, but basically "linux" codebases - but the formatter is always clang-format.
BeeOnRope 7y ago
I'm talking about clang-format, a formatting tool for C and C++ code, which has nothing to do with compiling your code with clang. You can use it with any compiler. That said, the title is still a lie in that clang-format is not really at a fault here at all: clang-format triggered the issue on my codebase, by sorting header files, and this in turn triggered a libc performance issue. So as I initially say it, a commit which purely clang-format 'd my codebase caused a massive regression - but the full story is doesn't implicate clang-format at all, really.
- glandium 7y ago

pydave 7y ago

> Are my standard algorithms letting me down? Does std::transform have some fatal flaw? Not really. Well, not at all. This part lost me because I can't see how you can use std::transform (which is in <algorithm>) without hitting this flaw. Unless you edit the headers? I guess you mean the algorithm isn't flawed. I also didn't understand the title because I don't use clang-format and I skimmed past the single mention of it (my fault). Regardless, interesting post.

deleted 7y ago
[deleted]
BeeOnRope 7y ago
The title was basically clickbait. It reflects how I encountered the issue, but the investigation doesn't have much of that, because I was starting from a point where I suddenly had a slow and fast algorithm (the actual scenario was more complicated than shown). The investigation doesn't have much to do with clang-format because I'm just trying to see why a raw loop is faster, and ultimately has nothing to do with "raw" vs "std" at all, really. Only when I understood that includes and include order matter was I able to map it back to a header order swap made by clang-format. > This part lost me because I can't see how you can use std::transform (which is in <algorithm>) without hitting this flaw. No, two ways. You can get the fast performance with std::transform if you happen to include <ctype.h> before <algorithm>. The order matters. Similarly, you can get the slow performance with a raw loop if you happen to include <algorithm> before <ctype.h> in the file for the raw loop. In fact, I'd say that the raw loop and std::transform will have the same performance almost all the time. If they are in the same file, they will have the same performance. If they are in a C++ file, you are almost certainly including some C++ header that triggers the issue. Only in special cases, like a coding convention that separates C and C++ headers with whitespace with C first (clang-format doesn't sort headers separated by whitespace) are you likely to come into it "natively". Even if you do, it's absolutely no fault of std::transform or <algorithm> - it's a weird effect of interaction between C, C++ and OS headers, nothing inherent to the C++ algorithms.
- TwoBit 7y ago

fyp 7y ago

The "three observations that aren’t really part of the story" were really interesting and doesn't seem to be fully resolved. Will you be discussing them more in the future post?

BeeOnRope 7y ago
Maybe. This is really a three part answer, since there were three quite-different effects I glossed over. The first one is interesting, but it is also well-covered elsewhere, e.g., [1], [2] and [3]. That said, I have a ton of stuff to say on branch prediction, so I probably will at some point. Those posts are slow going, however. It may never emerge. For the second and third points, I am partly blocked by the fact the effect disappeared or is only intermittently reproducible. I am more interested in the third uninteresting effect than the second. The second thing could be a lot of things, since it shows up when running a python script with results piped to disk. What I've seen in the past is that the "driver" program (driver.py) undergoes some kind of phase change, e.g., suddenly causing a bunch of context switches or allocating/freeing a munch of memory which affects the program under test (despite CPU pinning, since there are shared resources). They are phantoms (but still interesting!). The third one is very interesting to me, because it involves a CPU-bound loop with only L1 hits. It goes to the core of the Skylake uarch and the kind of fine-grained uarch details I'm interested in. However, it disappeared as I was tracking it down. I was getting a mix of 2.07 cycles and 1.57 cycles (slow and fast) running the benchmark repeatedly, trying differnet perf counters and doing the needful to track it down and then suddenly I only got 2.07. A new plot showed "blue" only at ~2.07. So I don't know. I have seen similar things though, and I've tracked down an similar "mode shift" and will post on it soon. Stay tuned. --- [1] https://lemire.me/blog/2019/11/12/unrolling-your-loops-can-improve-branch-prediction/ https://lemire.me/blog/2019/11/12/unrolling-your-loops-can-i... [2] https://lemire.me/blog/2019/10/16/benchmarking-is-hard-processors-learn-to-predict-branches/ https://lemire.me/blog/2019/10/16/benchmarking-is-hard-proce... [3] https://lemire.me/blog/2019/10/15/mispredicted-branches-can-multiply-your-running-times/ https://lemire.me/blog/2019/10/15/mispredicted-branches-can-...

gpm 7y ago

I'm going to take the contrarian view here, and say it's not clickbait. I admit that I'm not a C++ expert, but naively I would never have expected clang-format to have any effect on my code. If you stopped and suggested header re-ordering to me my first thought would have been "I guess it must not do that. Maybe in practice header re-ordering doesn't actually matter with reasonable code?". The title here doesn't describe the contents of the article, but it describes a very important consequence of it that C++ programmers should be aware of. It gives a specific example of the consequence. I'm ok with that.

BeeOnRope 7y ago
I agree with most of that, although as the author I think it's still clickbait in the sense that: 1) clang-format is not really at fault here at all - the same effect could have happened by adding a new header, reordering the headers manually, switching to a system with a different libc version or a different transitive header include tree, etc. 2) The title doesn't tell you wtf is up. You have to get at least half way through the post to figure that out. It's in the vein of "This one food could wreck your health" posts. That said, other aspects of clickbait are advertising or monetization to profit from the clicks. Except for the crypto mining bot I load into your browser, and which subsequently writes itself to your MBR or UEFI partition, I have no ads, affiliate links, patreon-alikes or other monetization vectors.
- fizwhiz 7y ago
  > Except for the crypto mining bot I load into your browser, and which subsequently writes itself to your MBR or UEFI partition Guess I won't be clicking on the link now.
  - skrebbel 7y ago
    You're missing out. It's a great article, definitely worth all that monero.
- TehCorwiz 7y ago
  At least you’re forthcoming.

iameli 7y ago

I clicked because of the title and laughed when I read the reason, don't sweat about anyone accusing you of clickbait

jimbo1qaz 7y ago

I read the title and thought the article was about (clang-format's performance) on a (program simulating military tanks)... Apparently you meant "tanks" as a verb...

BeeOnRope 7y ago
Yes, although a quick search reveals that Oxford considers this a US usage so it was probably not the ideal choice for a title.

deleted 7y ago

[deleted]