9 ms·
Results of the Grand C++ Error Explosion Competition (2014)
- underdeserver 2y agoMultiplier: 5.9 billion That's... several orders of magnitude larger than I'd have guessed.
- ivanjermakov 2y agoI guess the limit is the stack size? Would that mean that output error size dependends on compiler flags?
- jepler 2y agoI don't know what compiler & flags they ran it on, but with gcc 12's preprocessor it looks much more than that: include files nest to a level of 200 before an error by default. with `-fmax-include-depth=10`, about 200kB of error is printed, and each subsequent level doubles that, so you'd be in for about 3e62 bytes of error output by my calculation... clang 14 terminates preprocessing as soon as any header exceeds maximum nesting, so only ~6kB of errors are produced by it.
- AlotOfReading 2y agoI like to increment the most significant digit of my terminal history size every time I run into a compiler error that exceeds the previous value. On Saturday, it went up from 40k to 50k. I laugh to hide my pain.
- codetrotter 2y agoWhen you get to 100k, will the next increment be to 200k then?
- AlotOfReading 2y agoI think the next increment from 100k would be to seriously consider a different language.
- bee_rider 2y agoYou must have a much more advanced relationship to errors than me. I don’t think I could tell the difference between 100k or 40k lines worth of errors, and it was only halfway through writing this message that I realized you might actually be talking about the size of an error? Oh no.
- Twisol 2y agoI suspect you'd notice simply because scrolling to the top of your history isn't sufficient anymore to find the first error. (Assuming one clears their shell history before compilation, which is very nearly required in this context...)
- lqet 2y agoI tutored a C++ beginners course on Linux a few times. I have had people ask me for help because the error messages were so long, their terminal history wasn't large enough to scroll back to the first error. Even if you have basic bash skills, just piping the compiler output to less or head also doesn't work, because the errors go to stderr. It's an incredible source of frustration for beginners.
- wglb 2y agoWouldn't it work if you did 2>&1 as part of the redirect?
- rocqua 2y agoYes, but if someone only just learned about less and piping, they probably won't know about fd 2 being stderr, fd based redirection, and hence don't understand it at all
- professoretc 2y agoFor my beginner programming class, I have them use a script `compile` which handles compilation and linking, while also enabling enabling all the warnings/errors I think are helpful, and also doing the above redirection. An absolute beginner doesn't gain anything from invoking `g++` directly.
- paulddraper 2y agoYes, but most beginners have never seen (or remember) that.
- wglb 2y agoKnowing about some of these shell bits would be useful for someone learning about C++.
- jprete 2y agoI know about that mechanism but I use it so infrequently that I have to look it up every time; the syntax doesn't really match anything else I know. It's also hard to look up because even the right search terms fetch a lot of nearby-concept results so I have to dig for the exact idea. (Probably this is an ideal LLM question though!)
- queuebert 2y agoPipe to command-line LLM: https://justine.lol/oneliners/ https://justine.lol/oneliners/
- renox 2y ago|& tee <log file> is your friend.
- gsliepen 2y ago10 years afterwards and GCC still produces the huge error messages. It seems Clang is now smart enough to stop compiling after too many errors.
- gmueckl 2y agoGCC also has (had?) an error limit. But because C++ compilers like to show you actual backtraces of how template instantiations led to the error condition, each single error message can easily have dozens of lines of context. That's how I got GCC to output 1MB of text for a single typo and the following parser confusion.
- hughw 2y agoI've mostly become adept at traversing six mile long compile error crawls to find the bit in my code causing the STL/Thrust complaint. clang is more legible for this than gcc. Still, I keep learning new patterns.
- WalterBright 2y agoThe D language compiler uses a technique I call "poisoning" which has greatly reduced cascading error messages. The idea is whenever an error is found in an AST node, the AST node is replaced with an "error" node. Any combination of an error node with another node is replaced with an error node. Error messages for error nodes are suppressed. It works far better than attempting to repair the AST into some plausible state. It's analogous to the propagation of NaN values in floating point code.
- jansvoboda11 2y agoDo you cut off this poisoning at any point?
- WalterBright 2y agoIf something else depends on the successful semantic analysis of those poisoned nodes, they get poisoned as well. It's all based on dependency.
- tester756 2y agoBut with this approach you arent able to provide e.g intellisense or other IDE hints for valid things within this node, right?
- WalterBright 2y agoA hint can be provided where the original error is diagnosed. I doubt hints on cascaded errors would be of any use.
- tester756 2y agoif you have function like "public intd Test() { ... typing new line }" then you will not provide hints when writing that new line due to "intd" being invalid type?
- 2y ago
- cgh 2y agoSome of the category descriptions are great, eg for "Most lifelike": Suppose you are given a task of adding some new functionality to an existing code base. You have been told that the guy who wrote it was “really smart” and that his code is of “enterprise quality”. You check out the code and open a random file in an editor. It appears on the screen. After just one microsecond of looking at the code you have lost your will to live and want nothing more than to beat your head against the table until you lose consciousness. This entry could be that code.
- deleted 2y ago[deleted]
- axra 2y agoThis is super interesting. I will do a 2024 edition.
- lqet 2y agoThe difference between an experienced C++ programmer and a C++ novice is that the novice regularly believes that his code is broken beyond repair for eternity and has accidentally destroyed the compiler and most of the hard drive, while the former just yawns at 5,000 lines of error messages and adds the missing semicolon.
- erikerikson 2y agoAs an experienced c++ programmer, I just write TypeScript without semicolons. /troll?
- deleted 2y ago[deleted]
- baq 2y agoJust 5000? Things must've gotten much better since I last touched C++
- marcosdumay 2y agoIt's just a semicolon. The GP still didn't fix enough to get to the templates and linker errors.
- secondcoming 2y agoFor me these days the most unintuitive error messages are due to a missing `const`
- gpderetta 2y agoIt's funny because it is true.
- jcalvinowens 2y agoGenerally speaking, clang produces much clearer C++ compile errors than gcc. I can't give any specific examples off the top of my head, but I've seen GCC emit hundreds of lines of inscrutable errors where clang spits out one line that tells you exactly what is wrong.
- eru 2y agoThe very first entry in the competition is a good example already. You can try it in both g++ and clang++. Interesting, for the 'Biggest error, category Bare Hands' entry, clang++ gives me a much longer error message than g++: $ g++ var.c++ |& wc -c 8 810948 5670500 $ clang++ var.c++ |& wc -c 20 5232857 36625300 Clang++ has about 6.5 times as many characters in the error message as g++ here. Though looking at the actual error messages, that seems to be down to a deeper default maximum instantiation depth on clang++ (1024 there vs 900 in g++).
- IWeldMelons 2y agogcc still pruduces a tiny bit faster binaries in most cases, like ~10-20%.
- spookie 2y agoAlso, Clang has had more consistent L2 cache misses in my testing (C++). Which is funny, Rust is more in line with GCC on that regard.
- xuser1024 2y ago[flagged]
- maxlin 2y agoWhat's the point in not showing the actual results? Is the expectation really to have a compiler handy just to browse the results
- omoikane 2y agoDid you mean the page not showing the generated error messages? The actual error messages would be many gigabytes worth of text, and aren't as interesting as the entries that produced those error messages.
- maxlin 2y agoThat's just a visualization problem. No we don't need to get a 1mm accuracy voxel map from the whole of Ukraine to know what's going on there either. Random samples / "this repeated X times" etc would work fine. Having nothing makes the whole thing seem pointless
- kccqzy 2y agoThe multiplier is in the billions. The actual results are gigabytes large. Your browser cannot handle that.
- klyrs 2y agoEven if your browser can handle it, tumblr doesn't wanna serve it.
- kevindamm 2y agoEven if Tumblr wanted to serve it, who would want to download it!? although it probably deflates really well.
- kibwen 2y agoWe can come up with compression schemes that do even better than Deflate. For example, here's the compressed version of the output of the first program, in just a handful of bytes: #include ".//.//.//.//jeh.cpp" #include "jeh.cpp"
- quotemstr 2y agoHas anyone tried these code snippets on modern compilers and noted how much our error messages have improved?
- eru 2y agoI just fed the first example into both g++ and clang++. Clang++ gives you a few lines of spam and one line of decent error message. G++ still barfs reams and reams of errors.
- kazinator 2y agoCreative Computing magazine in the 1970's had this kind of competition: most errors out of the smallest program, in any language. Back that day, it was possible because parsers tried to repair bad input to try to keep going, in hopes of diagnosing as many real errors as possible, so as to reduce the number of iterations. Iterations used to be expensive: punching corrections onto cards, etc. If the parser repairs bad syntax, it can cause more errors. If a repair involves insertion, there is a risk of getting into an infinite loop of diagnostics, even. It's kind of anachronistic to have people playing this with C++. Why is this 2014? Did that error explosion competition die out? Maybe tumblr isn't where you find C++ people.
- voidUpdate 2y agoSeems that it died out. There is a note from 2015: "However, we received only a very small number of entries. Because of this we have decided to cancel the competition. Due to this lack of interest we suspect that the competition will not be run again next year." https://tgceec.tumblr.com/post/110094856488/results-of-tgceec-2015 https://tgceec.tumblr.com/post/110094856488/results-of-tgcee...
- eru 2y agoI tried the first entry 'Biggest error, category Anything' and was disappointed when the error message was really short. But I had made the 'mistake' of using clang++, with g++ I get the crazy long error as promised.
- not2b 2y agoIt might be interesting to see how much expansion could be obtained with Rust. The diagnostics can be verbose, and perhaps there's some kind of self-reference that could produce a similar blowup, though I'm not familiar enough with the corners to say.