8 ms·
The most terrifying concept is still the good old undefined behavior. How is it possible that a simple `gets()` call can pop a reverse shell and empty your bank
by H8crilA 2y ago
The most terrifying concept is still the good old undefined behavior. How is it possible that a simple `gets()` call can pop a reverse shell and empty your bank account? Or steal national secrets and potentially cause a war? Undefined truly means undefined, though few people realize the scope at first when they read those innocent sounding words:
The behavior is undefined
- flohofwoe 2y agoThe UB mess only started when compiler writers decided it is ok to exploit UB for optimisations which then leads to wildly unpredictable behaviour when you actually hit UB. But also: turn warnings to 11, use UBSAN, ASAN, TSAN and static analysers. These are all easy steps to make C and C++ code a lot more robust.
- H8crilA 2y agoNo, UB has always been just undefined. `gets()`, one of the most exploited functions in the history of computers, does not require any clever malice from the compiler. And there are a lot of such examples. In fact practice has shown that it has always been very, very hard to write a program that cannot reach UB. Thinking that you know what `undefined` really is results in undefined behavior.
- Xeamek 2y agoIt's not like UB is some magic functions that gets called when encountering UB. Every UB could verry well be wrapped in error checking and we could have 0 UB out there. We don't do that, but that's because of (our) choice, not some superficial entity that poses machine when running into UB.
- gpderetta 2y ago> Every UB could very well be wrapped in error checking Not in C and C++. How would you check that a pointer is safe to dereference for example?
- Xeamek 2y agoWhat do you mean? Just check if it's not null?
- gpderetta 2y agoNot all non-null pointers are safe to dereference.
- Xeamek 2y agoLike?
- flohofwoe 2y agoOn the hardware and assembly level, all unmapped memory regions are illegal to access and will cause a segfault. In high level languages it gets a lot more complicated (look up "pointer provenance").
- planede 2y agoLike one that is free'd or point to a stack object that is no longer in scope. Or one that points one past the last element of an array.
- Xeamek 2y agoIt's unsafe to do that, but it's not UB
- flohofwoe 2y agoI don't have all ca 200 UBs in my head (see https://gist.github.com/Earnestly/7c903f481ff9d29a3dd1 https://gist.github.com/Earnestly/7c903f481ff9d29a3dd1), but I'm pretty sure all types of illegal memory accesses count as UB. For instance from skimming the list: - An object is referred to outside of its lifetime (6.2.4). - The value of a pointer to an object whose lifetime has ended is used (6.2.4). - An array subscript is out of range, even if an object is apparently accessible with the given subscript (as in the lvalue expression a[1][7] given the declaration int a[4][5]) (6.5.6).
- mpweiher 2y agoThis turns out not to be the case. Compilers have been known to remove exactly that sort of error checking code, as there was undefined behavior present. It's a mess.
- Xeamek 2y agoYes, I know it's not possible for a specific c++ dev to do such checks, as compilers can optimize them away. What I meant is “we (as in) the c/c++ community (or rather the specification committee)” could tomorrow decide that, lets say, all null pointer dereferences should lead to program exit. Existence of UB is a choice.
- bigcheesegs 2y agoUB is a choice in that we could require symbolic execution of of all code. This is incredibly slow, so we don't. There are many cases of UB that would be cheap to check, but there are many more that are incredibly expensive to check.
- flohofwoe 2y agoJust because 'undefined behaviour' means 'theoretically anything can happen' doesn't mean that it will happen in practice. You also don't need to summon the UB ghosts to delete all your files, a regular logic bug works just as well, even in a language without any UB. In older compilers, UB behaviour was a lot more predictable because the code transformations done by optimizing compilers were by far not as complex as today. In the C standard, 'undefined behaviour' is basically a catch-all phrase for a lot of different things: from badly designed stdlib function that miss obvious security checks (like gets()) to the compiler generating non-sensical code. The latter is what's much more dangerous. It's trivial to just not use the C stdlib functions, but code generation problems caused by UB are much harder to catch.
- H8crilA 2y agoNo, it's the former that's more dangerous, and it has been there from the beginning. Every time you dereference a bad pointer (already free'd, corrupted, badly casted, ...), every time you go over the bounds of some array, every time you read memory that hasn't been initialized, every time you (write) access the same memory from two threads without synchronization, etc. - this is the bread and butter of binary exploitation. Exploits are fully expected under C/C++ spec, after all the behavior is undefined.
- flohofwoe 2y agoBut at the same time it is totally expected in C and C++ that range checking is the responsibility of the programmer, and most of those problems are trivial to catch with Valgrind or ASAN. E.g. yes, it is a problem, yes, 70% of all CVEs are caused by memory corruption, but the problem is totally obvious and trivial to debug. It's basically part of the contract when deciding to use C or C++. Compared to such 'obvious' memory corruption problems caused by invalid pointers or missing range checks, obscure code generation issues caused by UB are much harder to identify and the behaviour may also change randomly between compiler versions - this is the actually scary thing about UB, and it's also a relatively "new" thing.
- 2y ago
- mpweiher 2y ago> No, UB has always been just undefined This turns out not to be the case. There actually is text in the current standard as to what acceptable behavior is when encountering undefined behavior. This was turned into a "note" in later versions of the standard, but initially it was just as much a normative part of the standard as everything else. The text is still there. Compiler writers want to exploit UB, hence they say that this text effectively doesn't exist. https://blog.metaobject.com/2018/07/a-one-word-change-to-c-standard-to-make.html https://blog.metaobject.com/2018/07/a-one-word-change-to-c-s...
- H8crilA 2y agoWhat's the behavior of this piece of code according to the spec? You can choose any C/C++ spec: void f() { char x[100]; gets(x); }
- gpderetta 2y agoThe key here is the meaning of "ignoring" in "ignoring the situation completely with unpredictable results". Most programmers (and pretty much all compiler writers) interpret it as meaning that the compiler can assume it can't happen (ignore the possibility) and translate accordingly. Some, like mpweiher interpret it as meaning that the compiler must implement a trivial (but somehow unspecified) translation to the underlying hardware and let the behavior be what the hardware provides. edit: We went through this many many times.
- tialaramex 2y agoThis craziness (a belief that if you don't specify what something means it must have some "obvious" behaviour that is coincidentally always whatever the programmer wanted) is endemic in C and C++ communities. Right now, notionally serious people are making the case that C++ should enshrine the "bag-of-bits" model of pointers which is exactly this sort of stupidity. At first it looks like they want pointers are addresses which is kinda dumb but whatever, and you keep reading and they're like aha, also I should be able to dereference my bag-of-bits and have that work too... Full blown Provenance Via Integers, expect to spend the next forty years trying to specify how any of the optimisations you rely on can survive the resulting tangle, say hi to anybody who wanted Consume ordering and is still waiting.
- wolletd 2y agoWhat I don't get is, why are we still adding more undefined behaviour to the standard? For example, C++23 got std::unreachable. From my dips into Rust, I expected something similar to std::abort, terminating the program in some sane and possibly helpful way. However, in C++, std::unreachable just invokes undefined behaviour when called. It's not usable as a guard from programming errors, it's just an optimization hint. You still have to write the guard yourself. I'm left wondering about the use-cases for this.
- bun_terminator 2y agoIt's faster and/or easier to implement. Sometimes a lot
- H8crilA 2y agoAnd if you ever reach it then maybe the program will crash, or maybe demons will start flying out of your nose. No one really knows, and that's what makes undefined behavior exciting!
- flohofwoe 2y agostd::unreachable is actually a quite useful optimisation tool. For instance if you are sure that a switch-case default branch is actually unreachable you can put a std::unreachable into the default-branch to hint the compiler that it may remove a range check in front of the switch-case jump table access (since you promised to the compiler that the default branch is never reached). It's a double edged sword of course. If control flow actually hits the default branch, then all bets are off (because that means the code will access an out-of-range jump table slot and jump somewhere into the wilderness). AFAIK compilers are free to perform something like Rust's panic when std::unreachable is actually hit, but that makes only sense in debug mode. Because in the above example, the compiler would need to add a range check to figure out if panic needs to be called and that would completely defeat the idea of removing that range check in the first place.
- planede 2y agoI recommend omitting the default case and putting std::unreachable() outside the switch and omitting the default label for the sole reason that compilers are more likely to warn for a missed case label this way (-Wswitch vs -Wswitch-enum in gcc/clang, the former is included in -Wall, the latter isn't included even in -Wextra). This also allows expressing intent: no default label means that I meant to handle all cases, and having a default means that I opted into a fallback, please don't warn. That's probably why -Wswitch-enum isn't enabled by default, too many false positives without a convenient way to suppress the warning.
- deleted 2y ago[deleted]
- tialaramex 2y agoForget gets() - that's obviously a terrible idea, and most people see immediately why, it is no longer in modern C for example. Consider that abs() of signed integers has UB! Because the signed integers aren't balanced it's tricky to decide what should happen if we abs() the most negative value of a particular size of signed integer. In a language like Rust you get to specify what you meant here, if you don't then you get a panic when it happens. But C++ instead gives this... Undefined Behaviour! Or take division by zero. It's easy enough to say that you think the signed integers should have wrapping arithmetic. Doesn't make much sense, but hey, it's defined at least. However, even if you define overflow as wrapping, division by zero is a different problem. In Rust they can explicitly say this can't happen using NonZero types (e.g. NonZeroI16 is a 16-bit signed integer, but it can't be zero) and get the high performance machine code, but in C++ to get that same code they just say division by zero is Undefined Behaviour.
- nneonneo 2y agogets() might be (mostly) gone, but scanf(“%s”, buf) still exists and produces no warnings or errors from the compiler.
- planede 2y agoHow does arithmetic work with NonZeroI16 and remain efficient? What's the codegen of adding/subtracting two NonZeroI16 values?
- tialaramex 2y agoWhether we can efficiently do other arithmetic depends on what we're doing, but that's not necessarily on our critical path even if the division is.
- Measter 2y agoThey don't support general arithmetic. They do have member functions for abs, multiplication, negation and power operations, but they are all checked/saturating if they can result in 0. If you want to do actual arithmetic, you have to get the internal i16, perform the arithmetic, then rewrap it. https://godbolt.org/z/T5e36zzM5 https://godbolt.org/z/T5e36zzM5
- qiqitori 2y agoI don't think the standard (C99) uses the term "undefined" in its description of the gets() function. (Also gets has been removed from the standard in C11.) But maybe it uses the term to describe what happens when you change memory that shouldn't be changed. Well, what do you think the C standard should say in such a case, without knowledge of the system you're running on? There are of course some functions where it would have been nice to allow passing in an invalid value and the function just returning its normal error or "no" return value. E.g. isalnum might be nicer if it didn't crash for input > 255. (But perhaps it's not possible in all implementations without adding some other constraints that didn't exist before.)
- chipdart 2y ago> The most terrifying concept is still the good old undefined behavior. I think people tend to parrot undefined behavior as if it's some kind of gotcha when in practice this only means two things: * Don't write code that you didn't bothered to understand what it does, * You bothered to check what your implementation does and decided to write implementation-specific code for your own personal reasons. By definition, each and every single programming language in existence which is not specified in an official reference document is comprised exclusively of undefined behavior. Why? Because none of it's expected behavior is defined. That's what these nonsensical discussions boil down to. So why is this somehow expected to be such an insurmountable gotcha with C++ although the rest of the world uses languages which are entirely undefined without any concern? Boggles the mind.
- rstuart4133 2y ago> I think people tend to parrot undefined behavior as if it's some kind of gotcha when in practice this only means two things: No, not really. In fact an excellent counter example can be found on this very iceberg: https://blog.regehr.org/archives/161 https://blog.regehr.org/archives/161 C++ at the some optimisation levels says the example program disproved Fermat's Last Theorem, which should come as a surprise as their are no known counter examples. The program is also valid C. When compiled with good C compilers at all optimisation levels (both gnu and clang count as good), the program never exits because it doesn't find a counter example. But the gnu and clang c++ compilers do behave as described. The difference is caused by what the C and C++ language standards consider to be UB. Why? C++ (but not C) defines an infinite loop as undefined behaviour, and at high enough optimisation levels the compiler spots the infinite loop (which is the correct behaviour if the theorem is true) and goes with the other option. (As an aside, it's impressive it does detect this particular infinite loop.) Why is that a problem? Well lots of programs are deliberate infinite loops. In practice many don't qualify as UB because of various conditions listed in the article, but you would have to be a language lawyer to know them. The author gives real life examples of how he was caught by the surprise removal of his loop. Deliberate infinite loops are actually fairly common and are perfect correct code. Proving they are infinite is famously undecidable, so you can usually get away with it safe in the knowledge the compiler won't spot it. But compilers keep chipping away at the edge cases so one day you can find your program changes its behaviour drastically just because it was compiled with a new version of the compiler, or even just different optimisation options.