6 ms·

No, UB has always been just undefined. `gets()`, one of the most exploited functions in the history of computers, does not require any clever malice from the co

No, UB has always been just undefined. `gets()`, one of the most exploited functions in the history of computers, does not require any clever malice from the compiler. And there are a lot of such examples. In fact practice has shown that it has always been very, very hard to write a program that cannot reach UB. Thinking that you know what `undefined` really is results in undefined behavior.

Xeamek 2y ago
It's not like UB is some magic functions that gets called when encountering UB. Every UB could verry well be wrapped in error checking and we could have 0 UB out there. We don't do that, but that's because of (our) choice, not some superficial entity that poses machine when running into UB.
- gpderetta 2y ago
  > Every UB could very well be wrapped in error checking Not in C and C++. How would you check that a pointer is safe to dereference for example?
  - Xeamek 2y ago
    What do you mean? Just check if it's not null?
    - gpderetta 2y ago
      Not all non-null pointers are safe to dereference.
      - Xeamek 2y ago
        Like?
        flohofwoe 2y ago
        On the hardware and assembly level, all unmapped memory regions are illegal to access and will cause a segfault. In high level languages it gets a lot more complicated (look up "pointer provenance").
        planede 2y ago
        Like one that is free'd or point to a stack object that is no longer in scope. Or one that points one past the last element of an array.
        Xeamek 2y ago
        It's unsafe to do that, but it's not UB
        flohofwoe 2y ago
        I don't have all ca 200 UBs in my head (see https://gist.github.com/Earnestly/7c903f481ff9d29a3dd1 https://gist.github.com/Earnestly/7c903f481ff9d29a3dd1), but I'm pretty sure all types of illegal memory accesses count as UB. For instance from skimming the list: - An object is referred to outside of its lifetime (6.2.4). - The value of a pointer to an object whose lifetime has ended is used (6.2.4). - An array subscript is out of range, even if an object is apparently accessible with the given subscript (as in the lvalue expression a[1][7] given the declaration int a[4][5]) (6.5.6).
        planede 2y ago
        Technically the array subscript example is UB even before the dereference. a[1][7] is equivalent to `*(a[1] + 7)` and a[1] + 7 itself is UB. (6.5.7) Additive operators > If the pointer operand and the result do not point to elements of the same array object or one past the last element of the array object, the behavior is undefined. Interestingly a[1][5] is also UB, but for the dereferencing a past-the-end pointer, not for the arithmetic.
        gpderetta 2y ago
- mpweiher 2y ago
  This turns out not to be the case. Compilers have been known to remove exactly that sort of error checking code, as there was undefined behavior present. It's a mess.
  - Xeamek 2y ago
    Yes, I know it's not possible for a specific c++ dev to do such checks, as compilers can optimize them away. What I meant is “we (as in) the c/c++ community (or rather the specification committee)” could tomorrow decide that, lets say, all null pointer dereferences should lead to program exit. Existence of UB is a choice.
    - bigcheesegs 2y ago
      UB is a choice in that we could require symbolic execution of of all code. This is incredibly slow, so we don't. There are many cases of UB that would be cheap to check, but there are many more that are incredibly expensive to check.
flohofwoe 2y ago
Just because 'undefined behaviour' means 'theoretically anything can happen' doesn't mean that it will happen in practice. You also don't need to summon the UB ghosts to delete all your files, a regular logic bug works just as well, even in a language without any UB. In older compilers, UB behaviour was a lot more predictable because the code transformations done by optimizing compilers were by far not as complex as today. In the C standard, 'undefined behaviour' is basically a catch-all phrase for a lot of different things: from badly designed stdlib function that miss obvious security checks (like gets()) to the compiler generating non-sensical code. The latter is what's much more dangerous. It's trivial to just not use the C stdlib functions, but code generation problems caused by UB are much harder to catch.
- H8crilA 2y ago
  No, it's the former that's more dangerous, and it has been there from the beginning. Every time you dereference a bad pointer (already free'd, corrupted, badly casted, ...), every time you go over the bounds of some array, every time you read memory that hasn't been initialized, every time you (write) access the same memory from two threads without synchronization, etc. - this is the bread and butter of binary exploitation. Exploits are fully expected under C/C++ spec, after all the behavior is undefined.
  - flohofwoe 2y ago
    But at the same time it is totally expected in C and C++ that range checking is the responsibility of the programmer, and most of those problems are trivial to catch with Valgrind or ASAN. E.g. yes, it is a problem, yes, 70% of all CVEs are caused by memory corruption, but the problem is totally obvious and trivial to debug. It's basically part of the contract when deciding to use C or C++. Compared to such 'obvious' memory corruption problems caused by invalid pointers or missing range checks, obscure code generation issues caused by UB are much harder to identify and the behaviour may also change randomly between compiler versions - this is the actually scary thing about UB, and it's also a relatively "new" thing.
    - 2y ago
mpweiher 2y ago
> No, UB has always been just undefined This turns out not to be the case. There actually is text in the current standard as to what acceptable behavior is when encountering undefined behavior. This was turned into a "note" in later versions of the standard, but initially it was just as much a normative part of the standard as everything else. The text is still there. Compiler writers want to exploit UB, hence they say that this text effectively doesn't exist. https://blog.metaobject.com/2018/07/a-one-word-change-to-c-standard-to-make.html https://blog.metaobject.com/2018/07/a-one-word-change-to-c-s...
- H8crilA 2y ago
  What's the behavior of this piece of code according to the spec? You can choose any C/C++ spec: void f() { char x[100]; gets(x); }
  - gpderetta 2y ago
    The key here is the meaning of "ignoring" in "ignoring the situation completely with unpredictable results". Most programmers (and pretty much all compiler writers) interpret it as meaning that the compiler can assume it can't happen (ignore the possibility) and translate accordingly. Some, like mpweiher interpret it as meaning that the compiler must implement a trivial (but somehow unspecified) translation to the underlying hardware and let the behavior be what the hardware provides. edit: We went through this many many times.
    - tialaramex 2y ago
      This craziness (a belief that if you don't specify what something means it must have some "obvious" behaviour that is coincidentally always whatever the programmer wanted) is endemic in C and C++ communities. Right now, notionally serious people are making the case that C++ should enshrine the "bag-of-bits" model of pointers which is exactly this sort of stupidity. At first it looks like they want pointers are addresses which is kinda dumb but whatever, and you keep reading and they're like aha, also I should be able to dereference my bag-of-bits and have that work too... Full blown Provenance Via Integers, expect to spend the next forty years trying to specify how any of the optimisations you rely on can survive the resulting tangle, say hi to anybody who wanted Consume ordering and is still waiting.

Xeamek 2y ago

It's not like UB is some magic functions that gets called when encountering UB. Every UB could verry well be wrapped in error checking and we could have 0 UB out there. We don't do that, but that's because of (our) choice, not some superficial entity that poses machine when running into UB.

gpderetta 2y ago
> Every UB could very well be wrapped in error checking Not in C and C++. How would you check that a pointer is safe to dereference for example?
- Xeamek 2y ago
  What do you mean? Just check if it's not null?
  - gpderetta 2y ago
    Not all non-null pointers are safe to dereference.
    - Xeamek 2y ago
      Like?
      - flohofwoe 2y ago
        On the hardware and assembly level, all unmapped memory regions are illegal to access and will cause a segfault. In high level languages it gets a lot more complicated (look up "pointer provenance").
      - planede 2y ago
        Like one that is free'd or point to a stack object that is no longer in scope. Or one that points one past the last element of an array.
        Xeamek 2y ago
        It's unsafe to do that, but it's not UB
        flohofwoe 2y ago
        I don't have all ca 200 UBs in my head (see https://gist.github.com/Earnestly/7c903f481ff9d29a3dd1 https://gist.github.com/Earnestly/7c903f481ff9d29a3dd1), but I'm pretty sure all types of illegal memory accesses count as UB. For instance from skimming the list: - An object is referred to outside of its lifetime (6.2.4). - The value of a pointer to an object whose lifetime has ended is used (6.2.4). - An array subscript is out of range, even if an object is apparently accessible with the given subscript (as in the lvalue expression a[1][7] given the declaration int a[4][5]) (6.5.6).
        planede 2y ago
        Technically the array subscript example is UB even before the dereference. a[1][7] is equivalent to `*(a[1] + 7)` and a[1] + 7 itself is UB. (6.5.7) Additive operators > If the pointer operand and the result do not point to elements of the same array object or one past the last element of the array object, the behavior is undefined. Interestingly a[1][5] is also UB, but for the dereferencing a past-the-end pointer, not for the arithmetic.
        gpderetta 2y ago
mpweiher 2y ago
This turns out not to be the case. Compilers have been known to remove exactly that sort of error checking code, as there was undefined behavior present. It's a mess.
- Xeamek 2y ago
  Yes, I know it's not possible for a specific c++ dev to do such checks, as compilers can optimize them away. What I meant is “we (as in) the c/c++ community (or rather the specification committee)” could tomorrow decide that, lets say, all null pointer dereferences should lead to program exit. Existence of UB is a choice.
  - bigcheesegs 2y ago
    UB is a choice in that we could require symbolic execution of of all code. This is incredibly slow, so we don't. There are many cases of UB that would be cheap to check, but there are many more that are incredibly expensive to check.

flohofwoe 2y ago

Just because 'undefined behaviour' means 'theoretically anything can happen' doesn't mean that it will happen in practice. You also don't need to summon the UB ghosts to delete all your files, a regular logic bug works just as well, even in a language without any UB. In older compilers, UB behaviour was a lot more predictable because the code transformations done by optimizing compilers were by far not as complex as today. In the C standard, 'undefined behaviour' is basically a catch-all phrase for a lot of different things: from badly designed stdlib function that miss obvious security checks (like gets()) to the compiler generating non-sensical code. The latter is what's much more dangerous. It's trivial to just not use the C stdlib functions, but code generation problems caused by UB are much harder to catch.

H8crilA 2y ago
No, it's the former that's more dangerous, and it has been there from the beginning. Every time you dereference a bad pointer (already free'd, corrupted, badly casted, ...), every time you go over the bounds of some array, every time you read memory that hasn't been initialized, every time you (write) access the same memory from two threads without synchronization, etc. - this is the bread and butter of binary exploitation. Exploits are fully expected under C/C++ spec, after all the behavior is undefined.
- flohofwoe 2y ago
  But at the same time it is totally expected in C and C++ that range checking is the responsibility of the programmer, and most of those problems are trivial to catch with Valgrind or ASAN. E.g. yes, it is a problem, yes, 70% of all CVEs are caused by memory corruption, but the problem is totally obvious and trivial to debug. It's basically part of the contract when deciding to use C or C++. Compared to such 'obvious' memory corruption problems caused by invalid pointers or missing range checks, obscure code generation issues caused by UB are much harder to identify and the behaviour may also change randomly between compiler versions - this is the actually scary thing about UB, and it's also a relatively "new" thing.
  - 2y ago

mpweiher 2y ago

> No, UB has always been just undefined This turns out not to be the case. There actually is text in the current standard as to what acceptable behavior is when encountering undefined behavior. This was turned into a "note" in later versions of the standard, but initially it was just as much a normative part of the standard as everything else. The text is still there. Compiler writers want to exploit UB, hence they say that this text effectively doesn't exist. https://blog.metaobject.com/2018/07/a-one-word-change-to-c-standard-to-make.html https://blog.metaobject.com/2018/07/a-one-word-change-to-c-s...

H8crilA 2y ago
What's the behavior of this piece of code according to the spec? You can choose any C/C++ spec: void f() { char x[100]; gets(x); }
- gpderetta 2y ago
  The key here is the meaning of "ignoring" in "ignoring the situation completely with unpredictable results". Most programmers (and pretty much all compiler writers) interpret it as meaning that the compiler can assume it can't happen (ignore the possibility) and translate accordingly. Some, like mpweiher interpret it as meaning that the compiler must implement a trivial (but somehow unspecified) translation to the underlying hardware and let the behavior be what the hardware provides. edit: We went through this many many times.
  - tialaramex 2y ago
    This craziness (a belief that if you don't specify what something means it must have some "obvious" behaviour that is coincidentally always whatever the programmer wanted) is endemic in C and C++ communities. Right now, notionally serious people are making the case that C++ should enshrine the "bag-of-bits" model of pointers which is exactly this sort of stupidity. At first it looks like they want pointers are addresses which is kinda dumb but whatever, and you keep reading and they're like aha, also I should be able to dereference my bag-of-bits and have that work too... Full blown Provenance Via Integers, expect to spend the next forty years trying to specify how any of the optimisations you rely on can survive the resulting tangle, say hi to anybody who wanted Consume ordering and is still waiting.