7 ms·

The author seems to be missing this essential text: "the implementor may augment the language by providing a definition of the officially undefined behavior."

The author seems to be missing this essential text: "the implementor may augment the language by providing a definition of the officially undefined behavior." Making a system call is undefined behavior in the C standard, but it's not undefined behavior in clang-on-FreeBSD, because the implementors of clang on FreeBSD have defined what those system calls do. Ditto for "asm" (UD unless/until you're running on a compiler which defines what that does), all of the tricks which make "malloc" work, and all of his other examples of acceptable uses of code which the C standard does not define.

bitwize 5y ago
The thing about UB is that it tends to happen when the C standard refuses to specify when a program segment is erroneous or valid. Some C environments treat memory as a large array of undifferentiated bytes or words, by design. Other C environments have tagged, bounds-checked regions of memory, again by design. (For example, the C compiler for the Lisp machine.) Usually, indirecting through a null pointer or walking off the end of an array are erroneous, but sometimes you want to read from memory location 0, or scan through all of available memory. The C standard allows for both kinds of environments by stating that these behaviors are undefined, allowing the implementation to error out or do something sensible, depending on the environment. The idea that UB is carte blanche for implementations to do whatever is an unintended consequence of the vague language of the standard. Maybe a future C standard should use "safe" and "unsafe" instead of UB for some of these operations, and clarify that unsafe code will be erroneous in a safe environment and do something sensible but potentially dangerous in an unsafe environment so you must really know what you're doing.
- deleted 5y ago
  [deleted]
- Rusky 5y ago
  > The idea that UB is carte blanche for implementations to do whatever is an unintended consequence of the vague language of the standard. Whether or not this was originally intended, it's certainly become the way the standard is written and used today, so that's kind of beside the point. Further, this is not some new idea that arose from the C standard. It's a basic, core idea in both software engineering and computer science! You define some meaning for your input, which may or may not cover all possible inputs, so that you can go on to process it without considering inputs that don't make sense. Now, to be fair, the "guardrail-free" approach where UB is silent is a bit out of the ordinary. A lot of software that makes assumptions about its input will at least try to validate them first, and a lot of programming language research will avoid UB by construction. But C is in a unique place where neither of those approaches fully work. > The C standard allows for both kinds of environments by stating that these behaviors are undefined, allowing the implementation to error out or do something sensible, depending on the environment. This is true, but it doesn't mean that "something sensible" is actually something the programmer should rely on! That's just asking too much of UB- programmers need to work with the semantics implemented by their toolchain, not make up an intuitive/"sensible" meaning for their undefined program and then get mad when it doesn't work. For example, if you want to scan through a bunch of memory, tell the language that's what you're doing. Is that memory at a fixed address? Tell the linker about it so it can show up as a normal global object in the program. Is it dynamic? Memory allocators fabricate new objects in the abstract machine all the time, perhaps your compiler supports an attribute that means "this function returns a pointer to a new object." The solution is not just to shrug and say "do something sensible but potentially dangerous." It's to precisely define the operations available to the programmer, and then provide tools to help them avoid misuse. If an operation isn't in the language, we can add it! If it's too easy to mess up, we can implement sanitizers and static analyzers, or provide alternatives! Yelling about a supposed misreading of "undefined behavior" is never going to be anywhere near as effective.
  - vyodaiken 5y ago
    One issue is that under the prevailing interpretation, the existing semantics is not reliable. You do not know when or if the compilers will take advantage of UB to completely change the semantics they are providing. That's not tenable.
    - Rusky 5y ago
      That's not how it works. Taking advantage of UB doesn't change the semantics, it just exposes which behaviors were never in the semantics to begin with. Barring compiler or spec bugs, we do in principle know exactly when the compiler may take advantage of UB. That's the point of a document like the standard- it describes the semantics in a precise way. To be fair, the existing semantics are certainly complex and often surprising, and people sometimes disagree over what they are, perhaps even to an untenable degree, but that's a very different thing from being unreliable.
      - vyodaiken 5y ago
        The net result of your argument is the language has no semantics. I write and test with -O0 and show that f(k)=m. Then I run with -O3 and f(k)=random. Am I required to be an expert on C Standard and compiler development in order to know that, with no warning, my code has always been wrong? What about if f(k)=m under Gcc 10, but now under Gcc 10.1 that whole section of code is skipped? What you are asking programmers to do is to both master the fine points of UB (which is impractical) and look into the future to see what changes may be invisibly committed to the compiler code.
        Rusky 5y ago
        > What you are asking programmers to do is to both master the fine points of UB (which is impractical) and look into the future to see what changes may be invisibly committed to the compiler code. I am asking programmers to understand and avoid UB, but I am not asking them to look into the future. Future compilers will still implement the same semantics- that's, again, the point of having a spec! I don't disagree that avoiding C's UB unaided can be difficult, but that just means the solution is to make it easier- and that's exactly what I suggested above: "precisely define the operations available to the programmer, and then provide tools to help them avoid misuse." And this isn't a new idea. People have been making progress in this area for a long time: better documentation of the rules, sanitizers, static analyzers, changes to the spec to remove some forms of UB, new languages that reshuffle things to make it harder or impossible to invoke UB, etc.
marcosdumay 5y ago
The C standards have the perfectly fine name "implementation dependent" to describe those things. Undefined behavior is much less constrained than implementation dependent, adn thus more problematic.
- tom_mellior 5y ago
  > The C standards have the perfectly fine name "implementation dependent" to describe those things. That term is not used by the C standards. Do you mean "implementation-defined"? asm is not among the explicitly specified implementation-defined behaviors, it's listed under "Common extensions". I don't see any mention at all of syscalls in C99. (I'm working with http://www.dragonwins.com/courses/ECE1021/STATIC/REFERENCES/n869.pdf http://www.dragonwins.com/courses/ECE1021/STATIC/REFERENCES/... here.)
  - formerly_proven 5y ago
    I'm not sure why syscalls would be UB; it's just not something defined by the C standard. Edit: To clarify, I meant UB in the sense it is typically used in these discussions, where the standard more-or-less explicitly says "If you do X, the behavior is undefined." Not in the literal sense of "ISO C does not say anything about write(2), hence using write(2) is undefined behavior according to the C standard", which seems like a rather tautological and useless statement to me.
    - hvdijk 5y ago
      What do you think UB is if not something where the behaviour is not defined?
      - tsimionescu 5y ago
        UB, in this context, is very explicitly used in the standard: it is undefined behavior related to a construct that the standard describes.
        5y ago
jcranmer 5y ago
Neither system calls nor the asm keyword are undefined behavior in the sense that C uses the term. They are, simply put, not covered by the standard at all. System calls--assuming you're referring to the C prototypes you call--work as normal external function definitions, just having semantics which are defined by the library (i.e., the kernel) and not the C specification itself. The asm keyword is a compiler language extension and is effectively implementation-defined (as C would call it), although compilers today tend to poorly document the actual semantics of their extensions.
Joker_vD 5y ago
> all of the tricks which make "malloc" work, What are those, exactly? AFAIK, you can safely track memory addresses by storing them as intptr_t/uintptr_t.
- jacquesm 5y ago
  Oh, for instance, on some implementations there is a lot of interesting stuff just prior to the allocated block returned. Not exactly the pinnacle of elegance but it gets the job done.
- cperciva 5y ago
  The C standard says very little about how those types work. In particular, you can cast a pointer to one of them and then cast back to a pointer -- but only if you cast the exact same value back, and the intptr values are not guaranteed to be in any way meaningful. In particular, casting a pointer to intptr_t, doing arithmetic on it, and casting back is not guaranteed to do anything useful. It almost certainly will, since most systems treat it as roughly the same as casting to char *, but the standard does not guarantee it.
  - magnostherobot 5y ago
    Do you have an example of a situation in which you'd want to cast the result of arithmetic intptr_t values to a pointer? The situations I can think of off the top of my head would be better done as arithmetic between pointers.
    - bonzini 5y ago
      Arithmetic on pointers in turn is only defined if the pointers point within the same object (or right past the end of that object). One example of using intptr_t would be going from a pointer passed to free() to a metadata block for the memory that must be freed.
  - Joker_vD 5y ago
    > and casting back is not guaranteed to do anything useful I believe it's implementation-specific, precisely so that malloc/free can be implemented in conformant C. The only tricky part is how you can "bless" parts of large memory block originally pointed to by void* (returned from mmap(), for example) to safely become ints and char[]s and structs...

bitwize 5y ago

The thing about UB is that it tends to happen when the C standard refuses to specify when a program segment is erroneous or valid. Some C environments treat memory as a large array of undifferentiated bytes or words, by design. Other C environments have tagged, bounds-checked regions of memory, again by design. (For example, the C compiler for the Lisp machine.) Usually, indirecting through a null pointer or walking off the end of an array are erroneous, but sometimes you want to read from memory location 0, or scan through all of available memory. The C standard allows for both kinds of environments by stating that these behaviors are undefined, allowing the implementation to error out or do something sensible, depending on the environment. The idea that UB is carte blanche for implementations to do whatever is an unintended consequence of the vague language of the standard. Maybe a future C standard should use "safe" and "unsafe" instead of UB for some of these operations, and clarify that unsafe code will be erroneous in a safe environment and do something sensible but potentially dangerous in an unsafe environment so you must really know what you're doing.

deleted 5y ago
[deleted]
Rusky 5y ago
> The idea that UB is carte blanche for implementations to do whatever is an unintended consequence of the vague language of the standard. Whether or not this was originally intended, it's certainly become the way the standard is written and used today, so that's kind of beside the point. Further, this is not some new idea that arose from the C standard. It's a basic, core idea in both software engineering and computer science! You define some meaning for your input, which may or may not cover all possible inputs, so that you can go on to process it without considering inputs that don't make sense. Now, to be fair, the "guardrail-free" approach where UB is silent is a bit out of the ordinary. A lot of software that makes assumptions about its input will at least try to validate them first, and a lot of programming language research will avoid UB by construction. But C is in a unique place where neither of those approaches fully work. > The C standard allows for both kinds of environments by stating that these behaviors are undefined, allowing the implementation to error out or do something sensible, depending on the environment. This is true, but it doesn't mean that "something sensible" is actually something the programmer should rely on! That's just asking too much of UB- programmers need to work with the semantics implemented by their toolchain, not make up an intuitive/"sensible" meaning for their undefined program and then get mad when it doesn't work. For example, if you want to scan through a bunch of memory, tell the language that's what you're doing. Is that memory at a fixed address? Tell the linker about it so it can show up as a normal global object in the program. Is it dynamic? Memory allocators fabricate new objects in the abstract machine all the time, perhaps your compiler supports an attribute that means "this function returns a pointer to a new object." The solution is not just to shrug and say "do something sensible but potentially dangerous." It's to precisely define the operations available to the programmer, and then provide tools to help them avoid misuse. If an operation isn't in the language, we can add it! If it's too easy to mess up, we can implement sanitizers and static analyzers, or provide alternatives! Yelling about a supposed misreading of "undefined behavior" is never going to be anywhere near as effective.
- vyodaiken 5y ago
  One issue is that under the prevailing interpretation, the existing semantics is not reliable. You do not know when or if the compilers will take advantage of UB to completely change the semantics they are providing. That's not tenable.
  - Rusky 5y ago
    That's not how it works. Taking advantage of UB doesn't change the semantics, it just exposes which behaviors were never in the semantics to begin with. Barring compiler or spec bugs, we do in principle know exactly when the compiler may take advantage of UB. That's the point of a document like the standard- it describes the semantics in a precise way. To be fair, the existing semantics are certainly complex and often surprising, and people sometimes disagree over what they are, perhaps even to an untenable degree, but that's a very different thing from being unreliable.
    - vyodaiken 5y ago
      The net result of your argument is the language has no semantics. I write and test with -O0 and show that f(k)=m. Then I run with -O3 and f(k)=random. Am I required to be an expert on C Standard and compiler development in order to know that, with no warning, my code has always been wrong? What about if f(k)=m under Gcc 10, but now under Gcc 10.1 that whole section of code is skipped? What you are asking programmers to do is to both master the fine points of UB (which is impractical) and look into the future to see what changes may be invisibly committed to the compiler code.
      - Rusky 5y ago
        > What you are asking programmers to do is to both master the fine points of UB (which is impractical) and look into the future to see what changes may be invisibly committed to the compiler code. I am asking programmers to understand and avoid UB, but I am not asking them to look into the future. Future compilers will still implement the same semantics- that's, again, the point of having a spec! I don't disagree that avoiding C's UB unaided can be difficult, but that just means the solution is to make it easier- and that's exactly what I suggested above: "precisely define the operations available to the programmer, and then provide tools to help them avoid misuse." And this isn't a new idea. People have been making progress in this area for a long time: better documentation of the rules, sanitizers, static analyzers, changes to the spec to remove some forms of UB, new languages that reshuffle things to make it harder or impossible to invoke UB, etc.

marcosdumay 5y ago

The C standards have the perfectly fine name "implementation dependent" to describe those things. Undefined behavior is much less constrained than implementation dependent, adn thus more problematic.

tom_mellior 5y ago
> The C standards have the perfectly fine name "implementation dependent" to describe those things. That term is not used by the C standards. Do you mean "implementation-defined"? asm is not among the explicitly specified implementation-defined behaviors, it's listed under "Common extensions". I don't see any mention at all of syscalls in C99. (I'm working with http://www.dragonwins.com/courses/ECE1021/STATIC/REFERENCES/n869.pdf http://www.dragonwins.com/courses/ECE1021/STATIC/REFERENCES/... here.)
- formerly_proven 5y ago
  I'm not sure why syscalls would be UB; it's just not something defined by the C standard. Edit: To clarify, I meant UB in the sense it is typically used in these discussions, where the standard more-or-less explicitly says "If you do X, the behavior is undefined." Not in the literal sense of "ISO C does not say anything about write(2), hence using write(2) is undefined behavior according to the C standard", which seems like a rather tautological and useless statement to me.
  - hvdijk 5y ago
    What do you think UB is if not something where the behaviour is not defined?
    - tsimionescu 5y ago
      UB, in this context, is very explicitly used in the standard: it is undefined behavior related to a construct that the standard describes.
      - 5y ago

jcranmer 5y ago

Neither system calls nor the asm keyword are undefined behavior in the sense that C uses the term. They are, simply put, not covered by the standard at all. System calls--assuming you're referring to the C prototypes you call--work as normal external function definitions, just having semantics which are defined by the library (i.e., the kernel) and not the C specification itself. The asm keyword is a compiler language extension and is effectively implementation-defined (as C would call it), although compilers today tend to poorly document the actual semantics of their extensions.

Joker_vD 5y ago

> all of the tricks which make "malloc" work, What are those, exactly? AFAIK, you can safely track memory addresses by storing them as intptr_t/uintptr_t.

jacquesm 5y ago
Oh, for instance, on some implementations there is a lot of interesting stuff just prior to the allocated block returned. Not exactly the pinnacle of elegance but it gets the job done.
cperciva 5y ago
The C standard says very little about how those types work. In particular, you can cast a pointer to one of them and then cast back to a pointer -- but only if you cast the exact same value back, and the intptr values are not guaranteed to be in any way meaningful. In particular, casting a pointer to intptr_t, doing arithmetic on it, and casting back is not guaranteed to do anything useful. It almost certainly will, since most systems treat it as roughly the same as casting to char *, but the standard does not guarantee it.
- magnostherobot 5y ago
  Do you have an example of a situation in which you'd want to cast the result of arithmetic intptr_t values to a pointer? The situations I can think of off the top of my head would be better done as arithmetic between pointers.
  - bonzini 5y ago
    Arithmetic on pointers in turn is only defined if the pointers point within the same object (or right past the end of that object). One example of using intptr_t would be going from a pointer passed to free() to a metadata block for the memory that must be freed.
- Joker_vD 5y ago
  > and casting back is not guaranteed to do anything useful I believe it's implementation-specific, precisely so that malloc/free can be implemented in conformant C. The only tricky part is how you can "bless" parts of large memory block originally pointed to by void* (returned from mmap(), for example) to safely become ints and char[]s and structs...