11 ms·

Re: C as used/implemented in practice

deleted 11y ago
[deleted]
nathanb 11y ago
I have difficulty accepting "let's replace C with X", where X is a memory-managed language. As a systems programmer (I write SCSI driver code in C), I can't overemphasize how important it is to be able to address memory as a flat range of bytes, regardless of how that memory was originally handed to me. I need to have uint8_t* pointers into the middle of buffers which I can then typecast into byte-aligned structs. If your memory manager would not allow this or would move this memory around, that's a non-starter. I don't stick with C because I love it. If I'm writing something for my own purposes, I use Ruby. I've written some server code in Golang (non-production), and it's pretty nifty, even if the way it twists normal C syntax breaks my brain. I even dabble in the dark side (C++) personally and professionally from time to time. And in a previous life, I was reasonably proficient in C# (that's the CLR 2.0 timeframe; I'm completely useless at it in these crazy days of LINQ and the really nifty CLR 4 features...and there's probably even more stuff I haven't even become aware of). But none of those languages would let me do what I need to do: zero-copy writes from the network driver through to the RAID backend. And even if they did, the pain of rewriting our entire operating system in Go or Rust or whatever would be way more than the alleviated pain of using a "nicer" language. (We never use 'int', by the way. We use the C99 well-defined types in stdint.h. Could this value go greater than a uint32_t can represent? Make it a uint64_t. Does it need to be signed? No? Make sure it's unsigned. A lot of what he's complaining about is sloppy code. I don't care if your compiler isn't efficient when compiling sloppy code.)
- daeken 11y ago
  Having written a lot of driver/kernel code and also dabbled in Rust, I have a hard time seeing anything that Rust can't do in this regard. All of the memory manipulation you will typically do is doable in Rust, though you will certainly lose some of the safety if you're bouncing things between types. The only real limitation you'll have is the inability (AFAIK) to do inline assembly. There'd be no need to rewrite anything to work with Rust; the binaries emitted by the compiler should be just fine, assuming ABI compatibility. Maybe some changes to the way things are linked?
  - foogered 11y ago
    You should be able to use inline assembly in rust-nightly: https://doc.rust-lang.org/nightly/book/inline-assembly.html https://doc.rust-lang.org/nightly/book/inline-assembly.html
    - daeken 11y ago
      Awesome, thanks. I knew it was missing last time I checked, but I should never assume with such active development going on!
      - spoiler 11y ago
        cc: @foogered This is what's been so off-putting about Rust, to be honest. "I should never assume" is precisely what bothered me. I loved Rust at first, but it required so much maintenance with its constant changes between versions, that I stated to dislike it. I didn't have much time (and I enjoy learning new languages), but when I had to virtually unlearn something just learned the previous day, it became frustrating after a few iterations. A colleague at work once sneered at me when I suggested I could do my next [internal] project in Rust; I didn't end up using Rust, because I got self-conscious about the idea, and kinda got scared that I'll have to do much, much more work simply because completely valid code would not work in a few days. That feeling is kinda stuck with me to this very day, even though Rust wasn't a stable release back then, and now is. Not to mention, there's people I've talked to about Rust who still feel like Rust isn't mature and never will be—simply because of it's "reputation." [1]: They were referring to how dramatically and chaotically it changed before the first stable release.
        mathetic 11y ago
        >> They were referring to how dramatically and chaotically it changed before the first stable release. Exactly.
        pcwalton 11y ago
        What would you preferred Rust have done to avoid this "reputation"? I mean, as far as I can see, the options were "develop Rust in secret" or "make a language much less suited to this domain".
- aidenn0 11y ago
  Are you on a 64-bit target? Then a uint32_t gets converted to a 64-bit signed value before any arithmetic operator is applied. There are some real pitfalls here, though they are often exaggerated. Do you turn off the TBAA in your compiler? In my experience most systems programmers either turn it off, or don't know the rules. [edit] I forgot int is typically 32-bits on 64-bit targets. The same argument would still apply for uint16_t and smaller though.
  - dbaupp 11y ago
    Aren't numeric promotions only applied in mixed expressions? (However, I'm not a C language-lawyer, so I could very easily be incorrect.)
    - pcwalton 11y ago
      No, sadly. In some cases (floating point) what happens is even implementation-dependent (though queryable with FLT_EVAL_METHOD).
    - aidenn0 11y ago
      All smaller than int types are promoted to int before calculations are done. I was wrong that uint32_t is smaller than int though. For an actual example: uint32_t combineTwoUint16(uint16_t x, uint16_t y) { return x<<16 | y; } If int is larger than 16 bits, then this is technically undefined behavior in the case that x is greater than 2^15 as it's signed integer overflow.
      - vtsrh 11y ago
        Well the solution is simple: return (x+0U)<<16 | (y+0U); Would it be insane to have a inline library for every arithmetic operation, that would handle such cases and offer addition optional functionality?
        lmm 11y ago
        The intersection of people who are willing to rewrite all their arithmetic to use such a library with people who are not willing to switch to a non-C language is rather small.
- spoiler 11y ago
  I agree, and I'd like to add that its not just this particular author, but most people who criticise C about it's "insecurities" use sloppy code when they criticise C, which always bothers me. I'm far from being a C fan (I'm also a Ruby fanboy), but programming languages aren't safe, only code can be safe, and that depends entirely on the developer. Yes, it's "easier" to introduce some bugs in C than Ruby (or Go, or whatever), but that's because whoever wrote that code with the bug didn't know C well enough. Is that C's fault? Same can be said about any language, really. If you don't know that String#match returns nil on unsuccessful matches and try to call MatchData#[], you'll get a NPE (something along the lines of "undefined method `[]' for NilClass"). This is very similar to dereferencing a NULL pointer in C[1]. [1]: I know dereferencing a NULL pointer in C is undefined behaviour, but your program will crash—if you're lucky enough—when you try to work with NULL pointers when you don't expect them.
  - dbaupp 11y ago
    This is nonsense. C has a very weak type system and very weak runtime guarantees, making it much easier to introduce problems with no indication that something's up. Other languages with strong type systems and/or stronger runtime checks eliminate large classes of bugs that are very easy to trigger in C. So, yes, it is C's "fault" that it doesn't protect against classes of bugs that many other languages do. Sure, those languages have some of the same bugs that C does, but they're missing most of the very worst ones and that's really powerful. For example, a garbage collector protects against accessing dangling pointers: it's just not something the programmer has to worry about at all. Rejecting cricitisms of C's safety inadequacies with "just code better"/"just learn the language better" doesn't work in practice: there have been too many high-profile vulnerabilities in C software, many of which would've been much harder to trigger in other languages.
    - cremno 11y ago
      >C has a very weak type system and very weak runtime guarantees, making it much easier to introduce problems with no indication that something's up. Here's an interesting example I've stumbled upon a few weeks ago: https://stackoverflow.com/questions/31037149/type-safety-for-complex-arithmetic-in-c99/ https://stackoverflow.com/questions/31037149/type-safety-for... float _Complex -> float doesn't require any diagnostic even though the imaginary component is (silently) discarded. Clang has one (not enabled by default or -Wall/-Wextra) however current GCC versions haven't.
- to3m 11y ago
  I switched over to using size_t for array indexes a few years ago, and found this a big improvement. And once I switched to size_t, most of my uses of unsigned went away - either you care about the size (and you want a uintN_t), or you don't (and you probably want size_t).
  - lsiebert 11y ago
    I think this works best if you pass a pointer to an error structure in the parameters. I've found people often use signed integer types with arrays so they can return a -1 to signal that something went wrong.
- pcwalton 11y ago
  > But none of those languages would let me do what I need to do: zero-copy writes from the network driver through to the RAID backend. Why not?
- deathanatos 11y ago
  > I have difficulty accepting "let's replace C with X", where X is a memory-managed language. What about C++ (which adds RAII, which I believe is indispensable for writing correct code, especially over C) or Rust, which adds much better memory correctness? I'm in agreement where the article says, > C, and derivatives like C++, is a very dangerous language the write safety/correctness critical software in, and my personal opinion is that it is almost impossible to write security critical software in it Though I believe it can be done in C++, with some discipline (but much less than C would require). > As a systems programmer (I write SCSI driver code in C), I think SCSI driver code counts as a niche application > I can't overemphasize how important it is to be able to address memory as a flat range of bytes, regardless of how that memory was originally handed to me. I need to have uint8_t* pointers into the middle of buffers which I can then typecast into byte-aligned structs. My understanding is that C doesn't generally allow this; that's what the strict aliasing rule is, and what's "wrong"[1] with several of the examples in the article. IIRC, you can get a [unsigned] char * into a struct (but why?[2]), but attempting to cast a char * to a struct foo * is forbidden. (Of course, with amends to the thread's original purpose, which is asking what the common layman understands / uses / depends on. Type aliasing is not well understood in my opinion. I'm not entirely confident I've got it right in this post.) > If your memory manager would not allow this or would move this memory around, that's a non-starter. (same comments about Rust/C++) > We never use 'int', by the way. We use the C99 well-defined types in stdint.h. Could this value go greater than a uint32_t can represent? Make it a uint64_t. Does it need to be signed? No? Make sure it's unsigned. A lot of what he's complaining about is sloppy code. In my experience, this is a rare thing; especially while interviewing, I find the majority of candidates — claiming to be most comfortable in C (we allow language of choice, in the hopes that you choose your strongest!) — don't know what `size_t` is. [1]: The array copy code is correct, but the author is lamenting optimizations that cannot be taken unless we assume the pointers don't alias; the int-to-float code is UB (hence why he writes "miscompile" in quotes; it's UB, so by definition there's no wrong output (though an error might be nice); this is also why "obvious" is in quotes: humans know what the programmer meant, but what the programmer wrote is UB; I think this is telling about C: human expectation and the language don't align, from a language-design standpoint, this is not good). [2]: most of the time I see people reaching for a char-pointer-into-a-struct, or cast-char-pointer-to-struct, they're short circuiting actually decoding some I/O byte-stream into an in memory data structure. This is not portable, unless — maybe — if you do "packed" structures (which is still not portable, I believe), but then you're sacrificing performance by potentially having unaligned members in the struct (which are harder for the processor to deal with, and might require multiple (e.g., MIPS) or unaligned (e.g., x86, amd64)) loads/stores.
- tokenrove 11y ago
  Most languages with garbage collectors also have mechanisms for working with memory that doesn't get touched by the collector, to support FFI. Also, consider a language like Ada for driver work.
- pjmlp 11y ago
  > But none of those languages would let me do what I need to do: zero-copy writes from the network driver through to the RAID backend. Oberon, Modula-3 and D allow for it via their SYSTEM/Unsafe/@system modules, but the two former ones failed to get a dent into the OS market (for various reasons) and D still has some improvements to their memory model going on. Also Ada and SPARK are usually the languages to reach for in life critical systems. Also lets not forget before C became widespread outside UNIX, Modula-2 and Pascal dialects were saner alternatives.
- sklogic 11y ago
  > I need to have uint8_t* pointers into the middle of buffers which I can then typecast into byte-aligned structs. Quite a lot of CPUs would just trap here. Assuming that unaligned access is allowed is a sin.
  - Nursie 11y ago
    Really? Because this is very common practice in device or network code, in my experience.
    - sklogic 11y ago
      Yes, it is a very common crap indeed. I had a lot of pain porting some Linux filesystems to Sparc, for example (had to give up back then, that code was beyond any hope). Not to mention the endianess issues, an often sight in the crappy networking code.
  - bluetomcat 11y ago
    > Quite a lot of CPUs would just trap here Or even worse -- for example, ARM CPUs usually round-down the misaligned address to the closest boundary when alignment checking is disabled. This means that attempting to access a 4-byte int at location 11 will silently let the CPU access it at location 8. This can manifest in some very nasty bugs.
    - stephencanon 11y ago
      IIRC this behavior went away in ARMv6.
- zamalek 11y ago
  > move this memory around A lot of people seem to assume that Chris (the author) was talking about managed memory, which he never mentioned once. Managed memory is runtime safety, a type system is compile-time safety. He's complaining about the type system. As an example: > address memory as a flat range of bytes [...] I can then typecast into byte-aligned structs You should never have to do that. You shouldn't be able to do that. Your job should be far simpler. Look at unique_ptr: a whole class of bugs are eliminated by this ZERO cost abstraction. Possibly what Chris is advocating is being able to describe what an I/O port is to the compiler and then using that abstraction to write your SCSI driver. This intent should be compiled down to as-good machine code (if not better) than what your C compiler would have given you - in the same way that unique_ptr is compiled. I don't think any existing language gets this right.
  - sharpneli 11y ago
    > I don't think any existing language gets this right. Which is precisely the reason C is still used. Until we have a language that produces at least as good results as C and is safer we're not going to see any change in this area.
    - zamalek 11y ago
      Agreed. I was pointing out that my interpretation of the email was a hypothetical tone, instead of a factual tone. As far as I'm concerned C isn't good enough, but I generally keep my mouth shut about it because I can't offer anything constructive to the discussion: I don't know what a "good C" would look like.
  - albinofrenchy 11y ago
    > You should never have to do that. You shouldn't be able to do that. Someone, at somepoint, has to do this though. Custom memory allocators are more or less predicated on having a byte buffer you chop up and use like this. I think the best we get -- especially in driver code -- is well thought out design that have low cost abstractions between the device details and the application logic. But that seems like a library detail more than a compiler or language one.
mrpippy 11y ago
For the for loop example, is there some reason why clang doesn't output a warning like "Does 'i' really need to be signed? If so, explicitly make it a 'signed int'. Otherwise, change it to be unsigned"
- porges 11y ago
  The point here is not particularly about signedness, it's that UB allows better optimizations to be performed. If overflow is defined to wrap around then it's potentially an infinite loop (take N == MAXVALUE). With overflow defined as UB you can say the loop executes exactly N times (because you're not allowed to write code that overflows). So UB is both bad and a source of power :)
  - cowsandmilk 11y ago
    > The point here is not particularly about signedness But in the case of C, that is what it is about since unsigned integers have defined behavior, so you can only have UB and the optimizations when you use a signed integer.
    - jeffreyrogers 11y ago
      Yes, but more generally UB allows optimizations that wouldn't be allowed otherwise. The whole reason C has so many undefined behaviors is for the benefit of compiler writers.
      - Joky 11y ago
        I believe it is not the original reason: most UB are about funky HW. For instance for the non-wrap signed int it can be explained because C does not assume you have 2-complement hardware.
        jcranmer 11y ago
        More specifically, some early hardware would trap on signed overflow. A lot of undefined behavior in C actually comes from "some machine would cause a trap", and C predates the invention of precise trapping in out-of-order processors. The possibility of traps is generally the difference between undefined behavior or unspecified/implementation-defined behavior.
jeffreyrogers 11y ago
For those who don't know Chris Lattner[1], who wrote this post, is the primary author of LLVM and more recently of Swift, so he knows a bit about what he's talking about :) [1]: https://en.wikipedia.org/wiki/Chris_Lattner https://en.wikipedia.org/wiki/Chris_Lattner
ryanmarsh 11y ago
Have we lost sight of the fact that when we talk about a programming language we're really talking about how to put bits on CPU registers?
- kd0amg 11y ago
  We aren't. Most programming languages do not expose CPU registers to the programmer (because they are not semantically important), and most programmers do not think in terms of moving bits in and out of registers (and typically don't have much to gain from doing so). CPU registers are just an accident of implementation. Programming as a general activity is not inherently tied to a register-based CPU. In fact, programmers are often happy to have compilers/machines remove some put-bits-into-register activities.
JustSomeNobody 11y ago
What would be considered "security critical"? SSH? IPTables? Linux kernel?
- AgentME 11y ago
  Personally, as someone that doesn't want to be hacked, I would think anything I run that connects to the internet and isn't already very well sandboxed.
pjmlp 11y ago
"My hope is that the industry will eventually move to better systems programming languages, but that will take a very very long time..." -- Chris Lattner Yes, a very long time. Modula-2 was born in 1978, but we can go back to Algol and Lisp even.
mcguire 11y ago
"In the first example above, it is that 'int' is the default type people generally reach for, not 'long', and that array indexing is expressed with integers instead of iterators. This isn’t something that we’re going to 'fix' in the C language, the C community, or the body of existing C code." The majority of that message is pretty well said, but this particular part leaves me cold. The problem isn't that 'int' is the default type, not 'long', nor is it that array indexing isn't done with iterators. (Ever merged two arrays? It's pretty clear using int indexes or pointers, but iterators can get verbose. C++ does a very good job, though, by making iterators look like pointers.) The problem is that, in C, the primitive types don't specifically describe their sizes. If you want a 32-bit variable, you should be able to ask for an unsigned or signed 32-bit variable. If you want whatever is best on this machine, you should be able to ask for whatever is word-sized. Unfortunately, C went with char <= short <= int <= long (, longlong, etc.); in an ideal world, 'int' would be the machine's word size, but when all the world's a VAX, 'int' means 32-bits. That is one of the major victories with Rust: most primitive types are sized, with an additional word-sized type.
- Gibbon1 11y ago
  Then again with C99 you do have stdint.h which gives you defined width types as well as minimum width types. And others.
byuu 11y ago
I understand that in some cases, these heroic compiler optimizations can offer significant performance increases. We should keep C around as it is for when said performance is critical. But surely, we can design a language that has no undefined behavior, without substantial deviations from C's syntax, and without massive performance penalties. This language would be great for things that prize security over performance. And the trick is, we don't need to rewrite all software in existence in a new language to get here! C can be this language, all we need is a special compilation flag that replaces undefined behavior with defined behavior. Functions called inside a function's arguments? Say they evaluate left-to-right. Shift right on signed types? Say it's arithmetic. Size of a byte? Say it's 8-bits. memset(0x00) on something going out of scope? If the developer said to do it, do it anyway. Underlying CPU doesn't support this? Emulate it. If it can't be emulated, then don't use code that requires the safe flag on said architecture. Yeah, screw the PDP-11. And yeah, it'll be slower in some cases. Yes, even twice as slow in some cases. But still far better than moving to a bytecode or VM language. And when we have guaranteed behavior of C, we can write new DSLs that transcode to C, without carrying along all of C's undefined behavior with it. You want to talk about writing in higher-level languages like Python and then having C for the underlying performance critical portions? Why not defined-behavior C for the security-critical and cold portions of code, and undefined-behavior C for the critical portions? Maybe Google wouldn't accept the speed penalty; but I'd happily drop my personal VPS from ~8000 maximum simultaneous users to ~5000 if it greatly decreased the odds of being vulnerable to the next Heartbleed. But I'm not willing to completely abandon all C code, and drop down to ~200 simultaneous users, to write it in Ruby.
- pcwalton 11y ago
  > But surely, we can design a language that has no undefined behavior, without substantial deviations from C's syntax, and without massive performance penalties. …Including undefined behavior around memory allocation, in particular use-after-free? What to do about that is the big question, in my mind. Other forms of UB can mostly be patched up straightforwardly with a clean design (though there are some tough questions around bounds checks). But when it comes to UAF, there are basically three ways you can go about this and still remain a runtimeless systems language: 1. Compromise on "no UB" for use-after-free. UAF remains undefined behavior. Some variants of Ada with dynamic memory allocation have this, and I believe many Pascals did this. It's a popular approach in many new systems languages, like Jonathan Blow's Jai. 2. Disallow dynamic allocation. This is the approach taken by SPARK and other hardened variants of Ada. 3. Allow dynamic allocation, but statically check it with a region system. This is Rust's approach. Eliminating memory safety problems in this way while avoiding a GC is pretty much unique to that language, though it's obviously influenced by many other systems that came before it (C++, Cyclone). All of the options have serious downsides. Option (1) opens you up to what has become, in 2015, a very common RCE vector. Option (2) is very limiting and pretty much restricts your language to embedded development. Option (3) has large complexity and expressiveness costs (though once you've paid the cost you can get data race freedom without any extra work, which is nice). Altogether it's a really difficult problem with tough tradeoffs all around.
carlosrg 11y ago
Until I see really big and open source projects like WebKit or Clang itself moving to Swift or whatever, anything I read about moving to "better systems languages" is like reading a letter to Santa Claus. I doubt C++ is going anywhere, especially when C++ itself is not standing still and evolving (C++11, 14, 17...) while maintaining backwards compatibility.
- pjmlp 11y ago
  What about having early versions Mac OS written in Object Pascal, only to rewrite it in C for pleasure of the UNIX hordes?
  - GFK_of_xmaspast 11y ago
    I wasn't a mac user until later, are you saying that was done for reasons of ideology?
    - pjmlp 11y ago
      Yes. Mac OS was initially written in a mix of Object Pascal and Assembly. Even Photoshop 1.0 was, check the available source code. Object Pascal was the inspiration for Turbo Pascal 5.5 OOP extensions. After a few OS releases, UNIX was already getting user in the industry and pressure from users about C compiler availability grew. Apple introduced a new SDK with C and C++ support, including a C++ framework. Afterwards the new OS apis were written in C. Also by this time, Apple did their first attempt at the UNIX market with A/UX.
nikanj 11y ago
Once again, http://research.microsoft.com/en-us/people/mickens/thenightwatch.pdf http://research.microsoft.com/en-us/people/mickens/thenightw...

deleted 11y ago

[deleted]

I have difficulty accepting "let's replace C with X", where X is a memory-managed language. As a systems programmer (I write SCSI driver code in C), I can't overemphasize how important it is to be able to address memory as a flat range of bytes, regardless of how that memory was originally handed to me. I need to have uint8_t* pointers into the middle of buffers which I can then typecast into byte-aligned structs. If your memory manager would not allow this or would move this memory around, that's a non-starter. I don't stick with C because I love it. If I'm writing something for my own purposes, I use Ruby. I've written some server code in Golang (non-production), and it's pretty nifty, even if the way it twists normal C syntax breaks my brain. I even dabble in the dark side (C++) personally and professionally from time to time. And in a previous life, I was reasonably proficient in C# (that's the CLR 2.0 timeframe; I'm completely useless at it in these crazy days of LINQ and the really nifty CLR 4 features...and there's probably even more stuff I haven't even become aware of). But none of those languages would let me do what I need to do: zero-copy writes from the network driver through to the RAID backend. And even if they did, the pain of rewriting our entire operating system in Go or Rust or whatever would be way more than the alleviated pain of using a "nicer" language. (We never use 'int', by the way. We use the C99 well-defined types in stdint.h. Could this value go greater than a uint32_t can represent? Make it a uint64_t. Does it need to be signed? No? Make sure it's unsigned. A lot of what he's complaining about is sloppy code. I don't care if your compiler isn't efficient when compiling sloppy code.)

daeken 11y ago
Having written a lot of driver/kernel code and also dabbled in Rust, I have a hard time seeing anything that Rust can't do in this regard. All of the memory manipulation you will typically do is doable in Rust, though you will certainly lose some of the safety if you're bouncing things between types. The only real limitation you'll have is the inability (AFAIK) to do inline assembly. There'd be no need to rewrite anything to work with Rust; the binaries emitted by the compiler should be just fine, assuming ABI compatibility. Maybe some changes to the way things are linked?
- foogered 11y ago
  You should be able to use inline assembly in rust-nightly: https://doc.rust-lang.org/nightly/book/inline-assembly.html https://doc.rust-lang.org/nightly/book/inline-assembly.html
  - daeken 11y ago
    Awesome, thanks. I knew it was missing last time I checked, but I should never assume with such active development going on!
    - spoiler 11y ago
      cc: @foogered This is what's been so off-putting about Rust, to be honest. "I should never assume" is precisely what bothered me. I loved Rust at first, but it required so much maintenance with its constant changes between versions, that I stated to dislike it. I didn't have much time (and I enjoy learning new languages), but when I had to virtually unlearn something just learned the previous day, it became frustrating after a few iterations. A colleague at work once sneered at me when I suggested I could do my next [internal] project in Rust; I didn't end up using Rust, because I got self-conscious about the idea, and kinda got scared that I'll have to do much, much more work simply because completely valid code would not work in a few days. That feeling is kinda stuck with me to this very day, even though Rust wasn't a stable release back then, and now is. Not to mention, there's people I've talked to about Rust who still feel like Rust isn't mature and never will be—simply because of it's "reputation." [1]: They were referring to how dramatically and chaotically it changed before the first stable release.
      - mathetic 11y ago
        >> They were referring to how dramatically and chaotically it changed before the first stable release. Exactly.
      - pcwalton 11y ago
        What would you preferred Rust have done to avoid this "reputation"? I mean, as far as I can see, the options were "develop Rust in secret" or "make a language much less suited to this domain".
aidenn0 11y ago
Are you on a 64-bit target? Then a uint32_t gets converted to a 64-bit signed value before any arithmetic operator is applied. There are some real pitfalls here, though they are often exaggerated. Do you turn off the TBAA in your compiler? In my experience most systems programmers either turn it off, or don't know the rules. [edit] I forgot int is typically 32-bits on 64-bit targets. The same argument would still apply for uint16_t and smaller though.
- dbaupp 11y ago
  Aren't numeric promotions only applied in mixed expressions? (However, I'm not a C language-lawyer, so I could very easily be incorrect.)
  - pcwalton 11y ago
    No, sadly. In some cases (floating point) what happens is even implementation-dependent (though queryable with FLT_EVAL_METHOD).
  - aidenn0 11y ago
    All smaller than int types are promoted to int before calculations are done. I was wrong that uint32_t is smaller than int though. For an actual example: uint32_t combineTwoUint16(uint16_t x, uint16_t y) { return x<<16 | y; } If int is larger than 16 bits, then this is technically undefined behavior in the case that x is greater than 2^15 as it's signed integer overflow.
    - vtsrh 11y ago
      Well the solution is simple: return (x+0U)<<16 | (y+0U); Would it be insane to have a inline library for every arithmetic operation, that would handle such cases and offer addition optional functionality?
      - lmm 11y ago
        The intersection of people who are willing to rewrite all their arithmetic to use such a library with people who are not willing to switch to a non-C language is rather small.
spoiler 11y ago
I agree, and I'd like to add that its not just this particular author, but most people who criticise C about it's "insecurities" use sloppy code when they criticise C, which always bothers me. I'm far from being a C fan (I'm also a Ruby fanboy), but programming languages aren't safe, only code can be safe, and that depends entirely on the developer. Yes, it's "easier" to introduce some bugs in C than Ruby (or Go, or whatever), but that's because whoever wrote that code with the bug didn't know C well enough. Is that C's fault? Same can be said about any language, really. If you don't know that String#match returns nil on unsuccessful matches and try to call MatchData#[], you'll get a NPE (something along the lines of "undefined method `[]' for NilClass"). This is very similar to dereferencing a NULL pointer in C[1]. [1]: I know dereferencing a NULL pointer in C is undefined behaviour, but your program will crash—if you're lucky enough—when you try to work with NULL pointers when you don't expect them.
- dbaupp 11y ago
  This is nonsense. C has a very weak type system and very weak runtime guarantees, making it much easier to introduce problems with no indication that something's up. Other languages with strong type systems and/or stronger runtime checks eliminate large classes of bugs that are very easy to trigger in C. So, yes, it is C's "fault" that it doesn't protect against classes of bugs that many other languages do. Sure, those languages have some of the same bugs that C does, but they're missing most of the very worst ones and that's really powerful. For example, a garbage collector protects against accessing dangling pointers: it's just not something the programmer has to worry about at all. Rejecting cricitisms of C's safety inadequacies with "just code better"/"just learn the language better" doesn't work in practice: there have been too many high-profile vulnerabilities in C software, many of which would've been much harder to trigger in other languages.
  - cremno 11y ago
    >C has a very weak type system and very weak runtime guarantees, making it much easier to introduce problems with no indication that something's up. Here's an interesting example I've stumbled upon a few weeks ago: https://stackoverflow.com/questions/31037149/type-safety-for-complex-arithmetic-in-c99/ https://stackoverflow.com/questions/31037149/type-safety-for... float _Complex -> float doesn't require any diagnostic even though the imaginary component is (silently) discarded. Clang has one (not enabled by default or -Wall/-Wextra) however current GCC versions haven't.
to3m 11y ago
I switched over to using size_t for array indexes a few years ago, and found this a big improvement. And once I switched to size_t, most of my uses of unsigned went away - either you care about the size (and you want a uintN_t), or you don't (and you probably want size_t).
- lsiebert 11y ago
  I think this works best if you pass a pointer to an error structure in the parameters. I've found people often use signed integer types with arrays so they can return a -1 to signal that something went wrong.
pcwalton 11y ago
> But none of those languages would let me do what I need to do: zero-copy writes from the network driver through to the RAID backend. Why not?
deathanatos 11y ago
> I have difficulty accepting "let's replace C with X", where X is a memory-managed language. What about C++ (which adds RAII, which I believe is indispensable for writing correct code, especially over C) or Rust, which adds much better memory correctness? I'm in agreement where the article says, > C, and derivatives like C++, is a very dangerous language the write safety/correctness critical software in, and my personal opinion is that it is almost impossible to write security critical software in it Though I believe it can be done in C++, with some discipline (but much less than C would require). > As a systems programmer (I write SCSI driver code in C), I think SCSI driver code counts as a niche application > I can't overemphasize how important it is to be able to address memory as a flat range of bytes, regardless of how that memory was originally handed to me. I need to have uint8_t* pointers into the middle of buffers which I can then typecast into byte-aligned structs. My understanding is that C doesn't generally allow this; that's what the strict aliasing rule is, and what's "wrong"[1] with several of the examples in the article. IIRC, you can get a [unsigned] char * into a struct (but why?[2]), but attempting to cast a char * to a struct foo * is forbidden. (Of course, with amends to the thread's original purpose, which is asking what the common layman understands / uses / depends on. Type aliasing is not well understood in my opinion. I'm not entirely confident I've got it right in this post.) > If your memory manager would not allow this or would move this memory around, that's a non-starter. (same comments about Rust/C++) > We never use 'int', by the way. We use the C99 well-defined types in stdint.h. Could this value go greater than a uint32_t can represent? Make it a uint64_t. Does it need to be signed? No? Make sure it's unsigned. A lot of what he's complaining about is sloppy code. In my experience, this is a rare thing; especially while interviewing, I find the majority of candidates — claiming to be most comfortable in C (we allow language of choice, in the hopes that you choose your strongest!) — don't know what `size_t` is. [1]: The array copy code is correct, but the author is lamenting optimizations that cannot be taken unless we assume the pointers don't alias; the int-to-float code is UB (hence why he writes "miscompile" in quotes; it's UB, so by definition there's no wrong output (though an error might be nice); this is also why "obvious" is in quotes: humans know what the programmer meant, but what the programmer wrote is UB; I think this is telling about C: human expectation and the language don't align, from a language-design standpoint, this is not good). [2]: most of the time I see people reaching for a char-pointer-into-a-struct, or cast-char-pointer-to-struct, they're short circuiting actually decoding some I/O byte-stream into an in memory data structure. This is not portable, unless — maybe — if you do "packed" structures (which is still not portable, I believe), but then you're sacrificing performance by potentially having unaligned members in the struct (which are harder for the processor to deal with, and might require multiple (e.g., MIPS) or unaligned (e.g., x86, amd64)) loads/stores.
tokenrove 11y ago
Most languages with garbage collectors also have mechanisms for working with memory that doesn't get touched by the collector, to support FFI. Also, consider a language like Ada for driver work.
pjmlp 11y ago
> But none of those languages would let me do what I need to do: zero-copy writes from the network driver through to the RAID backend. Oberon, Modula-3 and D allow for it via their SYSTEM/Unsafe/@system modules, but the two former ones failed to get a dent into the OS market (for various reasons) and D still has some improvements to their memory model going on. Also Ada and SPARK are usually the languages to reach for in life critical systems. Also lets not forget before C became widespread outside UNIX, Modula-2 and Pascal dialects were saner alternatives.
sklogic 11y ago
> I need to have uint8_t* pointers into the middle of buffers which I can then typecast into byte-aligned structs. Quite a lot of CPUs would just trap here. Assuming that unaligned access is allowed is a sin.
- Nursie 11y ago
  Really? Because this is very common practice in device or network code, in my experience.
  - sklogic 11y ago
    Yes, it is a very common crap indeed. I had a lot of pain porting some Linux filesystems to Sparc, for example (had to give up back then, that code was beyond any hope). Not to mention the endianess issues, an often sight in the crappy networking code.
- bluetomcat 11y ago
  > Quite a lot of CPUs would just trap here Or even worse -- for example, ARM CPUs usually round-down the misaligned address to the closest boundary when alignment checking is disabled. This means that attempting to access a 4-byte int at location 11 will silently let the CPU access it at location 8. This can manifest in some very nasty bugs.
  - stephencanon 11y ago
    IIRC this behavior went away in ARMv6.
zamalek 11y ago
> move this memory around A lot of people seem to assume that Chris (the author) was talking about managed memory, which he never mentioned once. Managed memory is runtime safety, a type system is compile-time safety. He's complaining about the type system. As an example: > address memory as a flat range of bytes [...] I can then typecast into byte-aligned structs You should never have to do that. You shouldn't be able to do that. Your job should be far simpler. Look at unique_ptr: a whole class of bugs are eliminated by this ZERO cost abstraction. Possibly what Chris is advocating is being able to describe what an I/O port is to the compiler and then using that abstraction to write your SCSI driver. This intent should be compiled down to as-good machine code (if not better) than what your C compiler would have given you - in the same way that unique_ptr is compiled. I don't think any existing language gets this right.
- sharpneli 11y ago
  > I don't think any existing language gets this right. Which is precisely the reason C is still used. Until we have a language that produces at least as good results as C and is safer we're not going to see any change in this area.
  - zamalek 11y ago
    Agreed. I was pointing out that my interpretation of the email was a hypothetical tone, instead of a factual tone. As far as I'm concerned C isn't good enough, but I generally keep my mouth shut about it because I can't offer anything constructive to the discussion: I don't know what a "good C" would look like.
- albinofrenchy 11y ago
  > You should never have to do that. You shouldn't be able to do that. Someone, at somepoint, has to do this though. Custom memory allocators are more or less predicated on having a byte buffer you chop up and use like this. I think the best we get -- especially in driver code -- is well thought out design that have low cost abstractions between the device details and the application logic. But that seems like a library detail more than a compiler or language one.

mrpippy 11y ago

For the for loop example, is there some reason why clang doesn't output a warning like "Does 'i' really need to be signed? If so, explicitly make it a 'signed int'. Otherwise, change it to be unsigned"

porges 11y ago
The point here is not particularly about signedness, it's that UB allows better optimizations to be performed. If overflow is defined to wrap around then it's potentially an infinite loop (take N == MAXVALUE). With overflow defined as UB you can say the loop executes exactly N times (because you're not allowed to write code that overflows). So UB is both bad and a source of power :)
- cowsandmilk 11y ago
  > The point here is not particularly about signedness But in the case of C, that is what it is about since unsigned integers have defined behavior, so you can only have UB and the optimizations when you use a signed integer.
  - jeffreyrogers 11y ago
    Yes, but more generally UB allows optimizations that wouldn't be allowed otherwise. The whole reason C has so many undefined behaviors is for the benefit of compiler writers.
    - Joky 11y ago
      I believe it is not the original reason: most UB are about funky HW. For instance for the non-wrap signed int it can be explained because C does not assume you have 2-complement hardware.
      - jcranmer 11y ago
        More specifically, some early hardware would trap on signed overflow. A lot of undefined behavior in C actually comes from "some machine would cause a trap", and C predates the invention of precise trapping in out-of-order processors. The possibility of traps is generally the difference between undefined behavior or unspecified/implementation-defined behavior.

jeffreyrogers 11y ago

For those who don't know Chris Lattner[1], who wrote this post, is the primary author of LLVM and more recently of Swift, so he knows a bit about what he's talking about :) [1]: https://en.wikipedia.org/wiki/Chris_Lattner https://en.wikipedia.org/wiki/Chris_Lattner

ryanmarsh 11y ago

Have we lost sight of the fact that when we talk about a programming language we're really talking about how to put bits on CPU registers?

kd0amg 11y ago
We aren't. Most programming languages do not expose CPU registers to the programmer (because they are not semantically important), and most programmers do not think in terms of moving bits in and out of registers (and typically don't have much to gain from doing so). CPU registers are just an accident of implementation. Programming as a general activity is not inherently tied to a register-based CPU. In fact, programmers are often happy to have compilers/machines remove some put-bits-into-register activities.

JustSomeNobody 11y ago

What would be considered "security critical"? SSH? IPTables? Linux kernel?

AgentME 11y ago
Personally, as someone that doesn't want to be hacked, I would think anything I run that connects to the internet and isn't already very well sandboxed.

pjmlp 11y ago

"My hope is that the industry will eventually move to better systems programming languages, but that will take a very very long time..." -- Chris Lattner Yes, a very long time. Modula-2 was born in 1978, but we can go back to Algol and Lisp even.

mcguire 11y ago

"In the first example above, it is that 'int' is the default type people generally reach for, not 'long', and that array indexing is expressed with integers instead of iterators. This isn’t something that we’re going to 'fix' in the C language, the C community, or the body of existing C code." The majority of that message is pretty well said, but this particular part leaves me cold. The problem isn't that 'int' is the default type, not 'long', nor is it that array indexing isn't done with iterators. (Ever merged two arrays? It's pretty clear using int indexes or pointers, but iterators can get verbose. C++ does a very good job, though, by making iterators look like pointers.) The problem is that, in C, the primitive types don't specifically describe their sizes. If you want a 32-bit variable, you should be able to ask for an unsigned or signed 32-bit variable. If you want whatever is best on this machine, you should be able to ask for whatever is word-sized. Unfortunately, C went with char <= short <= int <= long (, longlong, etc.); in an ideal world, 'int' would be the machine's word size, but when all the world's a VAX, 'int' means 32-bits. That is one of the major victories with Rust: most primitive types are sized, with an additional word-sized type.

Gibbon1 11y ago
Then again with C99 you do have stdint.h which gives you defined width types as well as minimum width types. And others.

byuu 11y ago

I understand that in some cases, these heroic compiler optimizations can offer significant performance increases. We should keep C around as it is for when said performance is critical. But surely, we can design a language that has no undefined behavior, without substantial deviations from C's syntax, and without massive performance penalties. This language would be great for things that prize security over performance. And the trick is, we don't need to rewrite all software in existence in a new language to get here! C can be this language, all we need is a special compilation flag that replaces undefined behavior with defined behavior. Functions called inside a function's arguments? Say they evaluate left-to-right. Shift right on signed types? Say it's arithmetic. Size of a byte? Say it's 8-bits. memset(0x00) on something going out of scope? If the developer said to do it, do it anyway. Underlying CPU doesn't support this? Emulate it. If it can't be emulated, then don't use code that requires the safe flag on said architecture. Yeah, screw the PDP-11. And yeah, it'll be slower in some cases. Yes, even twice as slow in some cases. But still far better than moving to a bytecode or VM language. And when we have guaranteed behavior of C, we can write new DSLs that transcode to C, without carrying along all of C's undefined behavior with it. You want to talk about writing in higher-level languages like Python and then having C for the underlying performance critical portions? Why not defined-behavior C for the security-critical and cold portions of code, and undefined-behavior C for the critical portions? Maybe Google wouldn't accept the speed penalty; but I'd happily drop my personal VPS from ~8000 maximum simultaneous users to ~5000 if it greatly decreased the odds of being vulnerable to the next Heartbleed. But I'm not willing to completely abandon all C code, and drop down to ~200 simultaneous users, to write it in Ruby.

pcwalton 11y ago
> But surely, we can design a language that has no undefined behavior, without substantial deviations from C's syntax, and without massive performance penalties. …Including undefined behavior around memory allocation, in particular use-after-free? What to do about that is the big question, in my mind. Other forms of UB can mostly be patched up straightforwardly with a clean design (though there are some tough questions around bounds checks). But when it comes to UAF, there are basically three ways you can go about this and still remain a runtimeless systems language: 1. Compromise on "no UB" for use-after-free. UAF remains undefined behavior. Some variants of Ada with dynamic memory allocation have this, and I believe many Pascals did this. It's a popular approach in many new systems languages, like Jonathan Blow's Jai. 2. Disallow dynamic allocation. This is the approach taken by SPARK and other hardened variants of Ada. 3. Allow dynamic allocation, but statically check it with a region system. This is Rust's approach. Eliminating memory safety problems in this way while avoiding a GC is pretty much unique to that language, though it's obviously influenced by many other systems that came before it (C++, Cyclone). All of the options have serious downsides. Option (1) opens you up to what has become, in 2015, a very common RCE vector. Option (2) is very limiting and pretty much restricts your language to embedded development. Option (3) has large complexity and expressiveness costs (though once you've paid the cost you can get data race freedom without any extra work, which is nice). Altogether it's a really difficult problem with tough tradeoffs all around.

carlosrg 11y ago

Until I see really big and open source projects like WebKit or Clang itself moving to Swift or whatever, anything I read about moving to "better systems languages" is like reading a letter to Santa Claus. I doubt C++ is going anywhere, especially when C++ itself is not standing still and evolving (C++11, 14, 17...) while maintaining backwards compatibility.

pjmlp 11y ago
What about having early versions Mac OS written in Object Pascal, only to rewrite it in C for pleasure of the UNIX hordes?
- GFK_of_xmaspast 11y ago
  I wasn't a mac user until later, are you saying that was done for reasons of ideology?
  - pjmlp 11y ago
    Yes. Mac OS was initially written in a mix of Object Pascal and Assembly. Even Photoshop 1.0 was, check the available source code. Object Pascal was the inspiration for Turbo Pascal 5.5 OOP extensions. After a few OS releases, UNIX was already getting user in the industry and pressure from users about C compiler availability grew. Apple introduced a new SDK with C and C++ support, including a C++ framework. Afterwards the new OS apis were written in C. Also by this time, Apple did their first attempt at the UNIX market with A/UX.

nikanj 11y ago

Once again, http://research.microsoft.com/en-us/people/mickens/thenightwatch.pdf http://research.microsoft.com/en-us/people/mickens/thenightw...