7 ms·

> Can anybody make a strong case to me as to why are buffer overflows considered an issue in C when it takes like 10 minutes to write and test an array implemen

> Can anybody make a strong case to me as to why are buffer overflows considered an issue in C when it takes like 10 minutes to write and test an array implementation that prevents that from ever happening? The CVE database. Just because you 'can' write such an array implementation doesn't mean you will, doesn't mean your third party libs will, doesn't mean any of your legacy code uses it, and certainly doesn't mean you will properly test said array implementation correctly. The number of mitigations added to C compilers and OSes dealing mostly with C and C++ code. ASLR, W^X, /GS, -fstack-protector-all, AddressSanitizer, ... - note the lack of similar tools, or demand for them, for, say, JavaScript - despite it enjoying a similar ubiquity. I ask this in bad faith: I encourage you to share a single nontrivial codebase which actually creates the abstraction you've described and religiously adheres to using it throughout. As to why this is in bad faith: I'm definining "nontrivial" here to mean using 3rd party APIs - which will operate on C style arrays, not your project specific safe wrappers - and thus by definition won't be "religiously" sticking to said abstractions when using said APIs. By these definitions, the codebase I'm asking for doesn't exist - by definition. Even relaxing the "third party" rule, I haven't actually worked on a nontrivial C or C++ codebase without buffer overflow problems. Now, e.g. Rust will have the same problems when interacting with C APIs - and nontrivial programs will end up doing so eventually. However, by virtue of the language itself embracing safe-by-default, you're less likely to run into the same problems when consuming Rust APIs. You can also use third party static analysis tools to ensure you're using a "safe C subset" (such as MIRSA C), but "nobody" does that.

agumonkey 10y ago
It's crazy that it's not solved above the language level, if people really want zero cost abstraction and architecture friendliness at least tooling should check buffer logic and flag the binary in case Warnings have been ignored.
- dhsjxhx 10y ago
  >if people really want zero cost abstraction That one "if" is (by definition) not zero-cost.
  - steveklabnik 10y ago
    It is by definition a "zero-cost abstraction." Let's ask Stroustrup, who coined the term: > C++ implementations obey the zero-overhead principle: What you don’t use, you don’t pay for. And further: What you do use, you couldn’t hand code any better. Two points: What you don't use, you don't pay for: if you don't use array indexing, you won't get a bounds check. In addition, you can call an access method without a bounds check as well, so it truly is only if you use the checked version. What you do use, you couldn't hand-code any better: that bounds check is written the exact same way you'd write it in C. Therefore, this is a zero-cost abstraction.
    - aduffy 10y ago
      This is actually a very helpful comment. I used to think "zero-cost" meant "at compile-time", as in `newtype` in Haskell, etc. I'm guessing that's what the parent commenter thought as well, and I'd guess is what most people think when they hear the phrase.
      - steveklabnik 10y ago
        Thanks! It can definitely be a bit unintuitive at first. After all, everything has _some_ cost...
      - kqr 10y ago
        Well, you can still sort of view it that way. You can imagine the bounds check being a "compile time generation of the C code you'd write to check the bounds anyway".
        aduffy 10y ago
        The difference is that it's not dealt with entirely in the compile phase. i.e. language features that are checked at compilation and known to be true that are not needed at runtime. The Haskell `newtype` example I gave was meant to illustrate this, as newtype's are respected by the type system and then are treated as the underlying type at runtime.
phkahler 10y ago
I was thinking about mentioning MISRAble C. Outside the infotainment system, you should never have a buffer overrun on any micro in your car. Part of that comes from a ban on the use of malloc(). There is still a risk of array bounds problems, but those seem easier to avoid and more likely to be caught in testing. Why no malloc? It was originally due to the small memory sizes for code and data. The less standard library the better, and dynamic allocation may lead to heap fragmentation and a subsequent crash when malloc fails.
wangchow 10y ago
Honestly, we need a few AI coders to replace most of the developers in the world and then this won't be an issue. Bounds checking arrays and calloc instead of malloc isn't rocket science. It's a simple formula. The problem isn't the language it's the developers.
- BHSPitMonkey 10y ago
  Considering that a change in language completely solves the problem, it's hard to get on board with your thesis.
  - wangchow 10y ago
    My point is, a lot of people are spending time on this when it doesn't matter. In the limit that AI starts replacing human developers these subtle differences in language approaches zero. New languages here and there every day. Replace this replace that. When, in the end everyone is simply reinventing the "wheel" over-and-over. All these languages end up as assembly.
    - ketralnis 10y ago
      This sounds a lot like "why clean up my room when the heat death of the universe is coming anyway?", but if you do think that AI is going to supplant all of programming then be the change you want to see in the world! Get building and we'll see which one happens first. I honestly don't know who I'd put my money on between "AI takes over the world" and "programmers stop writing buffer overflows"
      - verdax_1 10y ago
        My guess is that if they make a general purpose programming AI then all other jobs will also be nonexistent besides being famous and doing YouTube reviews of movies. My thought is that the problems in the way of AI programming are more difficult and can be generalised to enough other jobs that programming is going to be the last job automated.
      - 10y ago
scott_karana 10y ago
Hate to tell you, but JavaScript implementations virtually all rely on C or C++ as well. And it's not limited to the VM itself: check out npm "native extensions" like `json`. Not to mention glibc, or the OSes themselves. By your definition, nothing is safe. And you're right ;)
- MichaelBurge 10y ago
  That's the old "formal proofs are useless because the proof checker might have a bug" argument. It's not very persuasive.
  - scott_karana 10y ago
    It's MaulingMonkey's argument ad absurdum is what is is :P
- MaulingMonkey 10y ago
  I'm aware. I note that Firefox is using some Rust code now - so perhaps that will change at some point, for at least one of the common JavaScript implementations, in the not too distant future. I don't imagine we'll see it for the majority within the decade - but who knows, maybe I'll be pleasantly surprised. I have less hope for the widespread adoption of OS kernels written in safer languages - given the general unwillingness to even use C++ there (although plenty of toy/'research' kernels in safer languages do exist.) Although maybe we'll see one within the next century? Perhaps a microkernel for use in containers? Of course, that still leaves bugs in the JITs, compilers, hardware, 'legacy' native interop, unsafe{} blocks, ...
  - rat87 10y ago
    > I note that Firefox is using some Rust code now - so perhaps that will change at some point, for at least one of the common JavaScript implementations, in the not too distant future. I don't imagine we'll see it for the majority within the decade - but who knows, maybe I'll be pleasantly surprised. Maybe I think currently there's no plan for it. Maybe after they finish servo to the degree where it supports all modern html features
Nacraile 10y ago
> I ask this in bad faith: I encourage you to share a single nontrivial codebase which actually creates the abstraction you've described and religiously adheres to using it throughout. As to why this is in bad faith: I'm definining "nontrivial" here to mean using 3rd party APIs - which will operate on C style arrays, not your project specific safe wrappers - and thus by definition won't be "religiously" sticking to said abstractions when using said APIs. By these definitions, the codebase I'm asking for doesn't exist - by definition. Even relaxing the "third party" rule, I haven't actually worked on a nontrivial C or C++ codebase without buffer overflow problems. I work on a C codebase that does this, although in the slightly weaker sense that it does drop the abstractions at a few isolated interaction points with external APIs (think openssl, linux system calls, and not a whole lot else). Yes, there is quite a lot of NIH. With essentially-uniform use of checked data structures, and an extremely comprehensive suite of automated tests getting run under ASAN (originally Valgrind), memory safety errors almost never get so far as being committed to the main branch. This is a complex, >1M SLOC distributed system that has seen several years of production use at this point, and as far as I can recall we have not seen a single memory safety related issue in production (a few have managed to to get as far as certification testing). General resouce-leak class issues have struck a few times, but are also pretty rare. Proprietary, naturally, so I can't actually show you (sorry), but it absolutely can be done in practice. It isn't even really all that difficult, it just needs to be done from the start, and then you just need a bit of discipline to keep it up.
- nxtrafalgar 10y ago
  >I work on a C codebase that does this And more power to you. Note the beginning of the parent comment, however: > Just because you 'can' write such an array implementation doesn't mean you will So yes, even if the codebase you work on does have these 'mythical', hard-to-achieve properties, that doesn't mean that most or even many C codebases will. Good engineering entails observing what problems actually occur and working to fix those. Memory safety issues do commonly occur in C codebases. Regardless of whether the fix in C is simple or even trivial, programmers aren't doing it. So, Rust has some value because it forces the programmer to produce code that is largely free from this type of issue. Enforcing norms like 'be more disciplined when writing C' or 'stop using external libraries' is much harder than simply using a different language.

agumonkey 10y ago

It's crazy that it's not solved above the language level, if people really want zero cost abstraction and architecture friendliness at least tooling should check buffer logic and flag the binary in case Warnings have been ignored.

dhsjxhx 10y ago
>if people really want zero cost abstraction That one "if" is (by definition) not zero-cost.
- steveklabnik 10y ago
  It is by definition a "zero-cost abstraction." Let's ask Stroustrup, who coined the term: > C++ implementations obey the zero-overhead principle: What you don’t use, you don’t pay for. And further: What you do use, you couldn’t hand code any better. Two points: What you don't use, you don't pay for: if you don't use array indexing, you won't get a bounds check. In addition, you can call an access method without a bounds check as well, so it truly is only if you use the checked version. What you do use, you couldn't hand-code any better: that bounds check is written the exact same way you'd write it in C. Therefore, this is a zero-cost abstraction.
  - aduffy 10y ago
    This is actually a very helpful comment. I used to think "zero-cost" meant "at compile-time", as in `newtype` in Haskell, etc. I'm guessing that's what the parent commenter thought as well, and I'd guess is what most people think when they hear the phrase.
    - steveklabnik 10y ago
      Thanks! It can definitely be a bit unintuitive at first. After all, everything has _some_ cost...
    - kqr 10y ago
      Well, you can still sort of view it that way. You can imagine the bounds check being a "compile time generation of the C code you'd write to check the bounds anyway".
      - aduffy 10y ago
        The difference is that it's not dealt with entirely in the compile phase. i.e. language features that are checked at compilation and known to be true that are not needed at runtime. The Haskell `newtype` example I gave was meant to illustrate this, as newtype's are respected by the type system and then are treated as the underlying type at runtime.

phkahler 10y ago

I was thinking about mentioning MISRAble C. Outside the infotainment system, you should never have a buffer overrun on any micro in your car. Part of that comes from a ban on the use of malloc(). There is still a risk of array bounds problems, but those seem easier to avoid and more likely to be caught in testing. Why no malloc? It was originally due to the small memory sizes for code and data. The less standard library the better, and dynamic allocation may lead to heap fragmentation and a subsequent crash when malloc fails.

wangchow 10y ago

Honestly, we need a few AI coders to replace most of the developers in the world and then this won't be an issue. Bounds checking arrays and calloc instead of malloc isn't rocket science. It's a simple formula. The problem isn't the language it's the developers.

BHSPitMonkey 10y ago
Considering that a change in language completely solves the problem, it's hard to get on board with your thesis.
- wangchow 10y ago
  My point is, a lot of people are spending time on this when it doesn't matter. In the limit that AI starts replacing human developers these subtle differences in language approaches zero. New languages here and there every day. Replace this replace that. When, in the end everyone is simply reinventing the "wheel" over-and-over. All these languages end up as assembly.
  - ketralnis 10y ago
    This sounds a lot like "why clean up my room when the heat death of the universe is coming anyway?", but if you do think that AI is going to supplant all of programming then be the change you want to see in the world! Get building and we'll see which one happens first. I honestly don't know who I'd put my money on between "AI takes over the world" and "programmers stop writing buffer overflows"
    - verdax_1 10y ago
      My guess is that if they make a general purpose programming AI then all other jobs will also be nonexistent besides being famous and doing YouTube reviews of movies. My thought is that the problems in the way of AI programming are more difficult and can be generalised to enough other jobs that programming is going to be the last job automated.
    - 10y ago

scott_karana 10y ago

Hate to tell you, but JavaScript implementations virtually all rely on C or C++ as well. And it's not limited to the VM itself: check out npm "native extensions" like `json`. Not to mention glibc, or the OSes themselves. By your definition, nothing is safe. And you're right ;)

MichaelBurge 10y ago
That's the old "formal proofs are useless because the proof checker might have a bug" argument. It's not very persuasive.
- scott_karana 10y ago
  It's MaulingMonkey's argument ad absurdum is what is is :P
MaulingMonkey 10y ago
I'm aware. I note that Firefox is using some Rust code now - so perhaps that will change at some point, for at least one of the common JavaScript implementations, in the not too distant future. I don't imagine we'll see it for the majority within the decade - but who knows, maybe I'll be pleasantly surprised. I have less hope for the widespread adoption of OS kernels written in safer languages - given the general unwillingness to even use C++ there (although plenty of toy/'research' kernels in safer languages do exist.) Although maybe we'll see one within the next century? Perhaps a microkernel for use in containers? Of course, that still leaves bugs in the JITs, compilers, hardware, 'legacy' native interop, unsafe{} blocks, ...
- rat87 10y ago
  > I note that Firefox is using some Rust code now - so perhaps that will change at some point, for at least one of the common JavaScript implementations, in the not too distant future. I don't imagine we'll see it for the majority within the decade - but who knows, maybe I'll be pleasantly surprised. Maybe I think currently there's no plan for it. Maybe after they finish servo to the degree where it supports all modern html features

Nacraile 10y ago

> I ask this in bad faith: I encourage you to share a single nontrivial codebase which actually creates the abstraction you've described and religiously adheres to using it throughout. As to why this is in bad faith: I'm definining "nontrivial" here to mean using 3rd party APIs - which will operate on C style arrays, not your project specific safe wrappers - and thus by definition won't be "religiously" sticking to said abstractions when using said APIs. By these definitions, the codebase I'm asking for doesn't exist - by definition. Even relaxing the "third party" rule, I haven't actually worked on a nontrivial C or C++ codebase without buffer overflow problems. I work on a C codebase that does this, although in the slightly weaker sense that it does drop the abstractions at a few isolated interaction points with external APIs (think openssl, linux system calls, and not a whole lot else). Yes, there is quite a lot of NIH. With essentially-uniform use of checked data structures, and an extremely comprehensive suite of automated tests getting run under ASAN (originally Valgrind), memory safety errors almost never get so far as being committed to the main branch. This is a complex, >1M SLOC distributed system that has seen several years of production use at this point, and as far as I can recall we have not seen a single memory safety related issue in production (a few have managed to to get as far as certification testing). General resouce-leak class issues have struck a few times, but are also pretty rare. Proprietary, naturally, so I can't actually show you (sorry), but it absolutely can be done in practice. It isn't even really all that difficult, it just needs to be done from the start, and then you just need a bit of discipline to keep it up.

nxtrafalgar 10y ago
>I work on a C codebase that does this And more power to you. Note the beginning of the parent comment, however: > Just because you 'can' write such an array implementation doesn't mean you will So yes, even if the codebase you work on does have these 'mythical', hard-to-achieve properties, that doesn't mean that most or even many C codebases will. Good engineering entails observing what problems actually occur and working to fix those. Memory safety issues do commonly occur in C codebases. Regardless of whether the fix in C is simple or even trivial, programmers aren't doing it. So, Rust has some value because it forces the programmer to produce code that is largely free from this type of issue. Enforcing norms like 'be more disciplined when writing C' or 'stop using external libraries' is much harder than simply using a different language.