7 ms·

It's 2023 and memory overwrite bugs are not just a thing theyre still number one

114 points by LinuxBender 3y ago

eatonphil 3y ago
Direct link to the list they're talking about: https://cwe.mitre.org/data/definitions/1425.html https://cwe.mitre.org/data/definitions/1425.html. And the list (top 10): * Out-of-bounds Write * Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting') * Improper Neutralization of Special Elements used in an SQL Command ('SQL Injection') * Use After Free * Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection') * Improper Input Validation * Out-of-bounds Read * Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal') * Cross-Site Request Forgery (CSRF) * Unrestricted Upload of File with Dangerous Type
- candiddevmike 3y ago
  By using a GC language or something like Rust, you (in theory) avoid a few of these, but the majority of them are logic errors that are hard to discover, IME.
  - bestouff 3y ago
    By using a GC language you avoid a few of them. By using Rust with a proper framework you should avoid most of them, if not all. A proper type system can save your ass very often.
    - srgpqt 3y ago
      Even C has a (simple) type system. The problem is that everyone (especially the standard library) take every opportunity to avoid it, instead casting everything to plain void pointers or char pointers. Thus the compiler can’t do much for safety checks. The good news is that you can build typesafe alternatives on top of the underlying std crap. Would be nice to have a standard <stdstruct.h> that provides common typesafe structs for things like os paths, sql strings, html strings, etc. just like <stdint.h> did for sized integers. I’ve rolled my own typesafe wrappers fo my current C project and it’s been great so far.
  - mrighele 3y ago
    A type system can help with injection issues. For example you could say that user provided strings are different from literals or from those that you provide. You then allow the user to concatenate them together only by passing through functions that quote the user provided ones. This should make very difficult to make injections, or at least avoid honest mistakes
    - usrusr 3y ago
      Types that carry a semantic difference, not a difference on the level of technical implementation are a fascinatingly underutilized concept. I'm sure that the idea has been around forever, and implementations available as well, but it never really made the big splash that I think it should have. I even carry a "what if" language concept in my mind (as in zero plans of implementation or anything like that, merely a thought experiment) where all names would be folded into types: in a lot of scopes you only ever have one of each. If you were to run Noah's ark you'd probably not bother remembering that you called the male penguin Frederick (or was it James?), you'd simply refer to him as the male penguin. The tools tools you'd want for dealing with all the situations where you need more than one in a scope would be just what the doctor ordered for that include semantic difference in typing thing.
      - grncdr 3y ago
        You may be interested to know that the Ceylon language has this (or had something very similar, I’ve not actually used it myself)
  - lolinder 3y ago
    The top 8 are either memory errors or errors that stem from joining strings. Memory errors are largely solved by using the right language, and we've known for decades not to join strings manually. SQL prepared statements are a thing, and modern templating engines escape HTML by default, with a suitably scary name for the opt-out function. Building similar tooling for OS calls and path traversal should be pretty trivial.
- marcosdumay 3y ago
  > * Cross-Site Request Forgery (CSRF) > * Unrestricted Upload of File with Dangerous Type Well, at least there are 2 without very refined standard methods to completely solve based on managed memory or strict types.
  - pphysch 3y ago
    > without very refined standard methods Doesn't every mature web framework prevent CSRF out-of-the-box with default middleware?
    - marcosdumay 3y ago
      To some extent, yes. There's also CORS. I'm not sure the problem can be completely solved without changes in browser behavior. And yep, that leaves 1 of the top 10 problems not being automatically solved by one-time measures. I didn't include it because differently from managed memory and strict types, I have never seen anybody fight against the use of web frameworks (just against heavy-featured ones, but the light ones also include CSRF protection).
- WalterBright 3y ago
  Array bounds errors in C, the number 1 problem, can be mitigated by adding this proposal to fix C: https://www.digitalmars.com/articles/C-biggest-mistake.html https://www.digitalmars.com/articles/C-biggest-mistake.html I don't really understand the resistance to it. It does not affect existing code at all, it's simple, it's easy to implement, and its worth has been proven in D. (It's one of D's most popular features.)
  - woodruffw 3y ago
    Do you have a link showing that the C standards body or working groups are resistant to this? My understanding is that they're currently more generally open to major security-oriented improvements than they ever have been in the past; it seems like now would be a good time for someone to put a slice/fat pointer proposal in front of them.
    - WalterBright 3y ago
      Every time I bring it up, people object to it, and nobody seems interested in proposing it to the Standards body. So what is my motivation in this, if I'm unwilling to make the effort to do a formal proposal? I know a lot about C, having implemented a C compiler from soup to nuts, and spent much of my career working with it. I have lot of experience with it, and with array overflows, both of my own doing and debugging other peoples' code. Very high on my list of reasons to do D was fixing that problem. I (along with Jan Knepper) developed the D array scheme. It works. It works great. It nails the problem so thoroughly that buffer overflow problems in D are very rare, because there just is little to no reason to use pointers to arrays in D. (I.e. buffer overflow problems in D happen when you don't use arrays.) Interestingly, it actually speeds up your code. How is that magic possible? It's because 0-terminated strings need to be constantly scanned to find the length, and arrays just know their length. It reduces memory consumption too! Because you can slice any string - not necessary to reallocate it. I've seen other extensions to C to solve the array overflow problem. They're all terrible, which is why one has to bludgeon people into using them. My proposal is simple, works, efficient, and people take to it like a duck to water. This problem also can only be solved at the Standards level, otherwise people will not use it. Writing an official proposal is a significant time sink, and I have enough to do working on D.
andrewstuart 3y ago
We need more than one “mainstream” (not functional) memory safe language.
- marcosdumay 3y ago
  Every mainstream language that isn't C or C++ is memory safe nowadays.
  - knorker 3y ago
    Except Go. "Go does have undefined behaviour: if your program has a race condition, the behaviour is undefined" -- Ian Lance Taylor, https://groups.google.com/g/golang-nuts/c/MB1QmhDd_Rk https://groups.google.com/g/golang-nuts/c/MB1QmhDd_Rk It annoys me to no end how Go could be created in the 21st century and willfully ignore so many lessons from the state of the art in language design. The more code I read and write (basically every day, by good programmers), the more disappointed I am in how much of a missed opportunity Go is. There are other fundamental flaws in Go, but that would be off topic.
    - marcosdumay 3y ago
      That's not a memory safety issue. And well, like people keep pointing on that discussion thread, C and C++ undefined behavior isn't just an operation you don't know the result; it's a much deeper and crazier thing that no other language has (unless when they inherit it by sharing compiling tools).
      - knorker 3y ago
        To your second point: Ian Lance Taylor explicitly says that Go has undefined behavior, which as you say is not just "don't know what it is". He (as he shows in this thread too) is, like me, very much aware of what UB is. He's even correcting people who are using the term "undefined" incorrectly. As for "memory safety", maybe you're right on the terminology. But this does mean that in Go a data race, being UB, can do anything. There are no guarantees of no use after free, or read/write out of bounds, anymore. If you create a data race in Go, you are in UB land. And that's one of the many many missed opportunities of Go.
charcircuit 3y ago
The project I work on uses C++ and memory bugs are a small percentage of actual bugs. Additionally none of the security bugs have been from memory bugs.
- chrisco255 3y ago
  What kind of project is it and what's the scope of its deployment?
keithalewis 3y ago
And virtually non-existent in software written for medical instruments.
- pjmlp 3y ago
  Now we just need to adopt the same development regulations and liability laws for the rest of the industry. Still looking forward for when health inspection, goods certificates, car street safety regulations, finally arrive into the computing industry .
  - anonzzzies 3y ago
    But how far would this go? If I throw some weekend project on GitHub, I need to spend time and money on this? Or is it only for commercially sold software? In both cases the large corps win again and it will hamper innovation, real free market, entreneurial spirit etc. Most (all?) of the regulated industries you cannot just get into and compete; it’s expensive and needs resources small companies cannot have. These industries are already regulated; maybe the software (which is also already regulated) needs stricter controls, but we already have this. Why would we need more? If you are in an industry, healthcare for instance, and you use a library with memory issues; that falls under the regulations of the company creating the software directly for the healthcare provider(s). Why is more needed?
    - pjmlp 3y ago
      Yes, so what? The little guy that wants to sell food on the street, needs to have same rules as the 5 star michelin restaurant in regards on hygine. At least in developed countries. And yet they manage to create their business, go figure. More is needed, because people have been educated to accept to pay for garbage software, in terms of quality. In all other industries, most people will return on the same day products they have bought, with comparable bad quality to such software.
scrame 3y ago
TIL the register is still around. good for them!
slaymaker1907 3y ago
As someone dealing with this stuff, I don’t think it’s even because these bugs are the most frequent, it’s just that these are easy to both detect (for the exploiter) and exploit.
kazinator 3y ago
Selection bias at work. Bugs in this category are likely to be serious enough to prompt CVE's. If CVE's are what your'e sampling in order go get an idea of what bugs are out there, memory overwrite bugs will be highly represented.

114 points by LinuxBender 3y ago

Direct link to the list they're talking about: https://cwe.mitre.org/data/definitions/1425.html https://cwe.mitre.org/data/definitions/1425.html. And the list (top 10): * Out-of-bounds Write * Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting') * Improper Neutralization of Special Elements used in an SQL Command ('SQL Injection') * Use After Free * Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection') * Improper Input Validation * Out-of-bounds Read * Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal') * Cross-Site Request Forgery (CSRF) * Unrestricted Upload of File with Dangerous Type

candiddevmike 3y ago
By using a GC language or something like Rust, you (in theory) avoid a few of these, but the majority of them are logic errors that are hard to discover, IME.
- bestouff 3y ago
  By using a GC language you avoid a few of them. By using Rust with a proper framework you should avoid most of them, if not all. A proper type system can save your ass very often.
  - srgpqt 3y ago
    Even C has a (simple) type system. The problem is that everyone (especially the standard library) take every opportunity to avoid it, instead casting everything to plain void pointers or char pointers. Thus the compiler can’t do much for safety checks. The good news is that you can build typesafe alternatives on top of the underlying std crap. Would be nice to have a standard <stdstruct.h> that provides common typesafe structs for things like os paths, sql strings, html strings, etc. just like <stdint.h> did for sized integers. I’ve rolled my own typesafe wrappers fo my current C project and it’s been great so far.
- mrighele 3y ago
  A type system can help with injection issues. For example you could say that user provided strings are different from literals or from those that you provide. You then allow the user to concatenate them together only by passing through functions that quote the user provided ones. This should make very difficult to make injections, or at least avoid honest mistakes
  - usrusr 3y ago
    Types that carry a semantic difference, not a difference on the level of technical implementation are a fascinatingly underutilized concept. I'm sure that the idea has been around forever, and implementations available as well, but it never really made the big splash that I think it should have. I even carry a "what if" language concept in my mind (as in zero plans of implementation or anything like that, merely a thought experiment) where all names would be folded into types: in a lot of scopes you only ever have one of each. If you were to run Noah's ark you'd probably not bother remembering that you called the male penguin Frederick (or was it James?), you'd simply refer to him as the male penguin. The tools tools you'd want for dealing with all the situations where you need more than one in a scope would be just what the doctor ordered for that include semantic difference in typing thing.
    - grncdr 3y ago
      You may be interested to know that the Ceylon language has this (or had something very similar, I’ve not actually used it myself)
- lolinder 3y ago
  The top 8 are either memory errors or errors that stem from joining strings. Memory errors are largely solved by using the right language, and we've known for decades not to join strings manually. SQL prepared statements are a thing, and modern templating engines escape HTML by default, with a suitably scary name for the opt-out function. Building similar tooling for OS calls and path traversal should be pretty trivial.
marcosdumay 3y ago
> * Cross-Site Request Forgery (CSRF) > * Unrestricted Upload of File with Dangerous Type Well, at least there are 2 without very refined standard methods to completely solve based on managed memory or strict types.
- pphysch 3y ago
  > without very refined standard methods Doesn't every mature web framework prevent CSRF out-of-the-box with default middleware?
  - marcosdumay 3y ago
    To some extent, yes. There's also CORS. I'm not sure the problem can be completely solved without changes in browser behavior. And yep, that leaves 1 of the top 10 problems not being automatically solved by one-time measures. I didn't include it because differently from managed memory and strict types, I have never seen anybody fight against the use of web frameworks (just against heavy-featured ones, but the light ones also include CSRF protection).
WalterBright 3y ago
Array bounds errors in C, the number 1 problem, can be mitigated by adding this proposal to fix C: https://www.digitalmars.com/articles/C-biggest-mistake.html https://www.digitalmars.com/articles/C-biggest-mistake.html I don't really understand the resistance to it. It does not affect existing code at all, it's simple, it's easy to implement, and its worth has been proven in D. (It's one of D's most popular features.)
- woodruffw 3y ago
  Do you have a link showing that the C standards body or working groups are resistant to this? My understanding is that they're currently more generally open to major security-oriented improvements than they ever have been in the past; it seems like now would be a good time for someone to put a slice/fat pointer proposal in front of them.
  - WalterBright 3y ago
    Every time I bring it up, people object to it, and nobody seems interested in proposing it to the Standards body. So what is my motivation in this, if I'm unwilling to make the effort to do a formal proposal? I know a lot about C, having implemented a C compiler from soup to nuts, and spent much of my career working with it. I have lot of experience with it, and with array overflows, both of my own doing and debugging other peoples' code. Very high on my list of reasons to do D was fixing that problem. I (along with Jan Knepper) developed the D array scheme. It works. It works great. It nails the problem so thoroughly that buffer overflow problems in D are very rare, because there just is little to no reason to use pointers to arrays in D. (I.e. buffer overflow problems in D happen when you don't use arrays.) Interestingly, it actually speeds up your code. How is that magic possible? It's because 0-terminated strings need to be constantly scanned to find the length, and arrays just know their length. It reduces memory consumption too! Because you can slice any string - not necessary to reallocate it. I've seen other extensions to C to solve the array overflow problem. They're all terrible, which is why one has to bludgeon people into using them. My proposal is simple, works, efficient, and people take to it like a duck to water. This problem also can only be solved at the Standards level, otherwise people will not use it. Writing an official proposal is a significant time sink, and I have enough to do working on D.

andrewstuart 3y ago

We need more than one “mainstream” (not functional) memory safe language.

marcosdumay 3y ago
Every mainstream language that isn't C or C++ is memory safe nowadays.
- knorker 3y ago
  Except Go. "Go does have undefined behaviour: if your program has a race condition, the behaviour is undefined" -- Ian Lance Taylor, https://groups.google.com/g/golang-nuts/c/MB1QmhDd_Rk https://groups.google.com/g/golang-nuts/c/MB1QmhDd_Rk It annoys me to no end how Go could be created in the 21st century and willfully ignore so many lessons from the state of the art in language design. The more code I read and write (basically every day, by good programmers), the more disappointed I am in how much of a missed opportunity Go is. There are other fundamental flaws in Go, but that would be off topic.
  - marcosdumay 3y ago
    That's not a memory safety issue. And well, like people keep pointing on that discussion thread, C and C++ undefined behavior isn't just an operation you don't know the result; it's a much deeper and crazier thing that no other language has (unless when they inherit it by sharing compiling tools).
    - knorker 3y ago
      To your second point: Ian Lance Taylor explicitly says that Go has undefined behavior, which as you say is not just "don't know what it is". He (as he shows in this thread too) is, like me, very much aware of what UB is. He's even correcting people who are using the term "undefined" incorrectly. As for "memory safety", maybe you're right on the terminology. But this does mean that in Go a data race, being UB, can do anything. There are no guarantees of no use after free, or read/write out of bounds, anymore. If you create a data race in Go, you are in UB land. And that's one of the many many missed opportunities of Go.

charcircuit 3y ago

The project I work on uses C++ and memory bugs are a small percentage of actual bugs. Additionally none of the security bugs have been from memory bugs.

chrisco255 3y ago
What kind of project is it and what's the scope of its deployment?

keithalewis 3y ago

And virtually non-existent in software written for medical instruments.

pjmlp 3y ago
Now we just need to adopt the same development regulations and liability laws for the rest of the industry. Still looking forward for when health inspection, goods certificates, car street safety regulations, finally arrive into the computing industry .
- anonzzzies 3y ago
  But how far would this go? If I throw some weekend project on GitHub, I need to spend time and money on this? Or is it only for commercially sold software? In both cases the large corps win again and it will hamper innovation, real free market, entreneurial spirit etc. Most (all?) of the regulated industries you cannot just get into and compete; it’s expensive and needs resources small companies cannot have. These industries are already regulated; maybe the software (which is also already regulated) needs stricter controls, but we already have this. Why would we need more? If you are in an industry, healthcare for instance, and you use a library with memory issues; that falls under the regulations of the company creating the software directly for the healthcare provider(s). Why is more needed?
  - pjmlp 3y ago
    Yes, so what? The little guy that wants to sell food on the street, needs to have same rules as the 5 star michelin restaurant in regards on hygine. At least in developed countries. And yet they manage to create their business, go figure. More is needed, because people have been educated to accept to pay for garbage software, in terms of quality. In all other industries, most people will return on the same day products they have bought, with comparable bad quality to such software.

scrame 3y ago

TIL the register is still around. good for them!

slaymaker1907 3y ago

As someone dealing with this stuff, I don’t think it’s even because these bugs are the most frequent, it’s just that these are easy to both detect (for the exploiter) and exploit.

kazinator 3y ago

Selection bias at work. Bugs in this category are likely to be serious enough to prompt CVE's. If CVE's are what your'e sampling in order go get an idea of what bugs are out there, memory overwrite bugs will be highly represented.