9 ms·

Curl audit: How a joke led to significant findings

159 points by campuscodi 4y ago

gigaflops 4y ago
Memory, memory, memory, and memory again. All 4 of them. Yet everyone’s idea of safety still seems to be “just write bug-free C code, bro”.
- afrcnc 4y ago
  to be fair, cURL was written when defacing websites was considered top-tier hacking
  - woodruffw 4y ago
    Where did you get that from? cURL was started in 1996, which was around the nadir of the first wave of major memory exploit techniques. Lots of people in the security industry made their name with stack and heap exploit techniques that are now considered staid.
    - cmeacham98 4y ago
      What memory safe languages/tools existed in 1996 and still exist today with some meaningful marketshare? Java (barely, and I think it was proprietary at the time)? It seems like "write memory safe C" was the best technique in 1996 if you wanted any semblance of portability/performance.
      - Groxx 4y ago
        I believe their (and this thread's) original point was that even absurdly widely used, studied, and continuously fuzzed C tools written and maintained by many serious experts for decades.... ... still have multiple easily-uncovered memory issues. Nobody writes safe C. Believing otherwise is delusional. This has been known for decades. But many believers still exist.
        an-unknown 4y ago
        > Nobody writes safe C. Believing otherwise is delusional. This has been known for decades. But many believers still exist. This is factually incorrect. You can write safe C code which is even correct according to some specification, as in "guaranteed bug free". This is not what most people do (or even know how to do) though.
        woodruffw 4y ago
        I'm not aware of a generalized safe subset of C. There are lots of compiler extensions and model annotations that offer security improvements, but very few will claim to offer spatial and temporary memory safety in the way that safe programming languages can. There are lots of specification driven C applications, which rely on (1) the specification being bug free, (2) the application of the specification being bug free, and (3) the model or proof assistant for the specification being sound and bug free. Getting all three of those is historically very difficult.
        Groxx 4y ago
        No realistic code that does that exists AFAIK. It may have sections that do it (which is great!), but not a total useful binary. It's a straw-man that's trotted out every time, and it's utterly meaningless beyond academic arguments. Even if it is true, it's not relevant for any practical purposes.
        jamincan 4y ago
        In the FOSDEM presentation about Rust coreutils, they mentioned that security concerns were not a motivating factor for the rewrite - the C implementation has had only 17 CVEs since 2003. https://fosdem.org/2023/schedule/event/rust_coreutils/ https://fosdem.org/2023/schedule/event/rust_coreutils/
        Groxx 4y ago
        That's a very positive sign for the quality of coreutils, but does exactly nothing to assert there are no flaws. And I'll point to CVE-2015-4041 as evidence in my favor - that's a buffer overflow: https://www.cvedetails.com/vulnerability-list/vendor_id-72/product_id-5075/GNU-Coreutils.html https://www.cvedetails.com/vulnerability-list/vendor_id-72/p...
      - woodruffw 4y ago
        I'm not sure what this has to do with my comment: my observation was only that memory corruption was a well understood vulnerability domain in 1996, not that there was a particularly wide variety of alternatives to C or C++.
      - pjmlp 4y ago
        Modula-2, Eiffel, Ada, Object Pascal,...
        deleted 4y ago
        [deleted]
      - eesmith 4y ago
        By 1996 people were building commercial web stores in Python. For example, see "The Real World Case Study: Python in a Commercial Environment" at https://legacy.python.org/workshops/1997-10/proceedings/stein.ps https://legacy.python.org/workshops/1997-10/proceedings/stei... .
        icedchai 4y ago
        Don't forget about Perl!
        eesmith 4y ago
        I didn't want to get involved in what "meaningful marketshare" means with respect to Perl. In the same vein, AOLServer (ca. 1995) used multithreaded Tcl, and Tcl is also still around.
        icedchai 4y ago
        I remember that era, and Perl CGI scripts were everywhere. Tcl was pretty niche. I used it for some personal projects, but never ran into any web stuff that used it.
        Groxx 4y ago
        1995 is when CPAN started (the first "real" module repository, and the unambiguous best one for a very long time), and yeah - Perl was enormously popular then and especially shortly after for "it's not bash and it's not C" purposes. PHP was still new and WordPress didn't yet exist, and Perl was (and is) consistently beating it by a wide margin in performance. For all the code-golf insanity that Perl supports, it's an extremely effective language with a truly stunning degree of both flexibility and performance.
      - rainingmonkey 4y ago
        Lua originated in 1993 and ran anywhere C would. Unfortunately it still lacks meaningful marketshare though.
        pdimitar 4y ago
        I really like Lua but I've mostly needed it to write less maddening shell scripts, and it sadly falls flat when you need parallelism. And I mean an easy and intuitive one -- I know it has multithreading libraries but they produce overly complicated and verbose code. Meanwhile you can just use `parallel` in your shell scripts with very little ceremony. Lua is great in many ways but looking at it in 2023, it's kind of stuck in the past.
      - kjs3 4y ago
        Ada. Common Lisp, depending on how you define memory safety. If you remove the 'still exist today with meaningful market share', in '96 you had more options (e.g. Modula-2/2+/3 family). There were others.
        kazinator 4y ago
        ISO Modula 2 has malloc/free-like routines, which you can easily misuse. https://www.modula2.org/reference/isomodules/isomodule.php?m=STORAGE.DEF https://www.modula2.org/reference/isomodules/isomodule.php?m... It also has pointer arithmetic, casting and sizeof in a SYSTEM module: https://www.modula2.org/reference/isomodules/isomodule.php?m=SYSTEM.DEF https://www.modula2.org/reference/isomodules/isomodule.php?m...
        kjs3 4y ago
        Which you don't have to use. Just like you can turn of Ada checks at compile time, leading to things like the Ariane-5 cockup. And Rust has 'unsafe', which as I understand allows you to do interesting things. So it's probably fair to say they all give you a footgun of some calibre, but you really have to want to take your foot off to use them.
        kazinator 4y ago
        All sorts of useful problem domains need dynamic allocation; you can hardly avoid using it at some point. I'd actually like to see a Modula-2 example of, say, dynamically allocating an array object, which is dynamically sized. Because obviously Modula has better arrays which know their size, which a C program has to express all by itself with size fields and pointers. I worked with Modula-2 quite a lot around 1990 but didn't do anything of the sort. I didn't know anything about any ISO standard or memory allocation functions provided thereby. A cursor search seems to indicate that it didn't exist until 1996; basically years after "peak Modula".
  - pjmlp 4y ago
    Morris worm happened in the 1980's.
- Karellen 4y ago
  > Yet everyone’s idea of safety still seems to be “just write bug-free C code, bro”. Does it though? I think "the industry" has pretty much come around on C being unsafe, and generally not a great choice of language for new projects. But the fact is that there's a lot of C code out there, and we can't just ditch or rewrite it all overnight. In an ideal world, we wouldn't be starting from here. But we are, and it's going to take time to get to where we need to be. Heck, take a look at the recent story about "Considering C99 for curl": https://news.ycombinator.com/item?id=33704054 https://news.ycombinator.com/item?id=33704054 From the linked article: > The slowest of the “big compilers” to adopt C99 was the Microsoft Visual C++ compiler, which did not adopt it properly until 2015 and added more compliance in 2019. A large number of our users/developers are still stuck on older MSVC versions so not even all users of this compiler suite can build C99 programs even today, in late 2022. What are the chances that the kinds of people who are still doing development on systems where the C compiler is from 2015 or earlier, or on a more niche system that Windows that lots of new safe languages haven't been ported to yet (and such ports aren't even on the radar), are going to able to start moving to developing in rust or whatever anytime soon?
  - returningfory2 4y ago
    > Does it though? I think "the industry" has pretty much come around on C being unsafe, and generally not a great choice of language for new projects. If you look at some of the Rust threads on HN right now [0], you find many commenters claiming that there is no reason to replace C with Rust. It could be a vocal minority, but there's still a sense that "not everyone is on board". [0] https://news.ycombinator.com/item?id=34788714 https://news.ycombinator.com/item?id=34788714
    - woodruffw 4y ago
      HN samples the C (and Rust) communities, but I don't think we should take HN's opinions as representative of either.
      - selfmodruntime 4y ago
        This. My indicator on the success of a technology has been the volume of disagreement among the HN crowd. Just look at React / Javascript frameworks. HN will hate on those with a fiery passion, while literal trillions of dollars are generated with them.
- deleted 4y ago
  [deleted]
- intelVISA 4y ago
  To be fair cURL is pretty aged now. I wrote a cURL-like recently and was surprised my ver benched 2x as fast. (though I cheated by not using crappy 3rd party libs)
- deterministic 4y ago
  I maintain very large scale C++ code and have zero memory issues. No memory leaks. No reading/writing after free. No out of bounds access. I achieve this using the many excellent tools available on Linux to find those problems and fixing them.
CharlesW 4y ago
TLDR: We fuzzed something that wasn't previously fuzzed — specifically, CLI arguments — and found some good bugs. Lesson: Don't forget to fuzz all the methods people might use to interface with your software.
- deleted 4y ago
  [deleted]
- mannykannot 4y ago
  ...including environment variables, I would think (though this investigation apparently did not extend to them.)
deleted 4y ago
[deleted]
andrewmcwatters 4y ago
Boy if they only used Rust, then these curl commands that no one would ever enter would have been memory safe.
- woodruffw 4y ago
  `curl` is widely integrated via its CLI: you'll find all kinds of programs (from simple shell scripts to actual compiled binaries) that interact with `curl` through command-line arguments. So the attacker model here isn't "nobody would type this" it's "does your SoHo router or other software shell out to cURL in an attacker-controllable way."
  - verall 4y ago
    If an attacker could control the curl CLI on your SoHo router, couldn't they just direct it to download their evil payload, and then use one of a million more common problems to run the evil? Or directly exfiltrate data through HTTP requests made by curl? I don't think anyone runs curl as root so there is no privilege escalation. Seriously I can't understand the threat model here.
    - woodruffw 4y ago
      There are different scopes: it's not the case that every controllable CLI invocation is also one where an attacker could change the URL entirely. There are cases where they might only be able to change the behavior of a single flag or option, or a subrange of characters in a specific protocol's URLs, etc. I think the realistic attack model here is: 1. SoHo router that has a public accessible endpoint that takes inputs boiling down to "safe" cURL flags; 2. Those flags can actually induce memory corruption; 3. An attacker is able to execute arbitrary remote code in the context of the cURL subprocess through a payload packed into the "safe" flag.
      - deleted 4y ago
        [deleted]
- 4y ago
nayuki 4y ago
[2021-02-12] <Daniel Stenberg> Will I ever rewrite curl in rust? I don't believe in rewrites, no matter which language. I believe in replacing code and fixing components gradually over time. That could mean that we have a curl written mostly in rust in 10 years. Or in 20 years. Or not. -- https://twitter.com/bagder/status/1360131939794042884 https://twitter.com/bagder/status/1360131939794042884 [2023-02-14] <The article> the fuzzer quickly uncovered memory corruption bugs, specifically use-after-free issues, double-free issues, and memory leaks
- selfmodruntime 4y ago
  People arguing against the usage of Rust just because they don‘t „believe“ in its advantages puzzle me. They are usually proven wrong with multiple CVE‘s shortly after. Rust was never expected to be a magic fix-all, but after the C++ committee‘s stubborn refusal of introducing some sense of memory safety into the language in favor of keeping ABI compatibility, it might turn out to be a self fulfilling prophecy.
  - regularjack 4y ago
    He says "I don't believe in rewrites" , he doesn't say "I don't believe in Rust".
    - Groxx 4y ago
      iirc he has been relatively in favor of Rust, but yeah. No need to rewrite, you can swap pieces incrementally, like many enormous, important systems are now doing. The C ABI is essentially fine / good enough, and is nearly the lingua franca of computing.
      - 8n4vidtmkvmk 4y ago
        ...which is all the more reason to not break the ABI, right? Isn't that the only hope of any kind of interop?

159 points by campuscodi 4y ago

Memory, memory, memory, and memory again. All 4 of them. Yet everyone’s idea of safety still seems to be “just write bug-free C code, bro”.

afrcnc 4y ago
to be fair, cURL was written when defacing websites was considered top-tier hacking
- woodruffw 4y ago
  Where did you get that from? cURL was started in 1996, which was around the nadir of the first wave of major memory exploit techniques. Lots of people in the security industry made their name with stack and heap exploit techniques that are now considered staid.
  - cmeacham98 4y ago
    What memory safe languages/tools existed in 1996 and still exist today with some meaningful marketshare? Java (barely, and I think it was proprietary at the time)? It seems like "write memory safe C" was the best technique in 1996 if you wanted any semblance of portability/performance.
    - Groxx 4y ago
      I believe their (and this thread's) original point was that even absurdly widely used, studied, and continuously fuzzed C tools written and maintained by many serious experts for decades.... ... still have multiple easily-uncovered memory issues. Nobody writes safe C. Believing otherwise is delusional. This has been known for decades. But many believers still exist.
      - an-unknown 4y ago
        > Nobody writes safe C. Believing otherwise is delusional. This has been known for decades. But many believers still exist. This is factually incorrect. You can write safe C code which is even correct according to some specification, as in "guaranteed bug free". This is not what most people do (or even know how to do) though.
        woodruffw 4y ago
        I'm not aware of a generalized safe subset of C. There are lots of compiler extensions and model annotations that offer security improvements, but very few will claim to offer spatial and temporary memory safety in the way that safe programming languages can. There are lots of specification driven C applications, which rely on (1) the specification being bug free, (2) the application of the specification being bug free, and (3) the model or proof assistant for the specification being sound and bug free. Getting all three of those is historically very difficult.
        Groxx 4y ago
        No realistic code that does that exists AFAIK. It may have sections that do it (which is great!), but not a total useful binary. It's a straw-man that's trotted out every time, and it's utterly meaningless beyond academic arguments. Even if it is true, it's not relevant for any practical purposes.
        jamincan 4y ago
        In the FOSDEM presentation about Rust coreutils, they mentioned that security concerns were not a motivating factor for the rewrite - the C implementation has had only 17 CVEs since 2003. https://fosdem.org/2023/schedule/event/rust_coreutils/ https://fosdem.org/2023/schedule/event/rust_coreutils/
        Groxx 4y ago
        That's a very positive sign for the quality of coreutils, but does exactly nothing to assert there are no flaws. And I'll point to CVE-2015-4041 as evidence in my favor - that's a buffer overflow: https://www.cvedetails.com/vulnerability-list/vendor_id-72/product_id-5075/GNU-Coreutils.html https://www.cvedetails.com/vulnerability-list/vendor_id-72/p...
    - woodruffw 4y ago
      I'm not sure what this has to do with my comment: my observation was only that memory corruption was a well understood vulnerability domain in 1996, not that there was a particularly wide variety of alternatives to C or C++.
    - pjmlp 4y ago
      Modula-2, Eiffel, Ada, Object Pascal,...
      - deleted 4y ago
        [deleted]
    - eesmith 4y ago
      By 1996 people were building commercial web stores in Python. For example, see "The Real World Case Study: Python in a Commercial Environment" at https://legacy.python.org/workshops/1997-10/proceedings/stein.ps https://legacy.python.org/workshops/1997-10/proceedings/stei... .
      - icedchai 4y ago
        Don't forget about Perl!
        eesmith 4y ago
        I didn't want to get involved in what "meaningful marketshare" means with respect to Perl. In the same vein, AOLServer (ca. 1995) used multithreaded Tcl, and Tcl is also still around.
        icedchai 4y ago
        I remember that era, and Perl CGI scripts were everywhere. Tcl was pretty niche. I used it for some personal projects, but never ran into any web stuff that used it.
        Groxx 4y ago
        1995 is when CPAN started (the first "real" module repository, and the unambiguous best one for a very long time), and yeah - Perl was enormously popular then and especially shortly after for "it's not bash and it's not C" purposes. PHP was still new and WordPress didn't yet exist, and Perl was (and is) consistently beating it by a wide margin in performance. For all the code-golf insanity that Perl supports, it's an extremely effective language with a truly stunning degree of both flexibility and performance.
    - rainingmonkey 4y ago
      Lua originated in 1993 and ran anywhere C would. Unfortunately it still lacks meaningful marketshare though.
      - pdimitar 4y ago
        I really like Lua but I've mostly needed it to write less maddening shell scripts, and it sadly falls flat when you need parallelism. And I mean an easy and intuitive one -- I know it has multithreading libraries but they produce overly complicated and verbose code. Meanwhile you can just use `parallel` in your shell scripts with very little ceremony. Lua is great in many ways but looking at it in 2023, it's kind of stuck in the past.
    - kjs3 4y ago
      Ada. Common Lisp, depending on how you define memory safety. If you remove the 'still exist today with meaningful market share', in '96 you had more options (e.g. Modula-2/2+/3 family). There were others.
      - kazinator 4y ago
        ISO Modula 2 has malloc/free-like routines, which you can easily misuse. https://www.modula2.org/reference/isomodules/isomodule.php?m=STORAGE.DEF https://www.modula2.org/reference/isomodules/isomodule.php?m... It also has pointer arithmetic, casting and sizeof in a SYSTEM module: https://www.modula2.org/reference/isomodules/isomodule.php?m=SYSTEM.DEF https://www.modula2.org/reference/isomodules/isomodule.php?m...
        kjs3 4y ago
        Which you don't have to use. Just like you can turn of Ada checks at compile time, leading to things like the Ariane-5 cockup. And Rust has 'unsafe', which as I understand allows you to do interesting things. So it's probably fair to say they all give you a footgun of some calibre, but you really have to want to take your foot off to use them.
        kazinator 4y ago
        All sorts of useful problem domains need dynamic allocation; you can hardly avoid using it at some point. I'd actually like to see a Modula-2 example of, say, dynamically allocating an array object, which is dynamically sized. Because obviously Modula has better arrays which know their size, which a C program has to express all by itself with size fields and pointers. I worked with Modula-2 quite a lot around 1990 but didn't do anything of the sort. I didn't know anything about any ISO standard or memory allocation functions provided thereby. A cursor search seems to indicate that it didn't exist until 1996; basically years after "peak Modula".
- pjmlp 4y ago
  Morris worm happened in the 1980's.
Karellen 4y ago
> Yet everyone’s idea of safety still seems to be “just write bug-free C code, bro”. Does it though? I think "the industry" has pretty much come around on C being unsafe, and generally not a great choice of language for new projects. But the fact is that there's a lot of C code out there, and we can't just ditch or rewrite it all overnight. In an ideal world, we wouldn't be starting from here. But we are, and it's going to take time to get to where we need to be. Heck, take a look at the recent story about "Considering C99 for curl": https://news.ycombinator.com/item?id=33704054 https://news.ycombinator.com/item?id=33704054 From the linked article: > The slowest of the “big compilers” to adopt C99 was the Microsoft Visual C++ compiler, which did not adopt it properly until 2015 and added more compliance in 2019. A large number of our users/developers are still stuck on older MSVC versions so not even all users of this compiler suite can build C99 programs even today, in late 2022. What are the chances that the kinds of people who are still doing development on systems where the C compiler is from 2015 or earlier, or on a more niche system that Windows that lots of new safe languages haven't been ported to yet (and such ports aren't even on the radar), are going to able to start moving to developing in rust or whatever anytime soon?
- returningfory2 4y ago
  > Does it though? I think "the industry" has pretty much come around on C being unsafe, and generally not a great choice of language for new projects. If you look at some of the Rust threads on HN right now [0], you find many commenters claiming that there is no reason to replace C with Rust. It could be a vocal minority, but there's still a sense that "not everyone is on board". [0] https://news.ycombinator.com/item?id=34788714 https://news.ycombinator.com/item?id=34788714
  - woodruffw 4y ago
    HN samples the C (and Rust) communities, but I don't think we should take HN's opinions as representative of either.
    - selfmodruntime 4y ago
      This. My indicator on the success of a technology has been the volume of disagreement among the HN crowd. Just look at React / Javascript frameworks. HN will hate on those with a fiery passion, while literal trillions of dollars are generated with them.
deleted 4y ago
[deleted]
intelVISA 4y ago
To be fair cURL is pretty aged now. I wrote a cURL-like recently and was surprised my ver benched 2x as fast. (though I cheated by not using crappy 3rd party libs)
deterministic 4y ago
I maintain very large scale C++ code and have zero memory issues. No memory leaks. No reading/writing after free. No out of bounds access. I achieve this using the many excellent tools available on Linux to find those problems and fixing them.

CharlesW 4y ago

TLDR: We fuzzed something that wasn't previously fuzzed — specifically, CLI arguments — and found some good bugs. Lesson: Don't forget to fuzz all the methods people might use to interface with your software.

deleted 4y ago
[deleted]
mannykannot 4y ago
...including environment variables, I would think (though this investigation apparently did not extend to them.)

deleted 4y ago

[deleted]

andrewmcwatters 4y ago

Boy if they only used Rust, then these curl commands that no one would ever enter would have been memory safe.

woodruffw 4y ago
`curl` is widely integrated via its CLI: you'll find all kinds of programs (from simple shell scripts to actual compiled binaries) that interact with `curl` through command-line arguments. So the attacker model here isn't "nobody would type this" it's "does your SoHo router or other software shell out to cURL in an attacker-controllable way."
- verall 4y ago
  If an attacker could control the curl CLI on your SoHo router, couldn't they just direct it to download their evil payload, and then use one of a million more common problems to run the evil? Or directly exfiltrate data through HTTP requests made by curl? I don't think anyone runs curl as root so there is no privilege escalation. Seriously I can't understand the threat model here.
  - woodruffw 4y ago
    There are different scopes: it's not the case that every controllable CLI invocation is also one where an attacker could change the URL entirely. There are cases where they might only be able to change the behavior of a single flag or option, or a subrange of characters in a specific protocol's URLs, etc. I think the realistic attack model here is: 1. SoHo router that has a public accessible endpoint that takes inputs boiling down to "safe" cURL flags; 2. Those flags can actually induce memory corruption; 3. An attacker is able to execute arbitrary remote code in the context of the cURL subprocess through a payload packed into the "safe" flag.
    - deleted 4y ago
      [deleted]
4y ago

nayuki 4y ago

[2021-02-12] <Daniel Stenberg> Will I ever rewrite curl in rust? I don't believe in rewrites, no matter which language. I believe in replacing code and fixing components gradually over time. That could mean that we have a curl written mostly in rust in 10 years. Or in 20 years. Or not. -- https://twitter.com/bagder/status/1360131939794042884 https://twitter.com/bagder/status/1360131939794042884 [2023-02-14] <The article> the fuzzer quickly uncovered memory corruption bugs, specifically use-after-free issues, double-free issues, and memory leaks

selfmodruntime 4y ago
People arguing against the usage of Rust just because they don‘t „believe“ in its advantages puzzle me. They are usually proven wrong with multiple CVE‘s shortly after. Rust was never expected to be a magic fix-all, but after the C++ committee‘s stubborn refusal of introducing some sense of memory safety into the language in favor of keeping ABI compatibility, it might turn out to be a self fulfilling prophecy.
- regularjack 4y ago
  He says "I don't believe in rewrites" , he doesn't say "I don't believe in Rust".
  - Groxx 4y ago
    iirc he has been relatively in favor of Rust, but yeah. No need to rewrite, you can swap pieces incrementally, like many enormous, important systems are now doing. The C ABI is essentially fine / good enough, and is nearly the lingua franca of computing.
    - 8n4vidtmkvmk 4y ago
      ...which is all the more reason to not break the ABI, right? Isn't that the only hope of any kind of interop?