7 ms·

Curl Security Audit

283 points by NeutronBoy 10y ago

zenincognito 10y ago
Mario and cure53 team are a bunch of great people. I learned a lot by reading a lot of reports from Cure53 published on their website[1]. Their repos on Github are a timesaver when looking for XSS. Filedescriptor[2] is a part of cure53 and has some amazing reports on hackerone. Mario is as humble as they get in the security scene.Tweet to him or just follow him generally and you will pick up on how to wreck browser apps. [1]https://cure53.de/#publications https://cure53.de/#publications [2]http://innerht.ml/ http://innerht.ml/
jzawodn 10y ago
Awesome to see heavily used (and often embedded) OSS software getting a sponsored audit like this. I wonder the same could happen for sqlite and others. Given the number of vulnerabilities that "state sponsored" folks are likely to know about, this seems like a very useful defense and a way to increase confidence in our building blocks.
- nas 10y ago
  It's great that these bugs have been found and fixed. I'm frightened by this report however. If a well managed open-source project like Curl has this number of pretty serious bugs that can be found by skilled auditors, what hope do we have of securing ourselves against state sponsored attacks? Really no hope, I think. Given the amount code on a modern computer, there must be no shortage of "zero-day" holes they can use. Maybe this is not news to most informed people. It shocks me though. I know the situation is bad, just didn't fully grok how bad.
  - elmigranto 10y ago
    Creating 100% secure software is not realistic. If you can't win, change the game. I'm not saying you should give up or that audits are a waste. They are a good thing, though you should also work on other approaches to the problem besides "work harder on security bugs".
  - perlgeek 10y ago
    This is one of the reasons that people are excited about Rust; it promises to eliminate a whole class of security bugs that are often found in C applications and libraries. Without having to resort to managed code.
    - petters 10y ago
      Completely agree. The first issue in the report is about reading lines from a file. Those kinds of bugs can and should be eliminated for good.
    - pjmlp 10y ago
      > Without having to resort to managed code. Which was already partially possible in the last decades by the lines of Algol family of languages starting on the Mesa/Cedar and Ada branches. What is more exciting about Rust is that a new generation of coders is rediscovering the ways of system programming before C took over the IT industry. Also how their efforts to adopt Cyclone regions are influencing Swift, C++, D, ParaSail and Pony designs.
    - cesarb 10y ago
      > Without having to resort to managed code. Even better: gradually replacing parts of the current C code with Rust code is possible, while keeping the same API and ABI. IIRC, someone is already trying it with librsvg.
      - steveklabnik 10y ago
        https://people.gnome.org/~federico/news-2016-10.html#25 https://people.gnome.org/~federico/news-2016-10.html#25
  - laumars 10y ago
    While you are right, you can approach security from graded risk perspective. eg non-validated user input on an internet facing interface is going to be a greater risk than a buffer overflow bug in cron (for example). So when hardening your systems the first thing you do is limit the amount of access the outside world has to your OS and application and ansure that all components that do access the outside world are up-to-date (curl is one of those libraries which would communicate to the outside world). So firewalling, OS updates, decent bit lenghs on your encryption keys, etc. Then you ensure all your applications are sandboxed so even if they are exploited the scope of data they can do is limited. But as you said, a sufficiently competent and determined hacker will likely find a way into many a "secure" system. But if you can limit the code that is exposed to the internet then you at least limit the amount of code that can be exploited (or rather force an attacker to use more than one attack in conjunction to gain system access). However if an attacker has physical access to your system then I'd just give up now as you've already lost.
  - tptacek 10y ago
    There's pretty much no good reason that the most widely used command line HTTP fetcher should be a giant C project.
    - rini17 10y ago
      For libcurl - easy linking to any C/C++ code is very good reason. And standalone curl is lighweight with minimal system dependencies.
      - tptacek 10y ago
        I understand libcurl well (I've used it before on projects), and don't dispute its utility. I also understand the value of a very small C-based HTTP fetcher for system bootstrapping (I'm not sure that's really what curl is). What I don't get is why the most widely used general purpose command line HTTP fetcher is a giant C program that we're still finding UAFs in in 2016. I respect the enormous effort that goes into maintaining curl (though I have my differences with the way security is handled). But it's past time curl is replaced by a Rust or Go program.
        tomcam 10y ago
        You're a pretty smart guy. Why not just knock out that little baby yourself?
        tptacek 10y ago
        I'm hoping someone else already is.
        JoachimSchipper 10y ago
        You might want to consider OpenBSD's ftp. It does a lot more than FTP.
        shakna 10y ago
        A statically linked curl for ARM64 is 205kb. A statically linked hello world in Rust for ARM64 is 680kb. Curl is used for a ton of embedded hardware, and is probably in your router. Until Rust can solve its massive binaries, that is somewhere on their roadmap, it just isn't appropriate for use in the embedded world, where kilobytes still count.
- deleted 10y ago
  [deleted]
- abecedarius 10y ago
  I hope we see much more of this. https://www.opentech.fund/ https://www.opentech.fund/ funds open source security audits, or did a few years ago when I worked on some of them. They focused on problems like private chat and secure drops for whistleblowers. On a smaller scale, https://defuse.ca/ https://defuse.ca/ set a good example with some one-person audits (under Research >> Audits on the site). Nothing's stopping anyone from learning by doing, the way most of us learned to program.
sapphire_tomb 10y ago
I just want to thank Daniel for volunteering his project for this level of scrutiny, and then putting in the very long hours to address the results of that process. I know software development is sometimes a thankless task, and I know he got some rather heavy handed response from Apple when he announced all these fixes at such "short notice".
- sleepychu 10y ago
  Can you provide more info on the Apple thing? Can't imagine the stance of a company using a library (for free?) to the library author that would allow any sort of offence as the "short notice".
dorianm 10y ago
Very impressive results from Cure53: https://cure53.de/pentest-report_curl.pdf https://cure53.de/pentest-report_curl.pdf CRL -01-001 Malicious server can inject cookies for other servers ( Medium) CRL -01-002 ConnectionExists () compares passwords with strequal () ( Medium) CRL -01-005 OOB write via unchecked multiplication in base 64_ encode () ( High) CRL -01-007 Double - free in aprintf () via unsafe size _t multiplication ( Medium) CRL -01-009 Double - free in krb 5 read _ data () due to missing realloc () check ( High) CRL -01-011 FTPS TLS session reuse ( Low) CRL -01-013 Heap overflow via integer truncation ( Medium) CRL -01-014 Negative array index via integer overflow in unescape _ word () ( High) CRL -01-021 UAF via insufficient locking for shared cookies ( High) Miscellaneous Issues CRL -01-003 Ambiguity in curl _ easy _ escape () argument ( Low) CRL -01-004 Metalink provides an oracle ( Info) CRL -01-006 Potentially unsafe size _t multiplications ( Medium) CRL -01-008 % n is supported in format strings ( Low) CRL -01-010 Slashes and .. are decoded in file URIs ( Low) CRL -01-012 Only the md 5 of the SSH host key fingerprint is checked CRL -01-015 Default Compile - time options lack support for PIE and RELRO ( Low) CRL -01-016 Unchecked snprintf () calls ( Low) CRL -01-017 Permit disabling ( insecure ) fallbacks ( Low) CRL -01-018 Null pointer dereference in the RTSP protocol ( Low) CRL -01-019 nss _ init _ sslver uses version info from NSS header ( Info) CRL -01-020 dup _ nickname () doesn't check for memory allocation failure ( Low) CRL -01-022 polarssl _ connect _ step 1() lacks matching unlock ( Info) CRL -01-023 ssl _ thread _ setup () leaves mutex buffer partially uninitialised ( Info)
72deluxe 10y ago
Very interesting, the report PDF makes for interesting and informative reading.
throwbsidbdk 10y ago
I've consistently wondered why cURL is so popular in languages with a solid web client(almost everything these days) . I've used it for quick hacks but definitely nothing in production.

283 points by NeutronBoy 10y ago

Mario and cure53 team are a bunch of great people. I learned a lot by reading a lot of reports from Cure53 published on their website[1]. Their repos on Github are a timesaver when looking for XSS. Filedescriptor[2] is a part of cure53 and has some amazing reports on hackerone. Mario is as humble as they get in the security scene.Tweet to him or just follow him generally and you will pick up on how to wreck browser apps. [1]https://cure53.de/#publications https://cure53.de/#publications [2]http://innerht.ml/ http://innerht.ml/

jzawodn 10y ago

Awesome to see heavily used (and often embedded) OSS software getting a sponsored audit like this. I wonder the same could happen for sqlite and others. Given the number of vulnerabilities that "state sponsored" folks are likely to know about, this seems like a very useful defense and a way to increase confidence in our building blocks.

nas 10y ago
It's great that these bugs have been found and fixed. I'm frightened by this report however. If a well managed open-source project like Curl has this number of pretty serious bugs that can be found by skilled auditors, what hope do we have of securing ourselves against state sponsored attacks? Really no hope, I think. Given the amount code on a modern computer, there must be no shortage of "zero-day" holes they can use. Maybe this is not news to most informed people. It shocks me though. I know the situation is bad, just didn't fully grok how bad.
- elmigranto 10y ago
  Creating 100% secure software is not realistic. If you can't win, change the game. I'm not saying you should give up or that audits are a waste. They are a good thing, though you should also work on other approaches to the problem besides "work harder on security bugs".
- perlgeek 10y ago
  This is one of the reasons that people are excited about Rust; it promises to eliminate a whole class of security bugs that are often found in C applications and libraries. Without having to resort to managed code.
  - petters 10y ago
    Completely agree. The first issue in the report is about reading lines from a file. Those kinds of bugs can and should be eliminated for good.
  - pjmlp 10y ago
    > Without having to resort to managed code. Which was already partially possible in the last decades by the lines of Algol family of languages starting on the Mesa/Cedar and Ada branches. What is more exciting about Rust is that a new generation of coders is rediscovering the ways of system programming before C took over the IT industry. Also how their efforts to adopt Cyclone regions are influencing Swift, C++, D, ParaSail and Pony designs.
  - cesarb 10y ago
    > Without having to resort to managed code. Even better: gradually replacing parts of the current C code with Rust code is possible, while keeping the same API and ABI. IIRC, someone is already trying it with librsvg.
    - steveklabnik 10y ago
      https://people.gnome.org/~federico/news-2016-10.html#25 https://people.gnome.org/~federico/news-2016-10.html#25
- laumars 10y ago
  While you are right, you can approach security from graded risk perspective. eg non-validated user input on an internet facing interface is going to be a greater risk than a buffer overflow bug in cron (for example). So when hardening your systems the first thing you do is limit the amount of access the outside world has to your OS and application and ansure that all components that do access the outside world are up-to-date (curl is one of those libraries which would communicate to the outside world). So firewalling, OS updates, decent bit lenghs on your encryption keys, etc. Then you ensure all your applications are sandboxed so even if they are exploited the scope of data they can do is limited. But as you said, a sufficiently competent and determined hacker will likely find a way into many a "secure" system. But if you can limit the code that is exposed to the internet then you at least limit the amount of code that can be exploited (or rather force an attacker to use more than one attack in conjunction to gain system access). However if an attacker has physical access to your system then I'd just give up now as you've already lost.
- tptacek 10y ago
  There's pretty much no good reason that the most widely used command line HTTP fetcher should be a giant C project.
  - rini17 10y ago
    For libcurl - easy linking to any C/C++ code is very good reason. And standalone curl is lighweight with minimal system dependencies.
    - tptacek 10y ago
      I understand libcurl well (I've used it before on projects), and don't dispute its utility. I also understand the value of a very small C-based HTTP fetcher for system bootstrapping (I'm not sure that's really what curl is). What I don't get is why the most widely used general purpose command line HTTP fetcher is a giant C program that we're still finding UAFs in in 2016. I respect the enormous effort that goes into maintaining curl (though I have my differences with the way security is handled). But it's past time curl is replaced by a Rust or Go program.
      - tomcam 10y ago
        You're a pretty smart guy. Why not just knock out that little baby yourself?
        tptacek 10y ago
        I'm hoping someone else already is.
        JoachimSchipper 10y ago
        You might want to consider OpenBSD's ftp. It does a lot more than FTP.
      - shakna 10y ago
        A statically linked curl for ARM64 is 205kb. A statically linked hello world in Rust for ARM64 is 680kb. Curl is used for a ton of embedded hardware, and is probably in your router. Until Rust can solve its massive binaries, that is somewhere on their roadmap, it just isn't appropriate for use in the embedded world, where kilobytes still count.
deleted 10y ago
[deleted]
abecedarius 10y ago
I hope we see much more of this. https://www.opentech.fund/ https://www.opentech.fund/ funds open source security audits, or did a few years ago when I worked on some of them. They focused on problems like private chat and secure drops for whistleblowers. On a smaller scale, https://defuse.ca/ https://defuse.ca/ set a good example with some one-person audits (under Research >> Audits on the site). Nothing's stopping anyone from learning by doing, the way most of us learned to program.

sapphire_tomb 10y ago

I just want to thank Daniel for volunteering his project for this level of scrutiny, and then putting in the very long hours to address the results of that process. I know software development is sometimes a thankless task, and I know he got some rather heavy handed response from Apple when he announced all these fixes at such "short notice".

sleepychu 10y ago
Can you provide more info on the Apple thing? Can't imagine the stance of a company using a library (for free?) to the library author that would allow any sort of offence as the "short notice".

dorianm 10y ago

Very impressive results from Cure53: https://cure53.de/pentest-report_curl.pdf https://cure53.de/pentest-report_curl.pdf CRL -01-001 Malicious server can inject cookies for other servers ( Medium) CRL -01-002 ConnectionExists () compares passwords with strequal () ( Medium) CRL -01-005 OOB write via unchecked multiplication in base 64_ encode () ( High) CRL -01-007 Double - free in aprintf () via unsafe size _t multiplication ( Medium) CRL -01-009 Double - free in krb 5 read _ data () due to missing realloc () check ( High) CRL -01-011 FTPS TLS session reuse ( Low) CRL -01-013 Heap overflow via integer truncation ( Medium) CRL -01-014 Negative array index via integer overflow in unescape _ word () ( High) CRL -01-021 UAF via insufficient locking for shared cookies ( High) Miscellaneous Issues CRL -01-003 Ambiguity in curl _ easy _ escape () argument ( Low) CRL -01-004 Metalink provides an oracle ( Info) CRL -01-006 Potentially unsafe size _t multiplications ( Medium) CRL -01-008 % n is supported in format strings ( Low) CRL -01-010 Slashes and .. are decoded in file URIs ( Low) CRL -01-012 Only the md 5 of the SSH host key fingerprint is checked CRL -01-015 Default Compile - time options lack support for PIE and RELRO ( Low) CRL -01-016 Unchecked snprintf () calls ( Low) CRL -01-017 Permit disabling ( insecure ) fallbacks ( Low) CRL -01-018 Null pointer dereference in the RTSP protocol ( Low) CRL -01-019 nss _ init _ sslver uses version info from NSS header ( Info) CRL -01-020 dup _ nickname () doesn't check for memory allocation failure ( Low) CRL -01-022 polarssl _ connect _ step 1() lacks matching unlock ( Info) CRL -01-023 ssl _ thread _ setup () leaves mutex buffer partially uninitialised ( Info)

72deluxe 10y ago

Very interesting, the report PDF makes for interesting and informative reading.

throwbsidbdk 10y ago

I've consistently wondered why cURL is so popular in languages with a solid web client(almost everything these days) . I've used it for quick hacks but definitely nothing in production.