8 ms·

Curl is C (2017)

85 points by taf2 6y ago

scaladev 6y ago
>C is not the primary reason for our past vulnerabilities As much as I respect Daniel for his work on curl, this claim is simply bullshit. https://msrc-blog.microsoft.com/2019/07/18/we-need-a-safer-systems-programming-language/ https://msrc-blog.microsoft.com/2019/07/18/we-need-a-safer-s... > ~70% of the vulnerabilities Microsoft assigns a CVE each year continue to be memory safety issues If you don't trust Microsoft or think their developers are incompetent, similar figures can be found in other studies.
- maeln 6y ago
  I think he was talking in the context of curl.
- jkrems 6y ago
  > As much as I respect Daniel for his work on curl, this claim is simply bullshit. He was talking about "our" in the context of curl, so clearly talking about curl itself. Claiming that the claim is bullshit based on a blog post by Microsoft about a completely unrelated sample set is... not great methodology and certainly not worthy of using the "BS" stamp.
- AshamedCaptain 6y ago
  I do have a problem with people using "number of CVEs" as a metric for anything...
- exyi 6y ago
  He is talking about curl, not all software. But the claim is still BS as debunked 4 years ago by simias: https://news.ycombinator.com/item?id=13966967 https://news.ycombinator.com/item?id=13966967 7 out of 11 are pretty clearly caused by C being an unsafe language. When I look at https://curl.se/docs/security.html https://curl.se/docs/security.html today, it does not seem to be 70%, but there is still quite a big new things called "double-free", "buffer overflow", ...
  - Snetry 6y ago
    I can't speak for the buffer overflows since those can happen in various ways but I'd sort double frees under logic errors which he admits curl to have
maeln 6y ago
(2017) Previous discussion: https://news.ycombinator.com/item?id=13966241 https://news.ycombinator.com/item?id=13966241
erickt 6y ago
Could (2017) be added to the title? Since then, Daniel has since rethought this post and is now writing a Rust backend for curl: https://daniel.haxx.se/blog/2020/10/09/rust-in-curl-with-hyper/ https://daniel.haxx.se/blog/2020/10/09/rust-in-curl-with-hyp...
- interfixus 6y ago
  Thank you for the link. In which Daniel Stenberg says: "A rewrite of curl to another language is not considered".
  - geofft 6y ago
    I'm reading it as saying it's not considered because it's a massive task and difficult to get it right, not because it wouldn't be worth doing.
    - bch 6y ago
      In this case, that’s pretty much the definition of “not worth doing”.
- enriquto 6y ago
  this is a frontend, not a rewrite
  - remram 6y ago
    You mean a backend?
deleted 6y ago
[deleted]
jitl 6y ago
Obligatory response with an analysis of memory vulnerabilities found in Curl: https://timmmm.github.io/curl-vulnerabilities-rust/ https://timmmm.github.io/curl-vulnerabilities-rust/ Discussion: https://news.ycombinator.com/item?id=25805576 https://news.ycombinator.com/item?id=25805576
- fulafel 6y ago
  Money quote from that article: "Results There are 95 bugs. By my count Rust would have prevented 53 of these." (Unfortunate that the parent comment is so low in the comments, as it's a convincing proven refutation of the main security argument of the "Curl is C" posting)
dale_glass 6y ago
I think something that is often missing in such posts is the appalling attitude towards undefined behavior in C compilers. Yes, okay. You've been careful. You've checked you're not accidentally overflowing a buffer, traced things with valgrind, all that good stuff. None of that stops the compiler from doing random bullshit such as this in the search for better optimization: https://gcc.godbolt.org/#%7B%22version%22%3A3%2C%22filterAsm%22%3A%7B%22labels%22%3Atrue%2C%22directives%22%3Atrue%2C%22commentOnly%22%3Atrue%7D%2C%22compilers%22%3A%5B%7B%22sourcez%22%3A%22MQSwdgxgNgrgJgUwAQB4IGcAucogEYB8AUEZgJ4AOCiAZkuJkgBQBUAYjJJiAPZgCUTfgG4SWAIbcISDl15gkAER6iiEqfTCMAogCdx6BAEEoUIUgDeRJEl0JMMXQvRksCALZMARLvdIAtLp0APReIkQAviQAbjwgcEgAcgjRCLoAwuKm1OZWNspIALxIegbGpsI2kSQMSO7i4LnWtvaOCspCohFAA%3D%3D%22%2C%22compiler%22%3A%22%2Fopt%2Fclang%2Bllvm-3.4.1-x86_64-unknown-ubuntu12.04%2Fbin%2Fclang%2B%2B%22%2C%22options%22%3A%22-Os%20-std%3Dc%2B%2B11%20-Wall%22%7D%5D%7D https://gcc.godbolt.org/#%7B%22version%22%3A3%2C%22filterAsm... Personally, as a programmer I dread that sort of thing. I want code to work, not to try to figure out how a compiler could interpret my code in the most twisted way possible, like some sort of evil genie. I think that's one area where modern compilers are going wrong -- fine combing the spec for loopholes in the attempt to optimize better results in code being compiled into something nobody in their right mind would have intended. Edit: For extra fun, try adding a std::cout to NeverCalled() -- it gets even more interesting.
- jstanley 6y ago
  Out of interest, what did you intend for the code you linked to do? You're calling an uninitialised function pointer. It could point to anything. EDIT: Actually, I was wrong. The function pointer should be null.
  - AshamedCaptain 6y ago
    Calling a NULL pointer is still undefined behavior.
    - jstanley 6y ago
      Sure, but "everyone knows" what happens when you call a null pointer, and it isn't system("rm -rf /"). This is the point GP was making.
kleiba 6y ago
Would Rust secure cURL? https://timmmm.github.io/curl-vulnerabilities-rust/ https://timmmm.github.io/curl-vulnerabilities-rust/
- eqvinox 6y ago
  This fails to consider the likelyhood of new logic bugs being introduced in any rewrite that didn't exist in the previous code. Unfortunately, I don't think there's good numbers available to estimate any numbers on these. But judging by the huge array of topics listed with bugs there (TFTP, NTLM, IMAP, Krb5, ...) I'm gonna say it's not possible to make a statement either way on whether the balance would be overall positive or negative. FWIW we have the same problem with a large 1996-started C codebase with a lot of logic complexity (routing protocols.) Our best guess - amplified by the fact that we don't do a lot of complex string/memory ops, but rather complicated algorithms - is that a rewrite would introduce worse bugs. But I'm not sure on that either. (Would be a nice MSc or PhD thesis topic...)
ho_schi 6y ago
The plain view from Steinberg is alleviating. C - and also C++ - are reliable tools for system and application programming. I think Rust is interesting and I see how C++ develops even further covering a bewildering broad range of low- and high-level programming features. Jumping on the next buzzword language or promising tool? No thanks. Languages like Java, C#, JavaScript or even Go are not itself secure but tools with different features. And another sort of own issues. You will face stack overflows, one-by-offs, missed checks and a lot of bad exception handling and a mad garbage-collector. This doesn't even cover most of our own logic programming mistakes - the most important category. Often people don't know how their tool actually works. What I've learned, good results are only achieved with much and steady work. And well designed APIs prevent problems. Regarding tools, one big improvement is the address-sanitizer in GCC and LLVM. I'm so thankful for this :)
junon 6y ago
It's pretty clear at this point that HN collectively tends toward the "C hating" camp. It's a sad state of affairs.
ajsnigrutin 6y ago
C is great. I never understood some peoples' desires to rewrite old stuff into new languages, just because it's a new language. X exists... and HN is then full of "X in python", "X in python3", "X in rust", "X in ruby", "X in go", "X in perl6" ... why? Especially with "new" languages, that lose popularity after a few years (remember all the ruby (-on rails,...) hype?) or languages that change with versions, and running old code in a new interpreter doesn't work (fscking python, and a bunch of python2.x code online that won't run on modern systems anymore, without a lot of work.... i have 20 year old perl books, and all the examples still work, even older C books, and everything works... but not python, "print foo" breaks in pyton3, not to mention integer divison that you don't even notice it failed, until something goes horribly wrong). curl is written in C and JustWorks(TM), it doesn't need to be written in anything else. </rant>
- cbdumas 6y ago
  The stability and ubiquity of C are probably it's biggest upside characteristics and, as you say, there is a lot to like there. But you have to admit the language is showing it's age and has some huge flaws. To call manual memory management a foot-gun would be a ridiculous understatement. Once you've used a language with modules and namespaces, going back to C is so painful. The C standard library is super minimal and has lots of... lets call them... quirks. Basically lots of things that are easy and safe in any of the languages you mentioned are difficult and dangerous in C.
- randmeerkat 6y ago
  C is difficult to learn and the largest community to get involved in / learn from is the kernel dev community, which is notoriously hostile to newbs. I suspect if the kernel community was more friendly (less hostile), and mentorship was more common in the C community, you would see less people searching for an alternative to C.
  - cozzyd 6y ago
    C is very easy to learn! It's difficult to use in some cases (e.g. string handling), but it's the right tool for a lot domains.
kouzant 6y ago
I would also add, some people suggesting to rewrite curl are younger than curl
djhaskin987 6y ago
Best part of the article for me: > C is not the primary reason for our past vulnerabilities > There. The simple fact is that most of our past vulnerabilities happened because of logical mistakes in the code. Logical mistakes that aren’t really language bound and they would not be fixed simply by changing language. Moving to Rust for new projects seems like a good idea, but I just had an experience moving from Clojure back to Java for a new project recently that helped me realize something: it's not about the language. When I moved back to Java, it wasn't about the language anymore. It was about building something cool. There's always this subtext to any programming I did in Clojure that was like "you're doing this cool thing in a cool language and that makes it better", but in java it was "hey let's make something really cool". That's what this guy is saying the most, I think. It's not about the language. It's about building something rock-solid, and rewrites aren't usually the answer when that's what it's about.

Curl is C (2017)

85 points by taf2 6y ago

>C is not the primary reason for our past vulnerabilities As much as I respect Daniel for his work on curl, this claim is simply bullshit. https://msrc-blog.microsoft.com/2019/07/18/we-need-a-safer-systems-programming-language/ https://msrc-blog.microsoft.com/2019/07/18/we-need-a-safer-s... > ~70% of the vulnerabilities Microsoft assigns a CVE each year continue to be memory safety issues If you don't trust Microsoft or think their developers are incompetent, similar figures can be found in other studies.

maeln 6y ago
I think he was talking in the context of curl.
jkrems 6y ago
> As much as I respect Daniel for his work on curl, this claim is simply bullshit. He was talking about "our" in the context of curl, so clearly talking about curl itself. Claiming that the claim is bullshit based on a blog post by Microsoft about a completely unrelated sample set is... not great methodology and certainly not worthy of using the "BS" stamp.
AshamedCaptain 6y ago
I do have a problem with people using "number of CVEs" as a metric for anything...
exyi 6y ago
He is talking about curl, not all software. But the claim is still BS as debunked 4 years ago by simias: https://news.ycombinator.com/item?id=13966967 https://news.ycombinator.com/item?id=13966967 7 out of 11 are pretty clearly caused by C being an unsafe language. When I look at https://curl.se/docs/security.html https://curl.se/docs/security.html today, it does not seem to be 70%, but there is still quite a big new things called "double-free", "buffer overflow", ...
- Snetry 6y ago
  I can't speak for the buffer overflows since those can happen in various ways but I'd sort double frees under logic errors which he admits curl to have

maeln 6y ago

(2017) Previous discussion: https://news.ycombinator.com/item?id=13966241 https://news.ycombinator.com/item?id=13966241

erickt 6y ago

Could (2017) be added to the title? Since then, Daniel has since rethought this post and is now writing a Rust backend for curl: https://daniel.haxx.se/blog/2020/10/09/rust-in-curl-with-hyper/ https://daniel.haxx.se/blog/2020/10/09/rust-in-curl-with-hyp...

interfixus 6y ago
Thank you for the link. In which Daniel Stenberg says: "A rewrite of curl to another language is not considered".
- geofft 6y ago
  I'm reading it as saying it's not considered because it's a massive task and difficult to get it right, not because it wouldn't be worth doing.
  - bch 6y ago
    In this case, that’s pretty much the definition of “not worth doing”.
enriquto 6y ago
this is a frontend, not a rewrite
- remram 6y ago
  You mean a backend?

deleted 6y ago

[deleted]

jitl 6y ago

Obligatory response with an analysis of memory vulnerabilities found in Curl: https://timmmm.github.io/curl-vulnerabilities-rust/ https://timmmm.github.io/curl-vulnerabilities-rust/ Discussion: https://news.ycombinator.com/item?id=25805576 https://news.ycombinator.com/item?id=25805576

fulafel 6y ago
Money quote from that article: "Results There are 95 bugs. By my count Rust would have prevented 53 of these." (Unfortunate that the parent comment is so low in the comments, as it's a convincing proven refutation of the main security argument of the "Curl is C" posting)

dale_glass 6y ago

I think something that is often missing in such posts is the appalling attitude towards undefined behavior in C compilers. Yes, okay. You've been careful. You've checked you're not accidentally overflowing a buffer, traced things with valgrind, all that good stuff. None of that stops the compiler from doing random bullshit such as this in the search for better optimization: https://gcc.godbolt.org/#%7B%22version%22%3A3%2C%22filterAsm%22%3A%7B%22labels%22%3Atrue%2C%22directives%22%3Atrue%2C%22commentOnly%22%3Atrue%7D%2C%22compilers%22%3A%5B%7B%22sourcez%22%3A%22MQSwdgxgNgrgJgUwAQB4IGcAucogEYB8AUEZgJ4AOCiAZkuJkgBQBUAYjJJiAPZgCUTfgG4SWAIbcISDl15gkAER6iiEqfTCMAogCdx6BAEEoUIUgDeRJEl0JMMXQvRksCALZMARLvdIAtLp0APReIkQAviQAbjwgcEgAcgjRCLoAwuKm1OZWNspIALxIegbGpsI2kSQMSO7i4LnWtvaOCspCohFAA%3D%3D%22%2C%22compiler%22%3A%22%2Fopt%2Fclang%2Bllvm-3.4.1-x86_64-unknown-ubuntu12.04%2Fbin%2Fclang%2B%2B%22%2C%22options%22%3A%22-Os%20-std%3Dc%2B%2B11%20-Wall%22%7D%5D%7D https://gcc.godbolt.org/#%7B%22version%22%3A3%2C%22filterAsm... Personally, as a programmer I dread that sort of thing. I want code to work, not to try to figure out how a compiler could interpret my code in the most twisted way possible, like some sort of evil genie. I think that's one area where modern compilers are going wrong -- fine combing the spec for loopholes in the attempt to optimize better results in code being compiled into something nobody in their right mind would have intended. Edit: For extra fun, try adding a std::cout to NeverCalled() -- it gets even more interesting.

jstanley 6y ago
Out of interest, what did you intend for the code you linked to do? You're calling an uninitialised function pointer. It could point to anything. EDIT: Actually, I was wrong. The function pointer should be null.
- AshamedCaptain 6y ago
  Calling a NULL pointer is still undefined behavior.
  - jstanley 6y ago
    Sure, but "everyone knows" what happens when you call a null pointer, and it isn't system("rm -rf /"). This is the point GP was making.

kleiba 6y ago

Would Rust secure cURL? https://timmmm.github.io/curl-vulnerabilities-rust/ https://timmmm.github.io/curl-vulnerabilities-rust/

eqvinox 6y ago
This fails to consider the likelyhood of new logic bugs being introduced in any rewrite that didn't exist in the previous code. Unfortunately, I don't think there's good numbers available to estimate any numbers on these. But judging by the huge array of topics listed with bugs there (TFTP, NTLM, IMAP, Krb5, ...) I'm gonna say it's not possible to make a statement either way on whether the balance would be overall positive or negative. FWIW we have the same problem with a large 1996-started C codebase with a lot of logic complexity (routing protocols.) Our best guess - amplified by the fact that we don't do a lot of complex string/memory ops, but rather complicated algorithms - is that a rewrite would introduce worse bugs. But I'm not sure on that either. (Would be a nice MSc or PhD thesis topic...)

ho_schi 6y ago

The plain view from Steinberg is alleviating. C - and also C++ - are reliable tools for system and application programming. I think Rust is interesting and I see how C++ develops even further covering a bewildering broad range of low- and high-level programming features. Jumping on the next buzzword language or promising tool? No thanks. Languages like Java, C#, JavaScript or even Go are not itself secure but tools with different features. And another sort of own issues. You will face stack overflows, one-by-offs, missed checks and a lot of bad exception handling and a mad garbage-collector. This doesn't even cover most of our own logic programming mistakes - the most important category. Often people don't know how their tool actually works. What I've learned, good results are only achieved with much and steady work. And well designed APIs prevent problems. Regarding tools, one big improvement is the address-sanitizer in GCC and LLVM. I'm so thankful for this :)

junon 6y ago

It's pretty clear at this point that HN collectively tends toward the "C hating" camp. It's a sad state of affairs.

ajsnigrutin 6y ago

C is great. I never understood some peoples' desires to rewrite old stuff into new languages, just because it's a new language. X exists... and HN is then full of "X in python", "X in python3", "X in rust", "X in ruby", "X in go", "X in perl6" ... why? Especially with "new" languages, that lose popularity after a few years (remember all the ruby (-on rails,...) hype?) or languages that change with versions, and running old code in a new interpreter doesn't work (fscking python, and a bunch of python2.x code online that won't run on modern systems anymore, without a lot of work.... i have 20 year old perl books, and all the examples still work, even older C books, and everything works... but not python, "print foo" breaks in pyton3, not to mention integer divison that you don't even notice it failed, until something goes horribly wrong). curl is written in C and JustWorks(TM), it doesn't need to be written in anything else. </rant>

cbdumas 6y ago
The stability and ubiquity of C are probably it's biggest upside characteristics and, as you say, there is a lot to like there. But you have to admit the language is showing it's age and has some huge flaws. To call manual memory management a foot-gun would be a ridiculous understatement. Once you've used a language with modules and namespaces, going back to C is so painful. The C standard library is super minimal and has lots of... lets call them... quirks. Basically lots of things that are easy and safe in any of the languages you mentioned are difficult and dangerous in C.
randmeerkat 6y ago
C is difficult to learn and the largest community to get involved in / learn from is the kernel dev community, which is notoriously hostile to newbs. I suspect if the kernel community was more friendly (less hostile), and mentorship was more common in the C community, you would see less people searching for an alternative to C.
- cozzyd 6y ago
  C is very easy to learn! It's difficult to use in some cases (e.g. string handling), but it's the right tool for a lot domains.

kouzant 6y ago

I would also add, some people suggesting to rewrite curl are younger than curl

djhaskin987 6y ago

Best part of the article for me: > C is not the primary reason for our past vulnerabilities > There. The simple fact is that most of our past vulnerabilities happened because of logical mistakes in the code. Logical mistakes that aren’t really language bound and they would not be fixed simply by changing language. Moving to Rust for new projects seems like a good idea, but I just had an experience moving from Clojure back to Java for a new project recently that helped me realize something: it's not about the language. When I moved back to Java, it wasn't about the language anymore. It was about building something cool. There's always this subtext to any programming I did in Clojure that was like "you're doing this cool thing in a cool language and that makes it better", but in java it was "hey let's make something really cool". That's what this guy is saying the most, I think. It's not about the language. It's about building something rock-solid, and rewrites aren't usually the answer when that's what it's about.