4 ms·

To be blunt, I don't think that this really matters very much. The reason is that if you used Pascal or Modula-2 , both of which use "counted strings" you would

To be blunt, I don't think that this really matters very much. The reason is that if you used Pascal or Modula-2 , both of which use "counted strings" you would very likely have the same kind of safety. I think the potential of Rust is greatly oversold, because it only ever gets compared to known-unsafe cases, such as string handling in C; there are plenty of other languages that handle this safely out of the box. And Modula-2 is in the GCC 13 compiler and FreePascal has been out for over a decade at this point; both languages are far more compact and have formal specifications...

aaomidi 3y ago
80% of bugs come back to memory safety… don’t think it’s overselling it. Sure other language can do better here too, but you’re not just getting memory safety from Rust.
- lelanthran 3y ago
  > 80% of bugs come back to memory safety… don’t think it’s overselling it. I think you are misremembering the statistic. It's 70% of security bugs that are related to memory safety: https://www.zdnet.com/article/chrome-70-of-all-security-bugs-are-memory-safety-issues/ https://www.zdnet.com/article/chrome-70-of-all-security-bugs... The way you say it, it means that out of every 100 bugs, 80 are due to memory safety. The reality is that out of every 100 security bugs, 70 are due to memory safety. For example, a codebase with a thousand bugs might only have 10 that are security bugs, of which 7 are due to memory safety. You imply that a codebase with a thousand bugs have 800 memory safety issues.
  - majewsky 3y ago
    The framing in the last two lines only makes sense if you consider the amount of total bugs as a prior. In practice, the more realistic prior is something that's more observable, e.g. the number or rate of published CVEs or of security breaches. If something like that can be reduced by 70%, that's much more significant than your framing of "0.7% of all bugs in this example are security bugs due to memory safety" makes it out to be.
    - c_crank 3y ago
      This would be an argument in favor of using a GC'ed language, because that would reduce those bugs by 99.9%.
    - lelanthran 3y ago
      I don't disagree with that, I was disagreeing with OP whose claim implies that out of every 1000 bugs, 800 of them are due to memory issues. OP was wildly off the mark.
  - tialaramex 3y ago
    Bugs mean the software's behaviour is outside what was specified, but for security we're allowed to depend on any aspect of the specified behaviour. So, all bugs are security bugs. Some bugs aren't as important as others, but they're all just bugs.
    - nullc 3y ago
      In security software like sudo you could argue that nearly all bugs are security bugs. But there are plenty of bugs in non-primarily-security software where it's really hard to argue that there is any security implication. In any case the reason for the figure is because when there is a buffer overflow that allows overwriting the stack its almost always a security bug because it usually allows untrusted input to make the program do anything. So of all the kinds of bugs a program can have the memory safety ones are usually security bugs, while other logic bugs may or may not have meaningful security implications. In a suid program that is designed to launch other programs (e.g. sudo), significant logic bugs are very likely to have security implications too. This is supported by sudo's CVE list-- the vast majority appear to be logic bugs with no memory safety involvement. One argument made for rust is that with memory safety "in the bag" developer time can be better invested in other areas. But it isn't safe to assume that: Rust is a very different language and other properties of it such as its far greater complexity than C may make logic bugs more likely. As far as I know the defect rate in rust software has never been seriously studied. It would be a huge relief if there were some serious analysis that could provide some confidence that rust wouldn't make the software quality situation worse. Especially since I'm not convinced that trading memory safety bugs for other bugs alone would be an improvement: At least memory safety flaws are always bugs, so when valgrind, memcheck, stack protector, etc tell us about them during testing or formal methods tell us they exist (which hardly can exist for rust given that the language isn't even formally specified) we always know we have an issue. With general logic bugs it's much harder to be confident that a behavior is a bug. And, of course, memory safety issues can arise in rust, due to compiler/library bugs or because other restrictions or performance issues have caused users to use unsafe or depend on external library code... so it's unclear how much we can really say that memory safety is in-the-bag. I think to be confident that a critical piece of rust code lacks memory safety bugs we still must test comprehensively, including with fuzzing and we must review its dependencies which are almost always far far far far worse than in C (creating a whole issue for sneaking in bad code). Informally, my experience with software written in Rust is that it is far more likely to crash (panic) on the spot when I attempt to use it than software written in C-- usually because the author just hasn't bothered with error handling due of some mixture of the language making it somewhat harder to do, out of a culture that doesn't regard code that panics unexpectedly as shoddy work (because unlike a crash in C, it's unlikely to be the sign of a potential vulnerability), or in some cases because the goal was just "rewrite in rust" and the programmer didn't take much care or pride in their work or lacked the subject matter expertise to do it competently. It's heartening to see this effort has a test suite-- Of course (AFAIK) sudo doesn't have one itself but arguably sudo's test suite is the decades of use it's had in production. A newly written program has a lot more need for a test suite than an established one. I think rust programs are more likely to have some amount of tests compared to really old free software, but usually not enough to compensate for their newness.
woodruffw 3y ago
> I think the potential of Rust is greatly oversold, because it only ever gets compared to known-unsafe cases, such as string handling in C; there are plenty of other languages that handle this safely out of the box. The comparison is apt because of scope and scale: Rust is intended to integrate into and eventually replace C codebases; both Pascal and Modula-2 can integrate with C, but AFAIK do not place substantial emphasis on doing so (nor have substantial amounts of community interest in doing so). Similarly: people keep doing string handling in C, so that is what is going to continue to matter. They also keep doing other things in C that Rust prevents, which are also going to continue to matter. The fact that they're known to be unsafe doesn't change that. Put another way: it's unclear that the interest in Rust (which is immense) would ever translate into equivalent interest in Modula-2, any technical merits aside. The reality is that Rust does these things well, and a large number of people like it and are successfully using it to replace or augment C codebases. Both conditions are necessary.
gpderetta 3y ago
Having developers mindshare is a quality of its own.
nullc 3y ago
Or Ada for that matter! Memory safe languages are not something new, nor are memory safe languages that lack a slow runtime.

aaomidi 3y ago

80% of bugs come back to memory safety… don’t think it’s overselling it. Sure other language can do better here too, but you’re not just getting memory safety from Rust.

lelanthran 3y ago
> 80% of bugs come back to memory safety… don’t think it’s overselling it. I think you are misremembering the statistic. It's 70% of security bugs that are related to memory safety: https://www.zdnet.com/article/chrome-70-of-all-security-bugs-are-memory-safety-issues/ https://www.zdnet.com/article/chrome-70-of-all-security-bugs... The way you say it, it means that out of every 100 bugs, 80 are due to memory safety. The reality is that out of every 100 security bugs, 70 are due to memory safety. For example, a codebase with a thousand bugs might only have 10 that are security bugs, of which 7 are due to memory safety. You imply that a codebase with a thousand bugs have 800 memory safety issues.
- majewsky 3y ago
  The framing in the last two lines only makes sense if you consider the amount of total bugs as a prior. In practice, the more realistic prior is something that's more observable, e.g. the number or rate of published CVEs or of security breaches. If something like that can be reduced by 70%, that's much more significant than your framing of "0.7% of all bugs in this example are security bugs due to memory safety" makes it out to be.
  - c_crank 3y ago
    This would be an argument in favor of using a GC'ed language, because that would reduce those bugs by 99.9%.
  - lelanthran 3y ago
    I don't disagree with that, I was disagreeing with OP whose claim implies that out of every 1000 bugs, 800 of them are due to memory issues. OP was wildly off the mark.
- tialaramex 3y ago
  Bugs mean the software's behaviour is outside what was specified, but for security we're allowed to depend on any aspect of the specified behaviour. So, all bugs are security bugs. Some bugs aren't as important as others, but they're all just bugs.
  - nullc 3y ago
    In security software like sudo you could argue that nearly all bugs are security bugs. But there are plenty of bugs in non-primarily-security software where it's really hard to argue that there is any security implication. In any case the reason for the figure is because when there is a buffer overflow that allows overwriting the stack its almost always a security bug because it usually allows untrusted input to make the program do anything. So of all the kinds of bugs a program can have the memory safety ones are usually security bugs, while other logic bugs may or may not have meaningful security implications. In a suid program that is designed to launch other programs (e.g. sudo), significant logic bugs are very likely to have security implications too. This is supported by sudo's CVE list-- the vast majority appear to be logic bugs with no memory safety involvement. One argument made for rust is that with memory safety "in the bag" developer time can be better invested in other areas. But it isn't safe to assume that: Rust is a very different language and other properties of it such as its far greater complexity than C may make logic bugs more likely. As far as I know the defect rate in rust software has never been seriously studied. It would be a huge relief if there were some serious analysis that could provide some confidence that rust wouldn't make the software quality situation worse. Especially since I'm not convinced that trading memory safety bugs for other bugs alone would be an improvement: At least memory safety flaws are always bugs, so when valgrind, memcheck, stack protector, etc tell us about them during testing or formal methods tell us they exist (which hardly can exist for rust given that the language isn't even formally specified) we always know we have an issue. With general logic bugs it's much harder to be confident that a behavior is a bug. And, of course, memory safety issues can arise in rust, due to compiler/library bugs or because other restrictions or performance issues have caused users to use unsafe or depend on external library code... so it's unclear how much we can really say that memory safety is in-the-bag. I think to be confident that a critical piece of rust code lacks memory safety bugs we still must test comprehensively, including with fuzzing and we must review its dependencies which are almost always far far far far worse than in C (creating a whole issue for sneaking in bad code). Informally, my experience with software written in Rust is that it is far more likely to crash (panic) on the spot when I attempt to use it than software written in C-- usually because the author just hasn't bothered with error handling due of some mixture of the language making it somewhat harder to do, out of a culture that doesn't regard code that panics unexpectedly as shoddy work (because unlike a crash in C, it's unlikely to be the sign of a potential vulnerability), or in some cases because the goal was just "rewrite in rust" and the programmer didn't take much care or pride in their work or lacked the subject matter expertise to do it competently. It's heartening to see this effort has a test suite-- Of course (AFAIK) sudo doesn't have one itself but arguably sudo's test suite is the decades of use it's had in production. A newly written program has a lot more need for a test suite than an established one. I think rust programs are more likely to have some amount of tests compared to really old free software, but usually not enough to compensate for their newness.

woodruffw 3y ago

> I think the potential of Rust is greatly oversold, because it only ever gets compared to known-unsafe cases, such as string handling in C; there are plenty of other languages that handle this safely out of the box. The comparison is apt because of scope and scale: Rust is intended to integrate into and eventually replace C codebases; both Pascal and Modula-2 can integrate with C, but AFAIK do not place substantial emphasis on doing so (nor have substantial amounts of community interest in doing so). Similarly: people keep doing string handling in C, so that is what is going to continue to matter. They also keep doing other things in C that Rust prevents, which are also going to continue to matter. The fact that they're known to be unsafe doesn't change that. Put another way: it's unclear that the interest in Rust (which is immense) would ever translate into equivalent interest in Modula-2, any technical merits aside. The reality is that Rust does these things well, and a large number of people like it and are successfully using it to replace or augment C codebases. Both conditions are necessary.

gpderetta 3y ago

Having developers mindshare is a quality of its own.

nullc 3y ago

Or Ada for that matter! Memory safe languages are not something new, nor are memory safe languages that lack a slow runtime.