10 ms·

This article is super overcomplicated. All it had to say was that rust tells you when you keep reference to on stack variable after it goes out of scope. Contex

by shuger 4y ago

This article is super overcomplicated. All it had to say was that rust tells you when you keep reference to on stack variable after it goes out of scope. Context provided adds nothing. I must say - as someone who doesn't use rust - I haven't had this type of issue in years, and when I did it wasn't hard to debug. You get corrupted data, set data breakpoint and in the provided example you will see it being modified by unrelated operations on stack. From there there is only one conclusion. Authors reactions seems to be a bit exaggerated.

renox 4y ago
I agree with you on the first point, but I totally disagree with you on the second point: sure if you know how to reliably reproduce an issue it isn't complicated to debug, but this kind of issue can be difficult to reproduce, can create silent issues..
atoav 4y ago
It all depends on the complexity of your application. The type of guarantees Rust offers in my experience becomes exponentially more useful as your application grows in size.
- lelanthran 4y ago
  > It all depends on the complexity of your application. The type of guarantees Rust offers in my experience becomes exponentially more useful as your application grows in size. Sure, but the development pattern that get used for larger applications typically don't benefit from Rust's additional safety; for one you're going to be using a garbage collected language.
  - MaulingMonkey 4y ago
    > for one you're going to be using a garbage collected language. I wish it were true, but promise you it is not. As a counterpoint, I point to most "AAA" gamedev and OS development. In gamedev it's even a bit flipped: The smaller indie gamedevs can pay the GC hit for Unity's C#, web JS, actionscript flash back in the day, etc. - not much working data, not much garbage. Larger scale titles start missing vsync and having horrible stuttering when GCs are thrown into the mix too brazenly - they're still used on smaller scales (embedding browser tech for UI, limited scope scripting, etc.) but they have a lot of native, non-GCed code.
- MaulingMonkey 4y ago
  This indeed. Small local console app running on trusted data? Maybe an hour to track down some memory corruption if you're particularly unlucky, in which case shuger's kind of got a point: who cares? Large network-exposed app? Individual memory corruption heisenbugs have taken me weeks to track down (and weeks before that for QA to create a reliable repro for) - a needle in a huge haystack. They often predate my employment - having lurked semi-silently for who knows how long causing who knows how many unreported crashes. When release dates slip because of bug backlogs filled with memory safety related crash bugs, when ~70% of many vendor CVE reports are down to memory safety issues [1][2][3], and when you personally have to deal with the fallout of all that: shuger's point completely and utterly evaporates. [1] https://msrc-blog.microsoft.com/2019/07/18/we-need-a-safer-systems-programming-language/ https://msrc-blog.microsoft.com/2019/07/18/we-need-a-safer-s... [2] https://www.chromium.org/Home/chromium-security/memory-safety/ https://www.chromium.org/Home/chromium-security/memory-safet... [3] https://langui.sh/2019/07/23/apple-memory-safety/ https://langui.sh/2019/07/23/apple-memory-safety/
- DrBazza 4y ago
  Just because Rust, in particular, compiles successfully, that's no guarantee that the code isn't complex and difficult to understand. I can write a complex badly designed app in any language. Similarly I can write simple well designed large applications in any language too.
  - Daishiman 4y ago
    This fallacy gets repeated over and over again and it doesn't make it any less false. Languages are tools and some tools are actually better-built than others. If we can claim that a language like Brainfuck makes writing clear code extremely difficult, and Python or Rust make writing clear code easier, we've already established that there's a spectrum for language in expressiveness and clarity. Eliminating entire classes of bugs makes for better understanding.
  - ChadNauseam 4y ago
    Yeah, but that doesn’t mean all languages are created equal in that department. Let’s take this pseudocode: int a = 0; if (findindex(mylist, myvalue, &a)) { // dostuff with a } Here, findindex returns false if it can’t find the value. The problem is that there’s nothing forcing you to use the if, you can just forget it and you’ll be left with incorrect code. In Rust, this type of error is impossible to make by accident, because the findindex function would return an Option, and you have to explicitly handle both cases (or explicitly say you don’t care about one of the cases). Things like that, along with the lifetime system, make it easier to write good code. It’s like saying that it’s possible to destroy your foot with a shotgun and with a pencil – it’s possible, but it’s a lot easier do to by accident with the shotgun.
    - deleted 4y ago
      [deleted]
lelanthran 4y ago
Agree; this is not an issue that slows down my development or bughunts. You can get a long way towards safety without learning Rust. It's those rare cases that will get you. It's a trade-off; take the time to learn the language and deliver later, or just use what you already have to deliver a product now.[1] [1] During a Rust discussion some years back, when I was at a different company, on a specialised and large-ish product written in C++03. I went through about 3 years of tickets (limited to only the bugs reported). No open ticket was older than a few weeks. Out of maybe 1000 bugs, only a single one was something Rust would have prevented. I would think that most mature products will have similar stats, so the trade-off is not as obvious as it looks to be on the surface. Deliverables matter.
- vgel 4y ago
  It probably depends on project type and how complex your ownership models are, but that doesn't really track with large projects having a majority of their CVEs be memory safety issues that are far less likely in Rust[1] (e.g., https://www.chromium.org/Home/chromium-security/memory-safety/ https://www.chromium.org/Home/chromium-security/memory-safet...) [1] I say far less likely because obviously it's possible with unsafe Rust, but I've never had one happen, seen one happen in real code, or been affected by part of a dependency tree having one.
  - lelanthran 4y ago
    I'm not saying that a large number of CVEs won't be prevented in Rust, I'm saying that so few bugs are CVEs that the trade-off is not always worth it. If you have 1000s of bug reports, of which 5 are CVEs, and then have 3 of those 5 be preventable, most dev teams are still going to consider the cost/benefit of going through the pain of developing a long-term product in Rust, or of switching to Rust altogether.
    - bschwindHN 4y ago
      > of which 5 are CVEs Those 5 are just the ones you know about...
bestouff 4y ago
Yeah no. In my experience, when several people commit to a common C/C++ codebase this kind of issue become really exhausting when it happens more than once, and the symptoms may be so subtle it's a bitch to debug. Rust lowers your mental load. You spend more time being creative and way less time debugging "obvious" (or not) mechanical problems (reference not-on-stack-anymore variables, use-after-free, concurrent write access and all kind of compiler undefined behavior). That why garbage collected languages are so successful (they let you concentrate on the business logic) and for the first time it's available in a system language. Everything that can be done by your computer should be done by your computer. You should leave your precious brain cells available for the important stuff.
- shuger 4y ago
  I work in probably what is considered one of the least "safe" languages: C++ The issues that Rust is supposed to help with are simply not what we spent time on. All the bugs reported are pretty much exclusively root caused to "business logic". From recent time I can recall only one that was a programming mistake and not architecture/business logic related. It was a missing break in a switch that already had some fallthroughs so it didn't look incorrect at a glance. I do understand what Rust is supposed to provide but in practice it's simply an extremely minor source of bugs.
  - ostenning 4y ago
    Can you elaborate on the proficiency of your dev team, is this with juniors etc? Is it a large team? And what is the complexity of the project? I think this is important information
    - shuger 4y ago
      GPU driver, most devs are senior. Hundreds of thousands of lines of code in the "slice" my team is interested in. Team for our component has on it's own has probably over 40 people. Driver should be even more prone to programming bugs because most of it is about manipulating data in raw "untyped" memory.
est31 4y ago
> You get corrupted data, set data breakpoint and in the provided example you will see it being modified by unrelated operations on stack. That's provided that you can even reproduce the issue well, especially in an instrumented build which might be way slower than the non instrumented one. Often you get bug reports like "crash after one hour of usage" where basically every feature of the app has been heavily used by multiple users. Rust applications might still crash but they crash safely, which means your error messages are more meaningful.
ekidd 4y ago
> All it had to say was that rust tells you when you keep reference to on stack variable after it goes out of scope. That's the root cause, but it's not the interesting bit. The code in the article comes from a production compiler. And normally, the AST (abstract syntax tree) is a single data structure output by the parser. Ownership is simple: the entire AST has the same lifetime, and it's managed by a caller. This should be easy, right? But it turned out that there was a piece of code that sometimes "synthesized" extra, temporary AST nodes. And these nodes had a shorter lifetime than the rest of the AST. These are vicious bugs. You have some long-standing convention about how things work, but one little piece of code makes an exception (often for excellent reasons). Then another module decides to make an aggressive optimization that relies on the original assumption. But that assumption is now true only 99% of the time. It's a communication failure, and it might take years to actually turn into a bug. And that bug may manifest as extremely rare memory corruption that shows up in automated crash reports. Running down this kind of phantom memory corruption is one of the most frustrating things I've ever done. It often involved spending weeks staring at minidumps, looking for interesting patterns in crashes. There's that horrible moment when you realize that 20% of your crashes occur within a thousand instructions after a particular font-rendering function reports an error, accidentally corrupting the exception-unwinding machinery. And sure, I get it. Maybe your team is simply good enough that nobody ever makes a mistake like this. But if so, they're exceptional. I've worked on amazing teams that still get bitten by subtle miscommunications and misunderstandings.
SloopJon 4y ago
> All it had to say was that rust tells you when you keep reference to on stack variable after it goes out of scope. The author isn't just telling you that Rust is awesome because it tells you something, he's acknowledging the frustration in learning how to listen to the compiler. It's kind of like Jerry Pournelle describing the ups and downs of USB by documenting an epic journey that all started with trying to scan some handwritten notes for his next novel using a Canon scanner he borrowed from Alex that he's just now getting around to reviewing, because the pins of the parallel port are too bent to use the old Epson. Okay, so maybe it was a little overcomplicated.
cbarrick 4y ago
Earlier this month we integrated a C++ library written by my team with a server written by another team. We saw the data corruption, and we knew it was a reference issue, but it took quite a bit of effort to track down. The cause was confusion around string_view and string&, with different behavior when you pass each to a new thread. Rust would have caught this much earlier and saved 3 days work.
- deleted 4y ago
  [deleted]
fleventynine 4y ago
Reviewing commits for a security-critical project written in C or C++ can be incredibly tedious. I've spent an entire day trying to validate that the assumptions made by a 10-line change are memory-safe in the context of the larger program. These reviews are incredibly mentally draining, and even when I'm done I'm not 100% sure that I didn't miss something and let a vulnerability into the codebase. Rust is a breath of fresh air in comparison. Worrying about memory safety isn't even a concern for the vast majority of commits that don't touch modules with unsafe code. All assumptions made about the lifetimes of references are made explicit in the code, and checked by the compiler. On rust projects I find I have much more mental energy to use against other aspects of the problem.
zh3 4y ago
As someone considering learning Rust, the article put me off fast with the long preamble to even explaining what the issue was. I really hope it's not that complicated, but even your rebuttal fills me with fear - "corrupted data [......] being modified by unrelated operations on stack". How would you explain that to put a C programmers mind at ease?
ArrayBoundCheck 4y ago
It's unfortunate how many upvotes this got from the title
grogers 4y ago
This case might be trivial to debug, but when you start adding concurrency a lot of that ease goes out the window. Right now, our tests are mildly flaky because of asan crashes from use after stack frame issues. Reading the code it should be joining all the coroutines on destruction, but yet the asan violations are happening. It really isn't that trivial to debug. Luckily in our case it doesn't affect production since it's only on shutdown (probably...?).

by shuger 4y ago

renox 4y ago

I agree with you on the first point, but I totally disagree with you on the second point: sure if you know how to reliably reproduce an issue it isn't complicated to debug, but this kind of issue can be difficult to reproduce, can create silent issues..

atoav 4y ago

It all depends on the complexity of your application. The type of guarantees Rust offers in my experience becomes exponentially more useful as your application grows in size.

lelanthran 4y ago
> It all depends on the complexity of your application. The type of guarantees Rust offers in my experience becomes exponentially more useful as your application grows in size. Sure, but the development pattern that get used for larger applications typically don't benefit from Rust's additional safety; for one you're going to be using a garbage collected language.
- MaulingMonkey 4y ago
  > for one you're going to be using a garbage collected language. I wish it were true, but promise you it is not. As a counterpoint, I point to most "AAA" gamedev and OS development. In gamedev it's even a bit flipped: The smaller indie gamedevs can pay the GC hit for Unity's C#, web JS, actionscript flash back in the day, etc. - not much working data, not much garbage. Larger scale titles start missing vsync and having horrible stuttering when GCs are thrown into the mix too brazenly - they're still used on smaller scales (embedding browser tech for UI, limited scope scripting, etc.) but they have a lot of native, non-GCed code.
MaulingMonkey 4y ago
This indeed. Small local console app running on trusted data? Maybe an hour to track down some memory corruption if you're particularly unlucky, in which case shuger's kind of got a point: who cares? Large network-exposed app? Individual memory corruption heisenbugs have taken me weeks to track down (and weeks before that for QA to create a reliable repro for) - a needle in a huge haystack. They often predate my employment - having lurked semi-silently for who knows how long causing who knows how many unreported crashes. When release dates slip because of bug backlogs filled with memory safety related crash bugs, when ~70% of many vendor CVE reports are down to memory safety issues [1][2][3], and when you personally have to deal with the fallout of all that: shuger's point completely and utterly evaporates. [1] https://msrc-blog.microsoft.com/2019/07/18/we-need-a-safer-systems-programming-language/ https://msrc-blog.microsoft.com/2019/07/18/we-need-a-safer-s... [2] https://www.chromium.org/Home/chromium-security/memory-safety/ https://www.chromium.org/Home/chromium-security/memory-safet... [3] https://langui.sh/2019/07/23/apple-memory-safety/ https://langui.sh/2019/07/23/apple-memory-safety/
DrBazza 4y ago
Just because Rust, in particular, compiles successfully, that's no guarantee that the code isn't complex and difficult to understand. I can write a complex badly designed app in any language. Similarly I can write simple well designed large applications in any language too.
- Daishiman 4y ago
  This fallacy gets repeated over and over again and it doesn't make it any less false. Languages are tools and some tools are actually better-built than others. If we can claim that a language like Brainfuck makes writing clear code extremely difficult, and Python or Rust make writing clear code easier, we've already established that there's a spectrum for language in expressiveness and clarity. Eliminating entire classes of bugs makes for better understanding.
- ChadNauseam 4y ago
  Yeah, but that doesn’t mean all languages are created equal in that department. Let’s take this pseudocode: int a = 0; if (findindex(mylist, myvalue, &a)) { // dostuff with a } Here, findindex returns false if it can’t find the value. The problem is that there’s nothing forcing you to use the if, you can just forget it and you’ll be left with incorrect code. In Rust, this type of error is impossible to make by accident, because the findindex function would return an Option, and you have to explicitly handle both cases (or explicitly say you don’t care about one of the cases). Things like that, along with the lifetime system, make it easier to write good code. It’s like saying that it’s possible to destroy your foot with a shotgun and with a pencil – it’s possible, but it’s a lot easier do to by accident with the shotgun.
  - deleted 4y ago
    [deleted]

lelanthran 4y ago

Agree; this is not an issue that slows down my development or bughunts. You can get a long way towards safety without learning Rust. It's those rare cases that will get you. It's a trade-off; take the time to learn the language and deliver later, or just use what you already have to deliver a product now.[1] [1] During a Rust discussion some years back, when I was at a different company, on a specialised and large-ish product written in C++03. I went through about 3 years of tickets (limited to only the bugs reported). No open ticket was older than a few weeks. Out of maybe 1000 bugs, only a single one was something Rust would have prevented. I would think that most mature products will have similar stats, so the trade-off is not as obvious as it looks to be on the surface. Deliverables matter.

vgel 4y ago
It probably depends on project type and how complex your ownership models are, but that doesn't really track with large projects having a majority of their CVEs be memory safety issues that are far less likely in Rust[1] (e.g., https://www.chromium.org/Home/chromium-security/memory-safety/ https://www.chromium.org/Home/chromium-security/memory-safet...) [1] I say far less likely because obviously it's possible with unsafe Rust, but I've never had one happen, seen one happen in real code, or been affected by part of a dependency tree having one.
- lelanthran 4y ago
  I'm not saying that a large number of CVEs won't be prevented in Rust, I'm saying that so few bugs are CVEs that the trade-off is not always worth it. If you have 1000s of bug reports, of which 5 are CVEs, and then have 3 of those 5 be preventable, most dev teams are still going to consider the cost/benefit of going through the pain of developing a long-term product in Rust, or of switching to Rust altogether.
  - bschwindHN 4y ago
    > of which 5 are CVEs Those 5 are just the ones you know about...

bestouff 4y ago

Yeah no. In my experience, when several people commit to a common C/C++ codebase this kind of issue become really exhausting when it happens more than once, and the symptoms may be so subtle it's a bitch to debug. Rust lowers your mental load. You spend more time being creative and way less time debugging "obvious" (or not) mechanical problems (reference not-on-stack-anymore variables, use-after-free, concurrent write access and all kind of compiler undefined behavior). That why garbage collected languages are so successful (they let you concentrate on the business logic) and for the first time it's available in a system language. Everything that can be done by your computer should be done by your computer. You should leave your precious brain cells available for the important stuff.

shuger 4y ago
I work in probably what is considered one of the least "safe" languages: C++ The issues that Rust is supposed to help with are simply not what we spent time on. All the bugs reported are pretty much exclusively root caused to "business logic". From recent time I can recall only one that was a programming mistake and not architecture/business logic related. It was a missing break in a switch that already had some fallthroughs so it didn't look incorrect at a glance. I do understand what Rust is supposed to provide but in practice it's simply an extremely minor source of bugs.
- ostenning 4y ago
  Can you elaborate on the proficiency of your dev team, is this with juniors etc? Is it a large team? And what is the complexity of the project? I think this is important information
  - shuger 4y ago
    GPU driver, most devs are senior. Hundreds of thousands of lines of code in the "slice" my team is interested in. Team for our component has on it's own has probably over 40 people. Driver should be even more prone to programming bugs because most of it is about manipulating data in raw "untyped" memory.

est31 4y ago

> You get corrupted data, set data breakpoint and in the provided example you will see it being modified by unrelated operations on stack. That's provided that you can even reproduce the issue well, especially in an instrumented build which might be way slower than the non instrumented one. Often you get bug reports like "crash after one hour of usage" where basically every feature of the app has been heavily used by multiple users. Rust applications might still crash but they crash safely, which means your error messages are more meaningful.

ekidd 4y ago

> All it had to say was that rust tells you when you keep reference to on stack variable after it goes out of scope. That's the root cause, but it's not the interesting bit. The code in the article comes from a production compiler. And normally, the AST (abstract syntax tree) is a single data structure output by the parser. Ownership is simple: the entire AST has the same lifetime, and it's managed by a caller. This should be easy, right? But it turned out that there was a piece of code that sometimes "synthesized" extra, temporary AST nodes. And these nodes had a shorter lifetime than the rest of the AST. These are vicious bugs. You have some long-standing convention about how things work, but one little piece of code makes an exception (often for excellent reasons). Then another module decides to make an aggressive optimization that relies on the original assumption. But that assumption is now true only 99% of the time. It's a communication failure, and it might take years to actually turn into a bug. And that bug may manifest as extremely rare memory corruption that shows up in automated crash reports. Running down this kind of phantom memory corruption is one of the most frustrating things I've ever done. It often involved spending weeks staring at minidumps, looking for interesting patterns in crashes. There's that horrible moment when you realize that 20% of your crashes occur within a thousand instructions after a particular font-rendering function reports an error, accidentally corrupting the exception-unwinding machinery. And sure, I get it. Maybe your team is simply good enough that nobody ever makes a mistake like this. But if so, they're exceptional. I've worked on amazing teams that still get bitten by subtle miscommunications and misunderstandings.

SloopJon 4y ago

> All it had to say was that rust tells you when you keep reference to on stack variable after it goes out of scope. The author isn't just telling you that Rust is awesome because it tells you something, he's acknowledging the frustration in learning how to listen to the compiler. It's kind of like Jerry Pournelle describing the ups and downs of USB by documenting an epic journey that all started with trying to scan some handwritten notes for his next novel using a Canon scanner he borrowed from Alex that he's just now getting around to reviewing, because the pins of the parallel port are too bent to use the old Epson. Okay, so maybe it was a little overcomplicated.

cbarrick 4y ago

Earlier this month we integrated a C++ library written by my team with a server written by another team. We saw the data corruption, and we knew it was a reference issue, but it took quite a bit of effort to track down. The cause was confusion around string_view and string&, with different behavior when you pass each to a new thread. Rust would have caught this much earlier and saved 3 days work.

deleted 4y ago
[deleted]

fleventynine 4y ago

Reviewing commits for a security-critical project written in C or C++ can be incredibly tedious. I've spent an entire day trying to validate that the assumptions made by a 10-line change are memory-safe in the context of the larger program. These reviews are incredibly mentally draining, and even when I'm done I'm not 100% sure that I didn't miss something and let a vulnerability into the codebase. Rust is a breath of fresh air in comparison. Worrying about memory safety isn't even a concern for the vast majority of commits that don't touch modules with unsafe code. All assumptions made about the lifetimes of references are made explicit in the code, and checked by the compiler. On rust projects I find I have much more mental energy to use against other aspects of the problem.

zh3 4y ago

As someone considering learning Rust, the article put me off fast with the long preamble to even explaining what the issue was. I really hope it's not that complicated, but even your rebuttal fills me with fear - "corrupted data [......] being modified by unrelated operations on stack". How would you explain that to put a C programmers mind at ease?

ArrayBoundCheck 4y ago

It's unfortunate how many upvotes this got from the title

grogers 4y ago

This case might be trivial to debug, but when you start adding concurrency a lot of that ease goes out the window. Right now, our tests are mildly flaky because of asan crashes from use after stack frame issues. Reading the code it should be joining all the coroutines on destruction, but yet the asan violations are happening. It really isn't that trivial to debug. Luckily in our case it doesn't affect production since it's only on shutdown (probably...?).