9 ms·

I work in probably what is considered one of the least "safe" languages: C++ The issues that Rust is supposed to help with are simply not what we spent time on

by shuger 4y ago

I work in probably what is considered one of the least "safe" languages: C++ The issues that Rust is supposed to help with are simply not what we spent time on. All the bugs reported are pretty much exclusively root caused to "business logic". From recent time I can recall only one that was a programming mistake and not architecture/business logic related. It was a missing break in a switch that already had some fallthroughs so it didn't look incorrect at a glance. I do understand what Rust is supposed to provide but in practice it's simply an extremely minor source of bugs.

ostenning 4y ago
Can you elaborate on the proficiency of your dev team, is this with juniors etc? Is it a large team? And what is the complexity of the project? I think this is important information
- shuger 4y ago
  GPU driver, most devs are senior. Hundreds of thousands of lines of code in the "slice" my team is interested in. Team for our component has on it's own has probably over 40 people. Driver should be even more prone to programming bugs because most of it is about manipulating data in raw "untyped" memory.
  - the_mitsuhiko 4y ago
    GPU drivers are also some of the most buggiest stuff out there I used. When I worked in Games we routinely managed to make the GPU drivers crash which thankfully at the time was already no longer taking down your engine machine.
  - Daishiman 4y ago
    My experience with GPU drivers then is that your code is likely buggy AF and your team is in denial.
bestouff 4y ago
My manager told me exactly the same thing as you do; after I found a few UB in its code in the first 2 weeks he changed its stance.
- josefx 4y ago
  I had to deal with a senior dev. that gave me a talk about seniority after I ran valgrind over our software. Guy was so deep into the whole senior dev. power trip that he blamed third party libraries for his bugs, dev. tools for "incorrectly" identifying his bugs and wrote more bugs to work around his other bugs. Finding and fixing issues in C++ code can be easy with the available tools, getting people to use them on the other hand can can be like talking to a wall.
  - darthrupert 4y ago
    That sounds more like an expert beginner than senior.
    - im3w1l 4y ago
      Sounds like someone who senses his place in the hierarchy is threatened.
strictfp 4y ago
I've worked in a variety of languages, and returning to a c++ project recently I do see that we spend a lot more time thinking about how to write the code in a way that avoids problems. Meaning that there's a lot more architecturing required to reach a sane state. We have a sister product written in a dynamic language, and sometimes we have identical functionality. I've noticed that when a change is discussed, the c++ gang has architectured themselves into the current solution and therefore have a much harder time making changes. So for that reason I think it's easy to overlook these complexities when you're working in c++ alone; they feel natural and are just part of how you work. You forget that a lot of this architecturing just isn't necessary in a lot of other languages.
- jstimpfle 4y ago
  Maybe I'm wrong, but I believe that most of the architecting that you describe would be effectively what you do with regards to performance as well: Minimizing change of ownership, moving to a system with more static allocations with fewer "objects" that are linked into a variety of subsystems.
  - cyber_kinetist 4y ago
    …and when you do need a large number of dynamically allocced/deallocced objects, then using indices to arrays instead of pointers. Which kinda defeats the purpose of using the Rust borrow checker…
  - strictfp 4y ago
    Yes, that's true. In a sense, c++ requires good code structuring. That's also part of why I enjoy returning to c++, the people involved know how to structure code and create clean architecture. That said, sometimes c++ does get in the way. Creating trees or graphs can be cumbersome, and IMO it's very biased towards virtual methods to solve polymorphism. Extending lifetimes by pooling or similar is also quite common, and is in my eyes sometimes overdoing it. If you for instance use Rust, you can be a lot more confident that the compiler catches these issues, and be more conservative and efficient in the solution.
staticassertion 4y ago
Unsurprisingly the types of bugs reported are going to be around business logic errors and not obscure edge case that users won't run into naturally. The bugs are still there though.
- hgomersall 4y ago
  Right, and if you come across it you can't always reproduce it. "Oh it crashed, yeah it does that sometimes".
xyzzy123 4y ago
I think this mirrors my day to day experience with C++. On the other hand, fuzzing large c++ programs will routinely uncover memory safety issues in practically any large codebase that hasn't been absolutely beaten to death by fuzzers already. The issues are not usually so much "I returned this thing on the stack" they tend to be things like "this (very unexpected) sequence of api calls will result in a UAF in this deeply nested data structure over here on the heap".
lmm 4y ago
> The issues that Rust is supposed to help with are simply not what we spent time on. All the bugs reported are pretty much exclusively root caused to "business logic". Everyone claims this, probably because business logic bugs are more memorable. But I've never seen it match the real statistics. According to the best published data, null alone is something like 30-70% of bugs, you just don't remember them because they're uninteresting.
- Macha 4y ago
  It depends on the person's mental categorisation of bugs too. For example I could see someone classifying null bugs as business logic bugs because "the business logic didn't account for that information not always being there"
  - jhgb 4y ago
    It's a language bug, really. If you're expected to say what the individual things in your program are allowed to be (like you have to in C++, Java, C# etc.), and you say "foo is a BarBaz object", and the language and compiler allow you to set foo to something that isn't actually a BarBaz object and this is considered OK, then the language is botched.
- capitainenemo 4y ago
  I wonder if passing NaN is considered "business logic" Thinking of this interesting exchange long ago by an experienced c++ dev https://old.reddit.com/r/rust/comments/78bowa/hey_this_is_kyren_from_chucklefish_we_make_and/dot6uxk/ https://old.reddit.com/r/rust/comments/78bowa/hey_this_is_ky...
- akhmatova 4y ago
  There are and never will be (meaningful) statistics for the "N percent of bugs are caused by X" question. Every org's use cases are different, how do you get (let alone compare) data frm different orgs, who really counts their bugs anyway (and those that do at scale and in detail are probably doing suffering from some form of myopic management disorder or another), etc. All you can do is ask people their gut take based on their particular experience. For systems engineers, a lot of bugs are due to memory safety. For more consumer-oriented startups (or in most any bigcorp), yeah, it's "business logic" (or people's inability / unwillingness to communicate), etc. "We found that 70 percent of our bugs could have been prevented by moving to TypeScript", yeah sure.
mwcampbell 4y ago
Counterpoint: When I was on the Windows accessibility team at Microsoft, one of my most brilliant colleagues gave a presentation about what to look for when doing code reviews, and he emphasized three main categories: C++, COM (Microsoft's Component Object Model), and concurrency. Rust eliminates many of the issues in the first and third categories. And yes, we were writing modern C++ as much as we could in that legacy codebase; by 2019 we were using several C++17 features, as well as the latest C++ utility libraries for working with COM and WinRT. Given the state of the Rust windows crate, that team (which both I and that colleague left in late 2020) might even be able to use Rust in new code. I'm sure that would make him happy.
jackmott42 4y ago
I see many C++ programmers say stuff like this, and maybe it is the case that studios exist which can do C++ without mistakes related to memory management or sigils and so on, but I also observe that: 1. High quality teams like the Linux kernel team and PostgreSQL, do periodically have serious security bugs that are things Rust would have caught. 2. I see sometimes C++ instructors making the same claims you do and then spending half of a lesson tracking down a memory mistake, (Casey Muratori) or in the same month a tweet from a game engine developer saying they don't really see the value of garbage collection and then tweeting about how they spent 48 hours tracking down a memory mistake. (of course, for a game engine that is what you have to do sometimes!) There are however valid questions, like if Rust slows down your development say, 5%, would you get more net safety from spending 5% more time testing/fuzzing c++ code instead? etc.
- jesse__ 4y ago
  I'd venture that the development slowdown from using rust is closer to 50% than 5%. Even just compile times probably slow down that much, not to mention you have to write (sometimes) as much as 10x code to express yourself. The specific exercise I have in mind is a lockless thread queue. < 20 lines in C .. ~200 lines in Rust.
  - jackmott42 4y ago
    I'm sure there is a ton of variance. I have done some Rust projects where I run into absolutely no safety complaints from Rust because it just isn't the kind of code that does anything the borrow checker cares about. For those development time ends up typically faster than C/C++ due to various syntax and tooling niceties. Other projects will really get into domains where you have to work hard to satisfy the borrow checker and it can slow you down a lot. In a real application you won't be writing lockless thread queues for a big % of the time. But then for a real application the compile times will start to weigh on you more. (Though, C++ does not always compile fast either unless some care is taken to be sure it does)
    - ttfkam 4y ago
nemothekid 4y ago
>The issues that Rust is supposed to help with are simply not what we spent time on. There could be plenty of bugs in your codebase but just because you don't spend time on them doesn't mean they don't exist. Hundreds of millions of people used OpenSSL everyday for 2 years after the Heartbleed bug was introduced. It didn't cause any obviously broken code until someone exploited it to read credit card numbers off of a remote server.
- deleted 4y ago
  [deleted]
Agingcoder 4y ago
In practice for you. Where I am, things are very different! Buffer overflows and memory corruptions, threading issues, uninitialized variables, etc appear on a very regular basis, and end up being very difficult to debug, mostly because the moment where you corrupt memory and the moment it triggers a bug can be very far apart.

by shuger 4y ago

ostenning 4y ago

Can you elaborate on the proficiency of your dev team, is this with juniors etc? Is it a large team? And what is the complexity of the project? I think this is important information

shuger 4y ago
GPU driver, most devs are senior. Hundreds of thousands of lines of code in the "slice" my team is interested in. Team for our component has on it's own has probably over 40 people. Driver should be even more prone to programming bugs because most of it is about manipulating data in raw "untyped" memory.
- the_mitsuhiko 4y ago
  GPU drivers are also some of the most buggiest stuff out there I used. When I worked in Games we routinely managed to make the GPU drivers crash which thankfully at the time was already no longer taking down your engine machine.
- Daishiman 4y ago
  My experience with GPU drivers then is that your code is likely buggy AF and your team is in denial.

bestouff 4y ago

My manager told me exactly the same thing as you do; after I found a few UB in its code in the first 2 weeks he changed its stance.

josefx 4y ago
I had to deal with a senior dev. that gave me a talk about seniority after I ran valgrind over our software. Guy was so deep into the whole senior dev. power trip that he blamed third party libraries for his bugs, dev. tools for "incorrectly" identifying his bugs and wrote more bugs to work around his other bugs. Finding and fixing issues in C++ code can be easy with the available tools, getting people to use them on the other hand can can be like talking to a wall.
- darthrupert 4y ago
  That sounds more like an expert beginner than senior.
  - im3w1l 4y ago
    Sounds like someone who senses his place in the hierarchy is threatened.

strictfp 4y ago

I've worked in a variety of languages, and returning to a c++ project recently I do see that we spend a lot more time thinking about how to write the code in a way that avoids problems. Meaning that there's a lot more architecturing required to reach a sane state. We have a sister product written in a dynamic language, and sometimes we have identical functionality. I've noticed that when a change is discussed, the c++ gang has architectured themselves into the current solution and therefore have a much harder time making changes. So for that reason I think it's easy to overlook these complexities when you're working in c++ alone; they feel natural and are just part of how you work. You forget that a lot of this architecturing just isn't necessary in a lot of other languages.

jstimpfle 4y ago
Maybe I'm wrong, but I believe that most of the architecting that you describe would be effectively what you do with regards to performance as well: Minimizing change of ownership, moving to a system with more static allocations with fewer "objects" that are linked into a variety of subsystems.
- cyber_kinetist 4y ago
  …and when you do need a large number of dynamically allocced/deallocced objects, then using indices to arrays instead of pointers. Which kinda defeats the purpose of using the Rust borrow checker…
- strictfp 4y ago
  Yes, that's true. In a sense, c++ requires good code structuring. That's also part of why I enjoy returning to c++, the people involved know how to structure code and create clean architecture. That said, sometimes c++ does get in the way. Creating trees or graphs can be cumbersome, and IMO it's very biased towards virtual methods to solve polymorphism. Extending lifetimes by pooling or similar is also quite common, and is in my eyes sometimes overdoing it. If you for instance use Rust, you can be a lot more confident that the compiler catches these issues, and be more conservative and efficient in the solution.

staticassertion 4y ago

Unsurprisingly the types of bugs reported are going to be around business logic errors and not obscure edge case that users won't run into naturally. The bugs are still there though.

hgomersall 4y ago
Right, and if you come across it you can't always reproduce it. "Oh it crashed, yeah it does that sometimes".

xyzzy123 4y ago

I think this mirrors my day to day experience with C++. On the other hand, fuzzing large c++ programs will routinely uncover memory safety issues in practically any large codebase that hasn't been absolutely beaten to death by fuzzers already. The issues are not usually so much "I returned this thing on the stack" they tend to be things like "this (very unexpected) sequence of api calls will result in a UAF in this deeply nested data structure over here on the heap".

lmm 4y ago

> The issues that Rust is supposed to help with are simply not what we spent time on. All the bugs reported are pretty much exclusively root caused to "business logic". Everyone claims this, probably because business logic bugs are more memorable. But I've never seen it match the real statistics. According to the best published data, null alone is something like 30-70% of bugs, you just don't remember them because they're uninteresting.

Macha 4y ago
It depends on the person's mental categorisation of bugs too. For example I could see someone classifying null bugs as business logic bugs because "the business logic didn't account for that information not always being there"
- jhgb 4y ago
  It's a language bug, really. If you're expected to say what the individual things in your program are allowed to be (like you have to in C++, Java, C# etc.), and you say "foo is a BarBaz object", and the language and compiler allow you to set foo to something that isn't actually a BarBaz object and this is considered OK, then the language is botched.
capitainenemo 4y ago
I wonder if passing NaN is considered "business logic" Thinking of this interesting exchange long ago by an experienced c++ dev https://old.reddit.com/r/rust/comments/78bowa/hey_this_is_kyren_from_chucklefish_we_make_and/dot6uxk/ https://old.reddit.com/r/rust/comments/78bowa/hey_this_is_ky...
akhmatova 4y ago
There are and never will be (meaningful) statistics for the "N percent of bugs are caused by X" question. Every org's use cases are different, how do you get (let alone compare) data frm different orgs, who really counts their bugs anyway (and those that do at scale and in detail are probably doing suffering from some form of myopic management disorder or another), etc. All you can do is ask people their gut take based on their particular experience. For systems engineers, a lot of bugs are due to memory safety. For more consumer-oriented startups (or in most any bigcorp), yeah, it's "business logic" (or people's inability / unwillingness to communicate), etc. "We found that 70 percent of our bugs could have been prevented by moving to TypeScript", yeah sure.

mwcampbell 4y ago

Counterpoint: When I was on the Windows accessibility team at Microsoft, one of my most brilliant colleagues gave a presentation about what to look for when doing code reviews, and he emphasized three main categories: C++, COM (Microsoft's Component Object Model), and concurrency. Rust eliminates many of the issues in the first and third categories. And yes, we were writing modern C++ as much as we could in that legacy codebase; by 2019 we were using several C++17 features, as well as the latest C++ utility libraries for working with COM and WinRT. Given the state of the Rust windows crate, that team (which both I and that colleague left in late 2020) might even be able to use Rust in new code. I'm sure that would make him happy.

jackmott42 4y ago

I see many C++ programmers say stuff like this, and maybe it is the case that studios exist which can do C++ without mistakes related to memory management or sigils and so on, but I also observe that: 1. High quality teams like the Linux kernel team and PostgreSQL, do periodically have serious security bugs that are things Rust would have caught. 2. I see sometimes C++ instructors making the same claims you do and then spending half of a lesson tracking down a memory mistake, (Casey Muratori) or in the same month a tweet from a game engine developer saying they don't really see the value of garbage collection and then tweeting about how they spent 48 hours tracking down a memory mistake. (of course, for a game engine that is what you have to do sometimes!) There are however valid questions, like if Rust slows down your development say, 5%, would you get more net safety from spending 5% more time testing/fuzzing c++ code instead? etc.

jesse__ 4y ago
I'd venture that the development slowdown from using rust is closer to 50% than 5%. Even just compile times probably slow down that much, not to mention you have to write (sometimes) as much as 10x code to express yourself. The specific exercise I have in mind is a lockless thread queue. < 20 lines in C .. ~200 lines in Rust.
- jackmott42 4y ago
  I'm sure there is a ton of variance. I have done some Rust projects where I run into absolutely no safety complaints from Rust because it just isn't the kind of code that does anything the borrow checker cares about. For those development time ends up typically faster than C/C++ due to various syntax and tooling niceties. Other projects will really get into domains where you have to work hard to satisfy the borrow checker and it can slow you down a lot. In a real application you won't be writing lockless thread queues for a big % of the time. But then for a real application the compile times will start to weigh on you more. (Though, C++ does not always compile fast either unless some care is taken to be sure it does)
  - ttfkam 4y ago

nemothekid 4y ago

>The issues that Rust is supposed to help with are simply not what we spent time on. There could be plenty of bugs in your codebase but just because you don't spend time on them doesn't mean they don't exist. Hundreds of millions of people used OpenSSL everyday for 2 years after the Heartbleed bug was introduced. It didn't cause any obviously broken code until someone exploited it to read credit card numbers off of a remote server.

deleted 4y ago
[deleted]

Agingcoder 4y ago

In practice for you. Where I am, things are very different! Buffer overflows and memory corruptions, threading issues, uninitialized variables, etc appear on a very regular basis, and end up being very difficult to debug, mostly because the moment where you corrupt memory and the moment it triggers a bug can be very far apart.