4 ms·

But there is AddressSanitizer for Rust and for C too right? As I understand it AddressSanitizer consumes LLVM IR, so from its point of view some C, C++ or Rust

But there is AddressSanitizer for Rust and for C too right? As I understand it AddressSanitizer consumes LLVM IR, so from its point of view some C, C++ or Rust is all the same, and presumably also if you are a famous Russian streamer and you hand wrote LLVM IR instead of using a real programming language that too?

IshKebab 6mo ago
Yes I was including C in "C++". I dunno how much C Firefox uses. And I presume you can run AddressSanitizer with Rust but given Rust is memory safe by default, it's only going to find issues in `unsafe` code which is a tiny tiny fraction of most code. Google had a blog post a few months ago where they managed to put some actual numbers on this, because they almost shipped one Rust memory safety bug.
- tialaramex 6mo ago
  The lesson for other projects is very different if the reason these are all C++ bugs is just "We didn't ask Mythos for the bugs in Rust" versus if the difference is that asking Mythos for similar bugs in the Rust is futile because it won't find any. Some of this is tempered if the pattern is that Mythos finds bugs mostly in dusty old C++ but the rates are much, much lower in newer C++, the reverse of Google's earlier finding for human researchers.
  - IshKebab 6mo ago
    > The lesson for other projects is very different if the reason these are all C++ bugs is just "We didn't ask Mythos for the bugs in Rust" versus if the difference is that asking Mythos for similar bugs in the Rust is futile because it won't find any. The answer is both of those. They didn't ask for bugs in the Rust code because it wouldn't have found any. They've explicitly set it up to only look for memory safety bugs. It's not going to find any in a memory safe language.
    - uecker 6mo ago
      As long as the memory-safe subset of Rust is used exclusively.
      - IshKebab 6mo ago
        Not exclusively, just the vast majority of the time. Which it is. Read this: https://blog.google/security/rust-in-android-move-fast-fix-things/ https://blog.google/security/rust-in-android-move-fast-fix-t... Exactly the same as using the memory-safe subset of Python or Java.
        uecker 6mo ago
        There exist memory safety bugs in Rust projects, so you will find them. Or maybe not with AI, as there is not enough training data? The 70% number google claims is either BS or google-specific as other projects reported far lower numbers.
        IshKebab 6mo ago
        > There exist memory safety bugs in Rust projects, so you will find them. Or maybe not with AI, as there is not enough training data? No, there are simply too few memory safety bugs in Rust projects for AI to find any. It found 271 bugs in Firefox so you're talking around 0.3 bugs found in the same amount of Rust. > The 70% number google claims is either BS or google-specific as other projects reported far lower numbers. The post I linked didn't mention 70% so I guess you didn't read it. And if you're talking about the "70% of C/C++ security bugs are due to memory safety" stat, then no it isn't bullshit. The same (or very similar) number has been found by numerous companies and projects. Not that that stat is relevant here.
        uecker 6mo ago
        It is impossible to interpret this number (271) without looking into details. People certainly found plenty of memory safety and others bugs in Rust projects in the past, so I do not understand you claim that there too few to find any. Curl reported 40% and more recently it dropped to about 20% of issues caused by their use of C. And this even with the requirement to stick to old C89. OpenBSD reported 30%. I assume the 70% either have to do with C++ or - more likely - there is a huge selection bias.

IshKebab 6mo ago

Yes I was including C in "C++". I dunno how much C Firefox uses. And I presume you can run AddressSanitizer with Rust but given Rust is memory safe by default, it's only going to find issues in `unsafe` code which is a tiny tiny fraction of most code. Google had a blog post a few months ago where they managed to put some actual numbers on this, because they almost shipped one Rust memory safety bug.

tialaramex 6mo ago
The lesson for other projects is very different if the reason these are all C++ bugs is just "We didn't ask Mythos for the bugs in Rust" versus if the difference is that asking Mythos for similar bugs in the Rust is futile because it won't find any. Some of this is tempered if the pattern is that Mythos finds bugs mostly in dusty old C++ but the rates are much, much lower in newer C++, the reverse of Google's earlier finding for human researchers.
- IshKebab 6mo ago
  > The lesson for other projects is very different if the reason these are all C++ bugs is just "We didn't ask Mythos for the bugs in Rust" versus if the difference is that asking Mythos for similar bugs in the Rust is futile because it won't find any. The answer is both of those. They didn't ask for bugs in the Rust code because it wouldn't have found any. They've explicitly set it up to only look for memory safety bugs. It's not going to find any in a memory safe language.
  - uecker 6mo ago
    As long as the memory-safe subset of Rust is used exclusively.
    - IshKebab 6mo ago
      Not exclusively, just the vast majority of the time. Which it is. Read this: https://blog.google/security/rust-in-android-move-fast-fix-things/ https://blog.google/security/rust-in-android-move-fast-fix-t... Exactly the same as using the memory-safe subset of Python or Java.
      - uecker 6mo ago
        There exist memory safety bugs in Rust projects, so you will find them. Or maybe not with AI, as there is not enough training data? The 70% number google claims is either BS or google-specific as other projects reported far lower numbers.
        IshKebab 6mo ago
        > There exist memory safety bugs in Rust projects, so you will find them. Or maybe not with AI, as there is not enough training data? No, there are simply too few memory safety bugs in Rust projects for AI to find any. It found 271 bugs in Firefox so you're talking around 0.3 bugs found in the same amount of Rust. > The 70% number google claims is either BS or google-specific as other projects reported far lower numbers. The post I linked didn't mention 70% so I guess you didn't read it. And if you're talking about the "70% of C/C++ security bugs are due to memory safety" stat, then no it isn't bullshit. The same (or very similar) number has been found by numerous companies and projects. Not that that stat is relevant here.
        uecker 6mo ago
        It is impossible to interpret this number (271) without looking into details. People certainly found plenty of memory safety and others bugs in Rust projects in the past, so I do not understand you claim that there too few to find any. Curl reported 40% and more recently it dropped to about 20% of issues caused by their use of C. And this even with the requirement to stick to old C89. OpenBSD reported 30%. I assume the 70% either have to do with C++ or - more likely - there is a huge selection bias.