7 ms·

Rust isn't perfect. UB is a bug in rust, but it occasionally has bugs. Ideally you'd do rust and asan with good unit tests. And yes, if you only picked one, it

Rust isn't perfect. UB is a bug in rust, but it occasionally has bugs. Ideally you'd do rust and asan with good unit tests. And yes, if you only picked one, it should be rust, but don't just pick one. And just because you are using rust is no excuse to skip static analysis like prusti, coverage with gcov, llvm, or tarpaulin, and certainly not unit tests.

deleted 4y ago
[deleted]
tinglymintyfrsh 4y ago
There is unlikely ever to be a perfect, but a getting better by constraining behavior that breaks things. Throw all of the compile-time, profiling, checked builds, and binary-level tools at projects for defense-in-depth and checklists, no matter the platform or the application. Fuzzing, valgrind, gperftools, dtrace, {[amt],ub}san, etc. and formal methods like seL4 if you can afford the investment. :>
- galangalalgol 4y ago
  Does valgrind add anything to asan given the same unit tests? I haven't used in years because it is so slow.
  - saagarjha 4y ago
    Address sanitizer is not precise and can miss certain classes of bugs that Valgrind would catch.
    - tinglymintyfrsh 4y ago
      Yeap. Although Google killed off afl, exercise code with fuzzing too that unit and integration tests didn't catch.
lmm 4y ago
> And just because you are using rust is no excuse to skip static analysis like prusti, coverage with gcov, llvm, or tarpaulin, and certainly not unit tests. It absolutely is, and this kind of absolutism is what holds back the adoption of things like rust. In most real world software development, the target defect rate is not zero. The value proposition of something like rust for most businesses isn't that it lets you lower your defect rate; it's that it lets you maintain your existing (acceptable) defect rate at a much lower cost, by letting you drop your static analysis and most of your tests (reducing maintenance burdens) and still have a better bottom-line defect rate.
- consp 4y ago
  So your assumption is that most to all defects are language related and not programmer error? I will bet the farm your defect rate will remain the same without any actual validation of what people write. Plenty of errors due to language quirks but lots of times I see missing statements, fixed values which should be variable, input checking failures due to unknown input, incorrect assumptions and lots of non language gelated bugs which will not go away.
  - green_on_black 4y ago
    You're not accounting for the fact that Rust ships with a mandatory static analyzer called rustc. Btw Google recently gave a talk about switching from Java to Kotlin, which significantly reduced bugs, despite less mature tooling. So...
  - lmm 4y ago
    > So your assumption is that most to all defects are language related What does that even mean? > I will bet the farm your defect rate will remain the same without any actual validation of what people write. In what sense is a typechecker (or the rust borrow checker) not "actual validation", but a static analyser is? > Plenty of errors due to language quirks but lots of times I see missing statements, fixed values which should be variable, input checking failures due to unknown input, incorrect assumptions and lots of non language gelated bugs Most of those sound like type errors to me. Errors where the program isn't doing what the programmer thought it should can generally be avoided by using more precise types. (The more insidious case is where the program is doing exactly what the programmer thought it should, but they've misunderstood the specification or not thought through the implications - but no kind of testing can catch that kind of bug).

deleted 4y ago

[deleted]

tinglymintyfrsh 4y ago

There is unlikely ever to be a perfect, but a getting better by constraining behavior that breaks things. Throw all of the compile-time, profiling, checked builds, and binary-level tools at projects for defense-in-depth and checklists, no matter the platform or the application. Fuzzing, valgrind, gperftools, dtrace, {[amt],ub}san, etc. and formal methods like seL4 if you can afford the investment. :>

galangalalgol 4y ago
Does valgrind add anything to asan given the same unit tests? I haven't used in years because it is so slow.
- saagarjha 4y ago
  Address sanitizer is not precise and can miss certain classes of bugs that Valgrind would catch.
  - tinglymintyfrsh 4y ago
    Yeap. Although Google killed off afl, exercise code with fuzzing too that unit and integration tests didn't catch.

lmm 4y ago

> And just because you are using rust is no excuse to skip static analysis like prusti, coverage with gcov, llvm, or tarpaulin, and certainly not unit tests. It absolutely is, and this kind of absolutism is what holds back the adoption of things like rust. In most real world software development, the target defect rate is not zero. The value proposition of something like rust for most businesses isn't that it lets you lower your defect rate; it's that it lets you maintain your existing (acceptable) defect rate at a much lower cost, by letting you drop your static analysis and most of your tests (reducing maintenance burdens) and still have a better bottom-line defect rate.

consp 4y ago
So your assumption is that most to all defects are language related and not programmer error? I will bet the farm your defect rate will remain the same without any actual validation of what people write. Plenty of errors due to language quirks but lots of times I see missing statements, fixed values which should be variable, input checking failures due to unknown input, incorrect assumptions and lots of non language gelated bugs which will not go away.
- green_on_black 4y ago
  You're not accounting for the fact that Rust ships with a mandatory static analyzer called rustc. Btw Google recently gave a talk about switching from Java to Kotlin, which significantly reduced bugs, despite less mature tooling. So...
- lmm 4y ago
  > So your assumption is that most to all defects are language related What does that even mean? > I will bet the farm your defect rate will remain the same without any actual validation of what people write. In what sense is a typechecker (or the rust borrow checker) not "actual validation", but a static analyser is? > Plenty of errors due to language quirks but lots of times I see missing statements, fixed values which should be variable, input checking failures due to unknown input, incorrect assumptions and lots of non language gelated bugs Most of those sound like type errors to me. Errors where the program isn't doing what the programmer thought it should can generally be avoided by using more precise types. (The more insidious case is where the program is doing exactly what the programmer thought it should, but they've misunderstood the specification or not thought through the implications - but no kind of testing can catch that kind of bug).