4 ms·
This isn't control for confounding factors. For example: folks are more likely to rewrite stuff that is well-understood, and stuff that is well-understood is g
by pizlonator 1y ago
This isn't control for confounding factors.
For example: folks are more likely to rewrite stuff that is well-understood, and stuff that is well-understood is going to have shorter review times and lower rollback rate.
That gnarly horrid mess that only a few greybeards grok and has massive test coverage, a long tail of requirements enforced by tests and experience, and a culture of extreme rigor? Longer reviews, more rollbacks, and less likely to be rewritten.
- littlestymaar 1y ago> That gnarly horrid mess that only a few greybeards grok and has massive test coverage, a long tail of requirements enforced by tests and experience, and a culture of extreme rigor? Longer reviews, more rollbacks, and less likely to be rewritten. I'd say that this is likely the most likely to be rewritten actually, because high test coverage is a massive enabler in such a rewrite, and because having a project that “only a few greybeards grok” sounds like a big organizational liability. That being said, and while I'm pretty convinced that Rust bring massive benefits, I agree with you that these measurements shouldn't be taken as if it was a rigorous scientific proof. It's more of one additional anecdotal evidence that Rust is good.
- pizlonator 1y ago> It's more of one additional anecdotal evidence that Rust is good. But that means it's likely to be the worst kind of science: - Group of people agree that Rust is good. This is a belief they hold. - Same group of people feel the need to search for argument that their belief is good. - The group does "science" like this. And then the rest of us have a data point that we think we can trust, when in reality, it's just cherry picked data being used to convey an opinion.
- dwattttt 1y ago> And then the rest of us have a data point that we think we can trust, when in reality, it's just cherry picked data being used to convey an opinion. Calling what Google did here "science" and cherry picked is quite a disservice. It's observational data, but do you have any objection to the methodology they used? Or just (assumed?) bad vibes?
- pizlonator 1y agoIn science, you go out of your way to control for confounding factors. This isn't that.
- gpm 1y ago> In science, you go out of your way to control for confounding factors. There's tons of observational science done in a very similar fashion to the article where there is simply no way to control for confounding factors for the same reason that there is simply no way to properly control for it in the data available.
- dwattttt 1y agoHaving been close to someone who went through the PhD process to a career in research, this is a sadly common but romantic and incorrect view of science as practiced in the world today.
- pizlonator 1y agoA lot of what folks call science isn't science. So, I'm not being romantic. I'm being realistic. And I'm happy to call B.S. on a lot of published research, because doing so gives me more predictive power than that research
- gf000 1y agoGoing out of your way would involve committing unethical experiments, which is absolutely frowned upon by scientists. And many experiments are simply impossible to do in a manner that completely removed every outside factors. But that doesn't mean that an experiment's results are immediately bad.
- baq 1y agoIt’s a good start and even if error bars are wide enough to land a 747 in them their numbers show orders of magnitude scale differences. This should raise eyebrows at the very least in the biggest skeptics.
- 1y ago
- gpm 1y agoThe first chart does in fact a compelling reason to believe the effect is not that at all. If the "easy" code was predominantly being rewritten you would expect to % new memory unsafe code and % memory safety vulnerabilities to scale at different rates as the difficult to work on areas remained in C and kept causing new memory vulnerabilities. Personal experience also provides a compelling reason, my experience is absolutely that people are more inclined to rewrite the things that are causing trouble in a new language. It's not a blinded randomly controlled trial of course, it's observational data. You can't be completely sure there isn't a confounding factor that explains the data, but it seems far more likely than not that it is a real effect.
- pizlonator 1y agoI would expect memory safety vulns to be dropping in most C/C++ projects due to better practices
- selfmodruntime 1y agoYou would be severely disappointed.
- Ar-Curunir 1y agoThis contradicts what Google has reported about their own code, which is that most vulnerabilities are in new code
- pizlonator 1y agoI don't see a contradiction between the statement that vulns in C++ code are generally on a downward trajectory, and the statement that most vulns are in new code.
- josefx 1y agoI haven't looked at Googles style for c++ in a long time, but from what I remember it was actively hostile to best practices. Actively hostile to any improvement introduced by the language and actively hostile to the wider ecosystem of c++. Also Rob Pike was involved somehow at the time and that guy went around claiming that his C inspired GCed language would be the perfect replacement for C++ everywhere after the c++ standard simplified some common patterns for library authors because he was actively hostile towards reusable code. I am not sure I even want to know what the average Google C++ codebase looks like.
- deleted 1y ago[deleted]
- selfmodruntime 1y agoThis article is mostly about new code.
- super_flanker 1y agoThis, from my limited experience working on AOSP at Google, only the newer code were being written in Rust, not the existing code. But that was 2 years ago.
- schuyler2d 1y agoThat is somewhat mitigated by grouping comparisons of S/M/L change sizes. It would be interesting to group changes by line-deletions, say to indicate rewrites (and size of them)