10 ms·

Chromium bug bounty money tree browser

358 points by admp 3y ago

layer8 3y ago
It would be nice to also display the average reward per file on each node.
Sarkie 3y ago
This is a really cool view on where to aim efforts
wizzard0 3y ago
Impressive visualization! Also kind of keeps the vibe of the internal Google tooling, minimalistic and down-to-earth pragmatic
- londons_explore 3y ago
  Google tooling of 2010, before everything needed to load 10 megabytes of javascript and had loading spinners for everything.
  - rebane2001 3y ago
    I try to keep my projects simple, client-sided, and self-contained. The way this page is set up isn't perhaps the best practices for the modern web, but you can download just the html file and it'll probably still work in 30 years time.
  - rfoo 3y ago
    Hey, but at least we have rounded corners now, which is nowhere to be seen in your 2010s <table>-heavy borgmaster summary /s
    - diggan 3y ago
      From the submission CSS file: details { margin: 5px 5px 5px 12px; border: 1px solid #aaa; border-radius: 4px; padding: 4px; } Not even "designs from 2010" are safe from rounded corners. (the border-radius CSS property was first proposed in 2002 [!] as far as I can tell: https://www.w3.org/TR/2002/WD-css3-border-20021107/ https://www.w3.org/TR/2002/WD-css3-border-20021107/)
    - twodave 3y ago
      No, back then we would use images to make rounded corners. It was… as awful as it sounds…
      - ramses0 3y ago
        Triggered!
- deleted 3y ago
  [deleted]
londons_explore 3y ago
I feel like this is 1 step away from running "blame" and associating dollar values with the security bugs caused by each engineer...
- consp 3y ago
  Time to get the svn praise back and do it sarcasticly.
- rfoo 3y ago
  ... and I bet the top 1 engineer didn't actually add any bug to the code base, they just did a few repo-wide refactor.
  - deleted 3y ago
    [deleted]
  - arghwhat 3y ago
    "refactor, no functional changes" Narrator: There were functional changes
- deleted 3y ago
  [deleted]
cr3ative 3y ago
This is a very neat visualisation. Even if a little CPU intensive expanding areas! I'd hope the Chrome team has something similar internally, it looks really useful to understand the attack surface, as it were.
treffer 3y ago
Nice one, given that this probably went down to the diff level it would be interesting to see it weighted by LOC changed, e.g. 10 lines in file A and 1 line in file B would mean file A gets 1/11th of the money assigned because it was the majority of the bug.? Or perhaps a LOC changed / file LOC based distribution. That would be how buggy each file is, with a $$$ tag.
- initplus 3y ago
  This would show the effect you want. Code like tests are likely to be very verbose, while often the actual vulnerability will come down to a handful of characters.
  - treffer 3y ago
    Depends on which effect you are after. I was thinking of ROI for reading code. Regularly having a one line issue in a 100 LOC file is very different from a two line issue in a 10'000 LOC file. And yes, tests need to be excluded, of course. But looks like that's done?
perryizgr8 3y ago
Google has paid out nearly 10 million dollars in bounties just for Chromium? Just one reason nobody else is able to compete. The money supply is infinite.
- Sephr 3y ago
  It's open source. You're free to base your own browser on Chromium.
  - wslh 3y ago
    But if you fork it you cannot paid that amount to keep it bug "free".
    - quickthrower2 3y ago
      You can merge in the paid for fixes though. Depends how divergent you want to be.
  - BiteCode_dev 3y ago
    Then it's not much of a competition. It's just more chrome market share, with a different name.
    - Sephr 3y ago
      Open source is what you make of it. If you have no plans to change Chromium, then sure, "It's just more chrome market share". I would argue that you probably shouldn't be making a web browser unless you have plans to change the status quo for web browsers.
      - BiteCode_dev 3y ago
        That's the difference between theory and practice. In theory, sure. In practice, there are lany chrome forks and they all chrome with a few features on top. In theory you can beat usain bolt by just running faster than him. In practice nobody does.
captn3m0 3y ago
Such a cool idea, and great execution. Edit: Is the raw data somewhere? A sunburst or tree map would be worth trying out
keketi 3y ago
How about a version where the monetary amounts are normalized by the number of lines of code?
- Cthulhu_ 3y ago
  Why do you ask? Genuinely curious, because lines of code is a moot point in software and security.
  - matsemann 3y ago
    Because it's interesting. If "X" is 1000 lines of code and has cost $20k in bounties, and "Y" is 1 000 000 lines of code and has also cost $20k in bounties, it's interesting to see that feature X has relatively more high profile bugs when it probably does much less.
  - worldsayshi 3y ago
    I wouldn't say that it's a moot point in every context. The metric we'd get here would amount to "given that this developer made a loc change, what is the monetary risk involved". Developers that are high on this metric might want to allow down and think twice next time they commit. Or not. Metrics are likely not very useful in general.
  - dsabanin 3y ago
    It’s a valid measure of the amount of code. 10 bugs in 100k LOC speaks of a very different quality than 10 bugs in 1k LOC.
  - phyzome 3y ago
    It's the same as the idea of per-capita normalization.
- quickthrower2 3y ago
  Or normalize by number of word spilt on the bug (as a proxy for complexity)
- deleted 3y ago
  [deleted]
20after4 3y ago
This is actually really similar to something I've been wanting to build for a long time. In my case I've thought it would be useful to have a way to calculate the likelihood for a given change to break things based on the history of breaking changes in the same file or area of the file. Basically a riskiness score for each change. The risk score could be associated with each PR and would provide a signal for reviewers about which code should get a bit of extra attention as well as highlighting the risky changes when they are being deployed. The tricky part would be tracking the same part of the code as it moves up and down because of insertions/deletions above it which would cause problems for a naive algorithm based on line numbers. Just doing it at the file level, like this does, might be good enough to be useful though.
- withinboredom 3y ago
  Not just the code itself, but the author. I worked with a guy that wrote at least one bug every time he created a PR.
  - dspillett 3y ago
    > wrote at least one bug every time he created a PR. An economic hero. A man of the people. Creating job security for QA departments!
    - lpapez 3y ago
      A friend working in office of a big-tech company located in Denmark said "one bad engineer like me working in Copenhagen can put food on the table for 20 Bulgarians working in customer support". Since that day I always wanted to get into FAANG-type companies, writing buggy code is basically philantropy.
      - hypothesis 3y ago
        This thread made some people question their no-bugs-allowed ideal, which is apparently a misanthropy…
  - 3y ago
evmar 3y ago
It's interesting to browse the large collection under chrome/browser/ui and ponder over how many of them are use-after-free for data where the performance of manual memory management really doesn't matter. Like [1] which is around the lifecycle of a "choose a file" dialog. It feels like in the big picture sense it would be better to just always using some sort of smarter/slower pointers in this kind of code just for extra defense. I saw in [2] there is some sort of `raw_ptr<T>` type [3] that seems to intend to help, so maybe the crash in [2] was actually successfully defended against? It's too bad that there's not a good way to have a broader way to switch between dialects within a project where in one place it's "this portion of the code is perf-critical and carefully reviewed" and in another "this portion of the code is perf-oblivious and has lots of async state that's easy to get wrong". I've wondered if it's almost worth mixing two separate languages (like a GCed one for the latter) just to make the distinction clear. [Disclaimer: worked on this code many years ago, wouldn't be surprised if I caused >0 of these bugs...] [1] https://bugs.chromium.org/p/chromium/issues/detail?id=1201032 https://bugs.chromium.org/p/chromium/issues/detail?id=120103... [2] https://bugs.chromium.org/p/chromium/issues/detail?id=1323239 https://bugs.chromium.org/p/chromium/issues/detail?id=132323... [3] https://source.chromium.org/chromium/chromium/src/+/main:base/memory/raw_ptr.md https://source.chromium.org/chromium/chromium/src/+/main:bas...
- wongarsu 3y ago
  > I've wondered if it's almost worth mixing two separate languages (like a GCed one for the latter) just to make the distinction clear. Python, with perf critical sections written in C or Rust is pretty much that. From what I hear the Rust-Python bindings are especially good, and make the correctness part easier even in the performance critical parts. Or you can go the opposite way and call out to a scripting language from your fast language. Today everyone is hyped about wasm for that, but we also have about two decades of computer games using lua for that (and games are probably the single biggest category of performance sensitive software)
- Tuna-Fish 3y ago
  > It's too bad that there's not a good way to have a broader way to switch between dialects within a project where in one place it's "this portion of the code is perf-critical and carefully reviewed" and in another "this portion of the code is perf-oblivious and has lots of async state that's easy to get wrong". I've wondered if it's almost worth mixing two separate languages (like a GCed one for the latter) just to make the distinction clear. You are describing Rust's "unsafe" keyword. And this kind of code is very literally the original impetus of it. The language was sort of originally designed to implement a browser, after all.
DaleCurtis 3y ago
Very cool! I think it's missing some entries though. I'm pretty sure we've had at least one in third_party/ffmpeg. Those fixes often land upstream first which might make tracking difficult.
- rebane2001 3y ago
  It's using whatever Git Watcher comments on the monorail bugs.
  - DaleCurtis 3y ago
    IIRC, that bot used to be called bugdroid. I forget when it switched over, probably somewhere in 2020.
rei5 3y ago
Nitpick: don't include DEPS, AUTHORS, or BUILD.gn files.
paulirish 3y ago
I ported this to a treemap visualization[1]: https://vrp-treemap.surge.sh/ https://vrp-treemap.surge.sh/ The treemap library was authored by evmar, chrome OG who's also in this thread.

358 points by admp 3y ago

layer8 3y ago

It would be nice to also display the average reward per file on each node.

Sarkie 3y ago

This is a really cool view on where to aim efforts

wizzard0 3y ago

Impressive visualization! Also kind of keeps the vibe of the internal Google tooling, minimalistic and down-to-earth pragmatic

londons_explore 3y ago
Google tooling of 2010, before everything needed to load 10 megabytes of javascript and had loading spinners for everything.
- rebane2001 3y ago
  I try to keep my projects simple, client-sided, and self-contained. The way this page is set up isn't perhaps the best practices for the modern web, but you can download just the html file and it'll probably still work in 30 years time.
- rfoo 3y ago
  Hey, but at least we have rounded corners now, which is nowhere to be seen in your 2010s <table>-heavy borgmaster summary /s
  - diggan 3y ago
    From the submission CSS file: details { margin: 5px 5px 5px 12px; border: 1px solid #aaa; border-radius: 4px; padding: 4px; } Not even "designs from 2010" are safe from rounded corners. (the border-radius CSS property was first proposed in 2002 [!] as far as I can tell: https://www.w3.org/TR/2002/WD-css3-border-20021107/ https://www.w3.org/TR/2002/WD-css3-border-20021107/)
  - twodave 3y ago
    No, back then we would use images to make rounded corners. It was… as awful as it sounds…
    - ramses0 3y ago
      Triggered!
deleted 3y ago
[deleted]

londons_explore 3y ago

I feel like this is 1 step away from running "blame" and associating dollar values with the security bugs caused by each engineer...

consp 3y ago
Time to get the svn praise back and do it sarcasticly.
rfoo 3y ago
... and I bet the top 1 engineer didn't actually add any bug to the code base, they just did a few repo-wide refactor.
- deleted 3y ago
  [deleted]
- arghwhat 3y ago
  "refactor, no functional changes" Narrator: There were functional changes
deleted 3y ago
[deleted]

cr3ative 3y ago

This is a very neat visualisation. Even if a little CPU intensive expanding areas! I'd hope the Chrome team has something similar internally, it looks really useful to understand the attack surface, as it were.

treffer 3y ago

Nice one, given that this probably went down to the diff level it would be interesting to see it weighted by LOC changed, e.g. 10 lines in file A and 1 line in file B would mean file A gets 1/11th of the money assigned because it was the majority of the bug.? Or perhaps a LOC changed / file LOC based distribution. That would be how buggy each file is, with a $$$ tag.

initplus 3y ago
This would show the effect you want. Code like tests are likely to be very verbose, while often the actual vulnerability will come down to a handful of characters.
- treffer 3y ago
  Depends on which effect you are after. I was thinking of ROI for reading code. Regularly having a one line issue in a 100 LOC file is very different from a two line issue in a 10'000 LOC file. And yes, tests need to be excluded, of course. But looks like that's done?

perryizgr8 3y ago

Google has paid out nearly 10 million dollars in bounties just for Chromium? Just one reason nobody else is able to compete. The money supply is infinite.

Sephr 3y ago
It's open source. You're free to base your own browser on Chromium.
- wslh 3y ago
  But if you fork it you cannot paid that amount to keep it bug "free".
  - quickthrower2 3y ago
    You can merge in the paid for fixes though. Depends how divergent you want to be.
- BiteCode_dev 3y ago
  Then it's not much of a competition. It's just more chrome market share, with a different name.
  - Sephr 3y ago
    Open source is what you make of it. If you have no plans to change Chromium, then sure, "It's just more chrome market share". I would argue that you probably shouldn't be making a web browser unless you have plans to change the status quo for web browsers.
    - BiteCode_dev 3y ago
      That's the difference between theory and practice. In theory, sure. In practice, there are lany chrome forks and they all chrome with a few features on top. In theory you can beat usain bolt by just running faster than him. In practice nobody does.

captn3m0 3y ago

Such a cool idea, and great execution. Edit: Is the raw data somewhere? A sunburst or tree map would be worth trying out

keketi 3y ago

How about a version where the monetary amounts are normalized by the number of lines of code?

Cthulhu_ 3y ago
Why do you ask? Genuinely curious, because lines of code is a moot point in software and security.
- matsemann 3y ago
  Because it's interesting. If "X" is 1000 lines of code and has cost $20k in bounties, and "Y" is 1 000 000 lines of code and has also cost $20k in bounties, it's interesting to see that feature X has relatively more high profile bugs when it probably does much less.
- worldsayshi 3y ago
  I wouldn't say that it's a moot point in every context. The metric we'd get here would amount to "given that this developer made a loc change, what is the monetary risk involved". Developers that are high on this metric might want to allow down and think twice next time they commit. Or not. Metrics are likely not very useful in general.
- dsabanin 3y ago
  It’s a valid measure of the amount of code. 10 bugs in 100k LOC speaks of a very different quality than 10 bugs in 1k LOC.
- phyzome 3y ago
  It's the same as the idea of per-capita normalization.
quickthrower2 3y ago
Or normalize by number of word spilt on the bug (as a proxy for complexity)
deleted 3y ago
[deleted]

20after4 3y ago

This is actually really similar to something I've been wanting to build for a long time. In my case I've thought it would be useful to have a way to calculate the likelihood for a given change to break things based on the history of breaking changes in the same file or area of the file. Basically a riskiness score for each change. The risk score could be associated with each PR and would provide a signal for reviewers about which code should get a bit of extra attention as well as highlighting the risky changes when they are being deployed. The tricky part would be tracking the same part of the code as it moves up and down because of insertions/deletions above it which would cause problems for a naive algorithm based on line numbers. Just doing it at the file level, like this does, might be good enough to be useful though.

withinboredom 3y ago
Not just the code itself, but the author. I worked with a guy that wrote at least one bug every time he created a PR.
- dspillett 3y ago
  > wrote at least one bug every time he created a PR. An economic hero. A man of the people. Creating job security for QA departments!
  - lpapez 3y ago
    A friend working in office of a big-tech company located in Denmark said "one bad engineer like me working in Copenhagen can put food on the table for 20 Bulgarians working in customer support". Since that day I always wanted to get into FAANG-type companies, writing buggy code is basically philantropy.
    - hypothesis 3y ago
      This thread made some people question their no-bugs-allowed ideal, which is apparently a misanthropy…
- 3y ago

evmar 3y ago

It's interesting to browse the large collection under chrome/browser/ui and ponder over how many of them are use-after-free for data where the performance of manual memory management really doesn't matter. Like [1] which is around the lifecycle of a "choose a file" dialog. It feels like in the big picture sense it would be better to just always using some sort of smarter/slower pointers in this kind of code just for extra defense. I saw in [2] there is some sort of `raw_ptr<T>` type [3] that seems to intend to help, so maybe the crash in [2] was actually successfully defended against? It's too bad that there's not a good way to have a broader way to switch between dialects within a project where in one place it's "this portion of the code is perf-critical and carefully reviewed" and in another "this portion of the code is perf-oblivious and has lots of async state that's easy to get wrong". I've wondered if it's almost worth mixing two separate languages (like a GCed one for the latter) just to make the distinction clear. [Disclaimer: worked on this code many years ago, wouldn't be surprised if I caused >0 of these bugs...] [1] https://bugs.chromium.org/p/chromium/issues/detail?id=1201032 https://bugs.chromium.org/p/chromium/issues/detail?id=120103... [2] https://bugs.chromium.org/p/chromium/issues/detail?id=1323239 https://bugs.chromium.org/p/chromium/issues/detail?id=132323... [3] https://source.chromium.org/chromium/chromium/src/+/main:base/memory/raw_ptr.md https://source.chromium.org/chromium/chromium/src/+/main:bas...

wongarsu 3y ago
> I've wondered if it's almost worth mixing two separate languages (like a GCed one for the latter) just to make the distinction clear. Python, with perf critical sections written in C or Rust is pretty much that. From what I hear the Rust-Python bindings are especially good, and make the correctness part easier even in the performance critical parts. Or you can go the opposite way and call out to a scripting language from your fast language. Today everyone is hyped about wasm for that, but we also have about two decades of computer games using lua for that (and games are probably the single biggest category of performance sensitive software)
Tuna-Fish 3y ago
> It's too bad that there's not a good way to have a broader way to switch between dialects within a project where in one place it's "this portion of the code is perf-critical and carefully reviewed" and in another "this portion of the code is perf-oblivious and has lots of async state that's easy to get wrong". I've wondered if it's almost worth mixing two separate languages (like a GCed one for the latter) just to make the distinction clear. You are describing Rust's "unsafe" keyword. And this kind of code is very literally the original impetus of it. The language was sort of originally designed to implement a browser, after all.

DaleCurtis 3y ago

Very cool! I think it's missing some entries though. I'm pretty sure we've had at least one in third_party/ffmpeg. Those fixes often land upstream first which might make tracking difficult.

rebane2001 3y ago
It's using whatever Git Watcher comments on the monorail bugs.
- DaleCurtis 3y ago
  IIRC, that bot used to be called bugdroid. I forget when it switched over, probably somewhere in 2020.

rei5 3y ago

Nitpick: don't include DEPS, AUTHORS, or BUILD.gn files.

paulirish 3y ago

I ported this to a treemap visualization[1]: https://vrp-treemap.surge.sh/ https://vrp-treemap.surge.sh/ The treemap library was authored by evmar, chrome OG who's also in this thread.