9 ms·

A walk through Project Zero metrics

130 points by arkadiyt 5y ago

olliej 5y ago
I dislike many aspects of google, but project zero is not one of them and has greatly improve the overall security of the industry. Also their blogs describing how security exploits work are always super interesting
- burrows 5y ago
  > [P0] has greatly improve the overall security of the industry. What is the argument for this?
  - elisbce 5y ago
    Like the number of critical bugs they found and pushed the vendors to fix? Like finding bugs like Spectre that forever changed the chip industry's understanding of side channel attacks and subsequent designs?
  - olliej 5y ago
    They've been very effective at getting many large companies to actually fix security bugs, and release updates with those fixes in an actually reasonable time frame. They've been very strict about their "we will publish the details of this security bug in 90 days unless you can provide a very good reason not to".
  - tialaramex 5y ago
    Project Zero's argument is: > For nearly ten years, Google’s Project Zero has been working to make it more difficult for bad actors to find and exploit security vulnerabilities, significantly improving the security of the Internet for everyone. In that time, we have partnered with folks across industry to transform the way organizations prioritize and approach fixing security vulnerabilities and updating people’s software. This feels pretty reasonable, we don't have a spare universe to act as our control, but their intervention does seem likely to have made us more secure overall than otherwise.
    - burrows 5y ago
      Is “the security of the Internet” just a proxy for, “the number of vulns in the wild”?
      - tourist2d 5y ago
        Sure, you could also phrase that sentence a hundred other ways.
r00fus 5y ago
Is it meaningful to include "Linux" as a discrete vendor? How would you compare an OSS project to a company like Microsoft or Google?
- that_guy_iain 5y ago
  The Linux Kernel is a product and the Linux Foundation is it‘s vendor. I would assume they mean Them. Especially when they had Red Hat and Cannonical in „other“
  - marcan_42 5y ago
    The Linux Foundation is not the vendor for Linux. "Vendor" here refers to the Linux kernel community and how it acts when security issues are discovered. That is, if you want to be pedantic, it's Linus Torvalds and everyone under him. It doesn't really matter that it's not a legal entity, just "whoever is responsible for fixing the bug and releasing the fix officially". In security we call open source projects vendors because they act as such just like Apple would.
    - that_guy_iain 5y ago
      Linus Trovalds works for the Linux Foundation? So if we're being pedantic, which I think you are, then If it's Linux Torvalds in his professional capcitiy then it is for Linux Foundation.
      - marcan_42 5y ago
        Linus Torvalds does not answer to the Linux Foundation, and the Linux Foundation has little to do with how the Linux kernel handles security reports. It doesn't matter that they sponsor him; they aren't the "vendor" for Linux in any meaningful sense. They are just a nonprofit entity established to support Linux development in various ways.
        that_guy_iain 5y ago
deleted 5y ago
[deleted]
tester756 5y ago
that iOS vs Android table kinda makes no sense as they said iOS 76, Android Samsung 10, Android Pixel 6 >The first thing to note is that it appears that iOS received remarkably more bug reports from Project Zero than any flavor of Android did during this time period, but rather than an imbalance in research target selection, this is more a reflection of how Apple ships software. Security updates for "apps" such as iMessage, Facetime, and Safari/WebKit are all shipped as part of the OS updates, so we include those in the analysis of the operating system. On the other hand, security updates for standalone apps on Android happen through the Google Play Store, so they are not included here in this analysis. so kinda what's the point of putting that column there? people will use it as an argument that Android is safer :P
- amscanne 5y ago
  There are advantages and disadvantages to bundling applications with the core OS; having these security bugs become part of the OS release vehicle (along with the heavyweight process that implies) seems like a disadvantage. With respect to the table, I think there’s a decent argument either way.
- concinds 5y ago
  I would argue that Android is safer, specifically from 0days. Anyone with Android 6.0 or later (a 2015 OS) gets Chrome updates. Those updates are fast, automatic, and transparently happen in the background, with no user input or downtime (e.g. need to restart). They're also not arbitrarily delayed to fit into with some rigid OS update release schedule, like iOS or macOS. That's especially important in this era where people are skeptical of updates, or just too busy, and keep delaying and skipping updates. Every time I use a family member's device, and check the version, I see they haven't updated in months. Many aren't even aware there are updates available. I told the story before, but my grandma only has 4G, no Wi-Fi, and only has an iPhone, no laptop to download IPSWs from (and obviously average users can't do that); therefore she only gets iOS updates when I visit once or twice a year. Is Apple happy with that? Provocatively: is that the best they can do? Google proves it's not. Sure, the Android OS update situation isn't great, but your biggest 0day exposures will be apps exposed to the internet (Chrome, chat apps) and those all get regular, transparent background updates with no user input; and Android users are much safer for it, than they'd be if Google imitated Apple's OS update strategy. This is an Apple-wide problem, not just iOS. Anyone with OS X El Capitan 10.11 or later (again, a 2015 OS) is running the latest Google Chrome. Yet their Safari version is riddled with catastrophic 0day bugs (which can now be called 1,314-day bugs, since there have been 1,314 days since the last El Capitan security update). Add to that the fact that Apple doesn't really support "n-2" OSes with security updates, as tech people say, since their unstated policy seems to be that only bugs which Apple thinks are "exploited in the wild" will be backported to any OS that isn't the very latest; most (not all) other known security patches are never backported.[0] Apple's security stance is much worse than Microsoft's or Google's when comparing Apples to Apples. [0]: https://www.intego.com/mac-security-blog/apples-poor-patching-policies-potentially-make-users-security-and-privacy-precarious/ https://www.intego.com/mac-security-blog/apples-poor-patchin...
tptacek 5y ago
This will sound very weird, but I kind of hate that they include Google among the vendors they report to, provide a deadline and grace period for, and track responses from. It's actually not their responsibility to do anything like that; if Microsoft and Apple are unhappy that P0 is targeting them, they should respond by standing up their own P0 teams and hammering Google, rather than having everyone operate under the fiction that it's OK for Google to be the only major vendor doing this work. (I'm of course not saying P0 shouldn't target Google, just that Google shouldn't have to be publicly accountable to Google P0).
- lima 5y ago
  Why not? It really strengthens the message.
  - tptacek 5y ago
    I don't think it does strengthen the message, unless you think Google does such a good job responding to P0 that they're setting a standard Microsoft, Apple, and Adobe have to adhere to, and I think that's pretty debatable (the really important thing P0 does to set a standard is the 90 day deadline).
    - xxpor 5y ago
      It prevents google execs from burying P0 reports to other teams at Google though, which seems like the bigger risk?
  - deleted 5y ago
    [deleted]
  - jcims 5y ago
    What’s the value of the public’s opinion of their objectivity in this context? Let’s say that my mom thinks P0 are a bunch of bullies and blowhards. So what? I’m not saying there’s absolutely no brand damage to Google, but in the grand scheme it seems negligible outside of HN and similar venues.
- 5y ago
gsmith7890 5y ago
What was the most serious vulnerability or set of vulnerabilities identified by Project Zero?
- UncleMeat 5y ago
  Spectre is probably the most interesting one. As of right now it isn't as critical as other things since it is difficult to actually exploit but its implications are gargantuan, similar to the development of return-oriented-programming a while back.
- Godel_unicode 5y ago
  I'm partial to the watering hole attack they found with TAG that had a bunch of browser 0days in it, personally: https://googleprojectzero.blogspot.com/2021/01/introducing-in-wild-series.html https://googleprojectzero.blogspot.com/2021/01/introducing-i...
Terry_Roll 5y ago
Kudos for the TLDR. Interesting to not that vendors are taking bugs more seriously. Intelligence wise, I wonder what P) learn from the communication that takes place between P0 & the vendor, ie are there lots of questions asked over time giving insight into thought processes or does P0 see a massive data grab and then communication silence until the bug is fixed? For all software & hardware vendors, this will help raise the standards & revenue but it will also raise the barrier to entry for new entrants as sole developers or small teams will have to consider more than the basic function of their app/project/hw. Legislation like GDPR already means some projects may never get to fly today as regulatory burden is too great, and the bug front is another domain which is maturing adding to the burden.
taubek 5y ago
Do you think that faster response from Linux is due to fact that it is OSS or is corporate structure the reason for response longer time from big players?
zibzab 5y ago
Average days to fix: Linux at top, Oracle at the bottom. Why am I not surprised?

130 points by arkadiyt 5y ago

I dislike many aspects of google, but project zero is not one of them and has greatly improve the overall security of the industry. Also their blogs describing how security exploits work are always super interesting

burrows 5y ago
> [P0] has greatly improve the overall security of the industry. What is the argument for this?
- elisbce 5y ago
  Like the number of critical bugs they found and pushed the vendors to fix? Like finding bugs like Spectre that forever changed the chip industry's understanding of side channel attacks and subsequent designs?
- olliej 5y ago
  They've been very effective at getting many large companies to actually fix security bugs, and release updates with those fixes in an actually reasonable time frame. They've been very strict about their "we will publish the details of this security bug in 90 days unless you can provide a very good reason not to".
- tialaramex 5y ago
  Project Zero's argument is: > For nearly ten years, Google’s Project Zero has been working to make it more difficult for bad actors to find and exploit security vulnerabilities, significantly improving the security of the Internet for everyone. In that time, we have partnered with folks across industry to transform the way organizations prioritize and approach fixing security vulnerabilities and updating people’s software. This feels pretty reasonable, we don't have a spare universe to act as our control, but their intervention does seem likely to have made us more secure overall than otherwise.
  - burrows 5y ago
    Is “the security of the Internet” just a proxy for, “the number of vulns in the wild”?
    - tourist2d 5y ago
      Sure, you could also phrase that sentence a hundred other ways.

r00fus 5y ago

Is it meaningful to include "Linux" as a discrete vendor? How would you compare an OSS project to a company like Microsoft or Google?

that_guy_iain 5y ago
The Linux Kernel is a product and the Linux Foundation is it‘s vendor. I would assume they mean Them. Especially when they had Red Hat and Cannonical in „other“
- marcan_42 5y ago
  The Linux Foundation is not the vendor for Linux. "Vendor" here refers to the Linux kernel community and how it acts when security issues are discovered. That is, if you want to be pedantic, it's Linus Torvalds and everyone under him. It doesn't really matter that it's not a legal entity, just "whoever is responsible for fixing the bug and releasing the fix officially". In security we call open source projects vendors because they act as such just like Apple would.
  - that_guy_iain 5y ago
    Linus Trovalds works for the Linux Foundation? So if we're being pedantic, which I think you are, then If it's Linux Torvalds in his professional capcitiy then it is for Linux Foundation.
    - marcan_42 5y ago
      Linus Torvalds does not answer to the Linux Foundation, and the Linux Foundation has little to do with how the Linux kernel handles security reports. It doesn't matter that they sponsor him; they aren't the "vendor" for Linux in any meaningful sense. They are just a nonprofit entity established to support Linux development in various ways.
      - that_guy_iain 5y ago

deleted 5y ago

[deleted]

tester756 5y ago

that iOS vs Android table kinda makes no sense as they said iOS 76, Android Samsung 10, Android Pixel 6 >The first thing to note is that it appears that iOS received remarkably more bug reports from Project Zero than any flavor of Android did during this time period, but rather than an imbalance in research target selection, this is more a reflection of how Apple ships software. Security updates for "apps" such as iMessage, Facetime, and Safari/WebKit are all shipped as part of the OS updates, so we include those in the analysis of the operating system. On the other hand, security updates for standalone apps on Android happen through the Google Play Store, so they are not included here in this analysis. so kinda what's the point of putting that column there? people will use it as an argument that Android is safer :P

amscanne 5y ago
There are advantages and disadvantages to bundling applications with the core OS; having these security bugs become part of the OS release vehicle (along with the heavyweight process that implies) seems like a disadvantage. With respect to the table, I think there’s a decent argument either way.
concinds 5y ago
I would argue that Android is safer, specifically from 0days. Anyone with Android 6.0 or later (a 2015 OS) gets Chrome updates. Those updates are fast, automatic, and transparently happen in the background, with no user input or downtime (e.g. need to restart). They're also not arbitrarily delayed to fit into with some rigid OS update release schedule, like iOS or macOS. That's especially important in this era where people are skeptical of updates, or just too busy, and keep delaying and skipping updates. Every time I use a family member's device, and check the version, I see they haven't updated in months. Many aren't even aware there are updates available. I told the story before, but my grandma only has 4G, no Wi-Fi, and only has an iPhone, no laptop to download IPSWs from (and obviously average users can't do that); therefore she only gets iOS updates when I visit once or twice a year. Is Apple happy with that? Provocatively: is that the best they can do? Google proves it's not. Sure, the Android OS update situation isn't great, but your biggest 0day exposures will be apps exposed to the internet (Chrome, chat apps) and those all get regular, transparent background updates with no user input; and Android users are much safer for it, than they'd be if Google imitated Apple's OS update strategy. This is an Apple-wide problem, not just iOS. Anyone with OS X El Capitan 10.11 or later (again, a 2015 OS) is running the latest Google Chrome. Yet their Safari version is riddled with catastrophic 0day bugs (which can now be called 1,314-day bugs, since there have been 1,314 days since the last El Capitan security update). Add to that the fact that Apple doesn't really support "n-2" OSes with security updates, as tech people say, since their unstated policy seems to be that only bugs which Apple thinks are "exploited in the wild" will be backported to any OS that isn't the very latest; most (not all) other known security patches are never backported.[0] Apple's security stance is much worse than Microsoft's or Google's when comparing Apples to Apples. [0]: https://www.intego.com/mac-security-blog/apples-poor-patching-policies-potentially-make-users-security-and-privacy-precarious/ https://www.intego.com/mac-security-blog/apples-poor-patchin...

tptacek 5y ago

This will sound very weird, but I kind of hate that they include Google among the vendors they report to, provide a deadline and grace period for, and track responses from. It's actually not their responsibility to do anything like that; if Microsoft and Apple are unhappy that P0 is targeting them, they should respond by standing up their own P0 teams and hammering Google, rather than having everyone operate under the fiction that it's OK for Google to be the only major vendor doing this work. (I'm of course not saying P0 shouldn't target Google, just that Google shouldn't have to be publicly accountable to Google P0).

lima 5y ago
Why not? It really strengthens the message.
- tptacek 5y ago
  I don't think it does strengthen the message, unless you think Google does such a good job responding to P0 that they're setting a standard Microsoft, Apple, and Adobe have to adhere to, and I think that's pretty debatable (the really important thing P0 does to set a standard is the 90 day deadline).
  - xxpor 5y ago
    It prevents google execs from burying P0 reports to other teams at Google though, which seems like the bigger risk?
- deleted 5y ago
  [deleted]
- jcims 5y ago
  What’s the value of the public’s opinion of their objectivity in this context? Let’s say that my mom thinks P0 are a bunch of bullies and blowhards. So what? I’m not saying there’s absolutely no brand damage to Google, but in the grand scheme it seems negligible outside of HN and similar venues.
5y ago

gsmith7890 5y ago

What was the most serious vulnerability or set of vulnerabilities identified by Project Zero?

UncleMeat 5y ago
Spectre is probably the most interesting one. As of right now it isn't as critical as other things since it is difficult to actually exploit but its implications are gargantuan, similar to the development of return-oriented-programming a while back.
Godel_unicode 5y ago
I'm partial to the watering hole attack they found with TAG that had a bunch of browser 0days in it, personally: https://googleprojectzero.blogspot.com/2021/01/introducing-in-wild-series.html https://googleprojectzero.blogspot.com/2021/01/introducing-i...

Terry_Roll 5y ago

Kudos for the TLDR. Interesting to not that vendors are taking bugs more seriously. Intelligence wise, I wonder what P) learn from the communication that takes place between P0 & the vendor, ie are there lots of questions asked over time giving insight into thought processes or does P0 see a massive data grab and then communication silence until the bug is fixed? For all software & hardware vendors, this will help raise the standards & revenue but it will also raise the barrier to entry for new entrants as sole developers or small teams will have to consider more than the basic function of their app/project/hw. Legislation like GDPR already means some projects may never get to fly today as regulatory burden is too great, and the bug front is another domain which is maturing adding to the burden.

taubek 5y ago

Do you think that faster response from Linux is due to fact that it is OSS or is corporate structure the reason for response longer time from big players?

zibzab 5y ago

Average days to fix: Linux at top, Oracle at the bottom. Why am I not surprised?