9 ms·

The Chrome Root Program

92 points by dogecoinbase 6y ago

est31 6y ago
One major criticism of Firefox having its own roots was that it made life harder for rolling out custom roots in a corporate setting. You couldn't just use the OS provided means to distribute these roots, you had to use a Firefox specific way to add them. I guess this has similar implications for Chrome as well... Overall I'm in favour of this change though. Especially on Linux, distros haven't really proven to be good guardians of the root store. See the cons section in the rustls-native-certs readme: https://github.com/ctz/rustls-native-certs#should-i-use-this-or-webpki-roots https://github.com/ctz/rustls-native-certs#should-i-use-this...
- ocdtrekkie 6y ago
  Firefox has a single policy setting to use the OSes root store.
  - floatingatoll 6y ago
    It could be said that having to configure/test/deploy/maintain an "enterprise policy" for any given application is, technically, more work than not having to specify a policy for that application. The number of settings involved isn't strictly relevant to the difficulty of specifying a per-application policy, though it's certainly an opportunity for further complications!
    - sudosysgen 6y ago
      If you're deploying Firefox to begin with you can easily add this setting to your deploy system. If you aren't, then you probably don't have a customized CA policy.
- rockwotj 6y ago
  It seems the mkcert tool supports chrome and I believe Google has its own certs internally, so there's gotta be a way to do this - although it may be tied up with gsuite.
- sleevi 6y ago
  https://g.co/chrome/root-policy https://g.co/chrome/root-policy , which is linked to in that mail, states in the third paragraph: > If you’re an enterprise managing trusted CAs for your organization, including locally installed enterprise CAs, the policies described in this document do not apply to your CA. No changes are currently planned for how enterprise administrators manage those CAs within Chrome. CAs that have been installed by the device owner or administrator into the operating system trust store are expected to continue to work as they do today.
  - est31 6y ago
    > CAs that have been installed by the device owner or administrator into the operating system trust store are expected to continue to work as they do today. Huh that's a really cool idea. I wonder how it's implemented, as it might be useful for the rustls-native-certs crate as well... Edit: found it: Apparently there is a bool field is_issued_by_known_root that gets set by the differerent backends. https://chromium.googlesource.com/chromium/src/net/+/2f965222e5a39c960b1cc8c854025cb90eeb330a/cert/cert_verify_result.h#98 https://chromium.googlesource.com/chromium/src/net/+/2f96522...
    - sleevi 6y ago
      No, that’s not it: none of the code has landed yet It’s a big enough change that we’re explaining before landing. We’re taking an approach similar to Firefox’s implementation ( https://searchfox.org/mozilla-central/source/security/manager/ssl/EnterpriseRoots.cpp https://searchfox.org/mozilla-central/source/security/manage... ), which I helped advise on in the long ago days. It may end up using like that bool, but if you read through the commit history for how that code in Chrome works, you can see there are a lot of sharp edges.
      - deleted 6y ago
        [deleted]
      - jiggawatts 6y ago
        Firefox does Enterprise PKI wrong though. Very thoroughly wrong. It only reads in the Root CA certificates in the Machine store. It ignores the User store entirely. It ignores the Intermediate CA store as well. I bet Mozilla's logic is that Intermediate CA certificates can simply be "downloaded" dynamically, but Windows Server CAs use LDAP URLs by default, which Firefox doesn't support. It also ignores CTLs, CRLs, revoked certificates, and a bunch of other Windows PKI stores. Chrome was pleasant to use in that "it just worked", and this was a major reason it was adopted by large organisations and Firefox wasn't. Please don't break this.
- AviationAtom 6y ago
  There was talk about how the expiration of Let's Encrypt's intermediate cert at the end of the year will have ramifications, as Let's Encrypt's root cert is not included in older versions of Android, which much of the developing world makes use of. The suggested fix was that they download Firefox, due to it staying updated on older versions of Android, in addition to having it's own certificate store.
- silly-silly 6y ago
  Why havn't they been good guardians ?
  - est31 6y ago
    IDK they are volunteers and I respect their work. But they don't follow Mozilla as quickly as one could follow when it comes to removing CAs. E.g. in June 2019, Mozilla removed the Certinomis CA [0], but it took until June 2020 until Debian released a new version that had the CA removed [1] (version 20200601 of the ca-certificates package). [0]: https://bugzilla.mozilla.org/show_bug.cgi?id=1552374 https://bugzilla.mozilla.org/show_bug.cgi?id=1552374 [1]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=956411 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=956411
    - jart 6y ago
      My read here is that Debian was asked to upgrade ca-certificates in April 2020 and they rolled out an update within two weeks. That sounds reasonable. I'd be more concerned about Mozilla. Why did they include the Certinomis CA in the first place? I did a Google search and it appears to be a small business in France that's owned by a stamp printing company. They should totally have the power to read my emails. Quick grep of Mozilla root file tells me they grant 138 different companies root access to SSL. Why do we need that many people trying to make a buck selling prime numbers online? It's gonzo.
- deepstack 6y ago
  In some ways I get why software developers want to roll their own CA, less dependancy on the OS. There are several times when I visit a web site using latest browser, but get an error while viewing a SSL page because OS CA is too old, and there is no way to upgrade it. I'm talking win xp days of course. But yeah why rely on the OS. EDIT: kinda like how node_module thing got started in the first place. But I can see how this can turn into another browser war with Google and Mozilla each starting to bundle their own CA bundles. God, can we just please have DHT type for Root CA already?
- m-p-3 6y ago
  One benefit from this is that some Android versions aren't receiving updated root certs anymore, and older ones will encounter issues when browsing secured websites using LetsEncrypt.[1] Because Firefox stores its own root store it is unaffected by this, and Chrome will likely be unaffected to when this change happens. [1]: https://www.androidpolice.com/2020/11/07/many-websites-will-stop-working-on-older-android-versions-in-2021/ https://www.androidpolice.com/2020/11/07/many-websites-will-...
diogenesjunior 6y ago
I like Chrome OS, but a lot of things have been lacking lately.
- est31 6y ago
  Note that this is about Chrome, the browser, not Chrome OS, the operating system. Chrome OS has its own root store which apparently based on the Gentoo root store: > The certificates in this store are installed as a shared library (libnss) by the package chromiumos-overlay/dev-libs/nss. It is a fork of a Gentoo upstream package with the same name, with added/removed certificates based on Google's policy. https://chromium.googlesource.com/chromiumos/docs/+/master/ca_certs.md https://chromium.googlesource.com/chromiumos/docs/+/master/c... Given that Chrome now gets its own root program, I wonder what the future of this separate root store is.
  - sleevi 6y ago
    The overlay is only used by Chrome on ChromeOS (the doc goes into some of the details there), so effectively, the “Chrome OS” root store is being sunset. On a practical, technical level, I’ll probably just leave the overlay in place because libnss gets grumpy when there’s no ckbi shared object to load. Chrome itself won’t be consulting ckbi (i.e. it will take the same path as Linux, which will be to consult the Chrome store, even on Chromium OS builds, and only use NSS for user-added roots and client certs). Whether we keep that overlay in sync with the Chrome store, or stop patching it and let it match upstream, it’s neither here nor there, and mostly just about what ends up being easiest and causing the least confusion. I’m lazy, but in practice the contents there won’t matter, since it’ll be baked into Chrome proper. The etc-based store is a little more complicated, but it’s also not used by Chrome on CrOS, nor by the system apps that only talk to Google (they use the Google service set from pki.goog). Any code using that set doesn’t use the Chromium cert verifier, which as the mail and https://g.co/chrome/root-policy https://g.co/chrome/root-policy explain, go sort of hand in hand with the root program, so it may just continue to use upstream as it does today.
strombofulous 6y ago
Anybody who thinks this isn't being done as a way to stop people from MITMing traffic, just like they did on Android, is fooling themselves. A few years ago google started to move into the "extinguish" phase, and it's only becoming more apparent every time they announce a change
- tptacek 6y ago
  That doesn't even make sense. People who MITM traffic don't get Web PKI certs to do it with; they inject their own CA certs, a feature that continues to exist in the Chrome root program world and, indeed, has to in order for enterprises to deploy Chrome.
- sleevi 6y ago
  Happy to be up front here: as we call out in https://g.co/chrome/root-policy https://g.co/chrome/root-policy , the goal is to support the things the user installed and authorized. The priority of constituencies for the Web is a little different than on Android. Android gives a lot of deference to app developers, even against user wishes (think things like screenshot detection or making it hard to intercept traffic without rooting). This gives it a more “secure” reputation for some purposes, but certainly isn’t always what users want. The web has a different set of priorities, as spelled out in https://www.w3.org/TR/html-design-principles/#priority-of-constituencies https://www.w3.org/TR/html-design-principles/#priority-of-co... I’m not going to argue which is better, but I will acknowledge there are tradeoffs in both. Chrome, the browser, tries to be the user’s agent to the Web. Sometimes, however, being the users agent isn’t always cut and dry obvious. Extensions are a great way to empower users to mix the Web as they want, but they can also be a really harmful abuse vector. Finding the balance is tricky, and I know folks aren’t always happy with how that balance is struck, but a lot of thought and community engagement definitely goes in to it. Now, how does this relate to what you were speculating about? Well, installing roots is similarly one of those things that’s dual-use. I love tools like Fiddler and CharlesProxy, and have fought hard to make sure they work well (see the token binding discussions, which would have totally broke these tools). These tools use locally installed certs to really empower developers and users, and I don’t want to break that! But there’s also stuff like https://security.googleblog.com/2019/08/protecting-chrome-users-in-kazakhstan.html https://security.googleblog.com/2019/08/protecting-chrome-us... , which can completely destroy users’ privacy and security, using locally installed roots. So, like with extensions, there’s a balance to be had, between enabling great use cases, while also protecting users from abusive things. Obviously, as you note, problems like this could be “solved” by not allowing locally installed roots at all. When Kazakhstan required a root to be installed to compromise traffic, for example, Android and iOS users were largely protected precisely because “install this root” doesn’t really work as KZ intended. That’s great! But it’s also got downsides, like breaking Fiddler and Charles. I can absolutely say that the reason for this new root program is exactly what is said on the policy, and similar to Mozilla’s explanation at https://blog.mozilla.org/security/2019/02/14/why-does-mozilla-maintain-our-own-root-certificate-store/ https://blog.mozilla.org/security/2019/02/14/why-does-mozill... . This isn’t about trying to block all locally installed certs, like Android, but about providing good, safe, cross-platform and interoperable defaults. But I also will acknowledge that there totally are concerns about abusive roots, whether from Unwanted Software, device compromise, or the like, and this will give better tools to help communicate, explain, and warn users as appropriate, so they have a chance to make an informed decision. For example, this can close off quite a few abuse vectors for malware, today, and that’s already a win. However, any big changes to how locally installed roots work are quite a while away, and, like so many of the changes that involve complex tradeoffs, will involve a lot of research to understand users’ needs and desires and use cases. So no, this isn’t some secret plot to prevent users from managing their device as they see fit, or from choosing the CAs they trust.
xfalcox 6y ago
I wonder if this will come before Let's Encrypt cut date for old Androids. Looks like no...
rbirkby 6y ago
What about Edge and other Chromium browsers? For Edgium on Linux, will this likely result in a change of store?
londons_explore 6y ago
This seems like a lot of work by Chrome developers and the community, for little gain. What's wrong with just using the OS trust store and code/logic, perhaps with a whitelist/blacklist for anything particularly problematic where the OS trust store isn't updated quick enough? Is Chrome going to decide that NTFS file operations are too slow next and implement their own ChromeFS in a little Chrome partition? No... Use what the OS provides...
- dubcanada 6y ago
  This is related to Android. They are doing it so they can use it on Android.
  - sleevi 6y ago
    No more related to Android than any other Chrome supported OS. The post linked in the article, https://g.co/chrome/root-policy https://g.co/chrome/root-policy makes it clear: the goal is to provide a consistent, cross-platform experience to the Web. Users, and developers, hate “it doesn’t work on my machine” bugs, which are incredibly common. Sites that work on Chrome on macOS 10, but not macOS 11, or work on Chrome on Windows 10 but not Chrome on Windows 7. Giving developers and site operators predictable guidance, so that it “works in Chrome, works in Firefox” is good, no different than what Web Platform features you can use as https://caniuse.com https://caniuse.com Yes, Android is important, but it’s a bit like saying the forest exists because of this specific tree here, when in fact it’s made up of hundreds of trees, of all sorts of types.

92 points by dogecoinbase 6y ago

One major criticism of Firefox having its own roots was that it made life harder for rolling out custom roots in a corporate setting. You couldn't just use the OS provided means to distribute these roots, you had to use a Firefox specific way to add them. I guess this has similar implications for Chrome as well... Overall I'm in favour of this change though. Especially on Linux, distros haven't really proven to be good guardians of the root store. See the cons section in the rustls-native-certs readme: https://github.com/ctz/rustls-native-certs#should-i-use-this-or-webpki-roots https://github.com/ctz/rustls-native-certs#should-i-use-this...

ocdtrekkie 6y ago
Firefox has a single policy setting to use the OSes root store.
- floatingatoll 6y ago
  It could be said that having to configure/test/deploy/maintain an "enterprise policy" for any given application is, technically, more work than not having to specify a policy for that application. The number of settings involved isn't strictly relevant to the difficulty of specifying a per-application policy, though it's certainly an opportunity for further complications!
  - sudosysgen 6y ago
    If you're deploying Firefox to begin with you can easily add this setting to your deploy system. If you aren't, then you probably don't have a customized CA policy.
rockwotj 6y ago
It seems the mkcert tool supports chrome and I believe Google has its own certs internally, so there's gotta be a way to do this - although it may be tied up with gsuite.
sleevi 6y ago
https://g.co/chrome/root-policy https://g.co/chrome/root-policy , which is linked to in that mail, states in the third paragraph: > If you’re an enterprise managing trusted CAs for your organization, including locally installed enterprise CAs, the policies described in this document do not apply to your CA. No changes are currently planned for how enterprise administrators manage those CAs within Chrome. CAs that have been installed by the device owner or administrator into the operating system trust store are expected to continue to work as they do today.
- est31 6y ago
  > CAs that have been installed by the device owner or administrator into the operating system trust store are expected to continue to work as they do today. Huh that's a really cool idea. I wonder how it's implemented, as it might be useful for the rustls-native-certs crate as well... Edit: found it: Apparently there is a bool field is_issued_by_known_root that gets set by the differerent backends. https://chromium.googlesource.com/chromium/src/net/+/2f965222e5a39c960b1cc8c854025cb90eeb330a/cert/cert_verify_result.h#98 https://chromium.googlesource.com/chromium/src/net/+/2f96522...
  - sleevi 6y ago
    No, that’s not it: none of the code has landed yet It’s a big enough change that we’re explaining before landing. We’re taking an approach similar to Firefox’s implementation ( https://searchfox.org/mozilla-central/source/security/manager/ssl/EnterpriseRoots.cpp https://searchfox.org/mozilla-central/source/security/manage... ), which I helped advise on in the long ago days. It may end up using like that bool, but if you read through the commit history for how that code in Chrome works, you can see there are a lot of sharp edges.
    - deleted 6y ago
      [deleted]
    - jiggawatts 6y ago
      Firefox does Enterprise PKI wrong though. Very thoroughly wrong. It only reads in the Root CA certificates in the Machine store. It ignores the User store entirely. It ignores the Intermediate CA store as well. I bet Mozilla's logic is that Intermediate CA certificates can simply be "downloaded" dynamically, but Windows Server CAs use LDAP URLs by default, which Firefox doesn't support. It also ignores CTLs, CRLs, revoked certificates, and a bunch of other Windows PKI stores. Chrome was pleasant to use in that "it just worked", and this was a major reason it was adopted by large organisations and Firefox wasn't. Please don't break this.
AviationAtom 6y ago
There was talk about how the expiration of Let's Encrypt's intermediate cert at the end of the year will have ramifications, as Let's Encrypt's root cert is not included in older versions of Android, which much of the developing world makes use of. The suggested fix was that they download Firefox, due to it staying updated on older versions of Android, in addition to having it's own certificate store.
silly-silly 6y ago
Why havn't they been good guardians ?
- est31 6y ago
  IDK they are volunteers and I respect their work. But they don't follow Mozilla as quickly as one could follow when it comes to removing CAs. E.g. in June 2019, Mozilla removed the Certinomis CA [0], but it took until June 2020 until Debian released a new version that had the CA removed [1] (version 20200601 of the ca-certificates package). [0]: https://bugzilla.mozilla.org/show_bug.cgi?id=1552374 https://bugzilla.mozilla.org/show_bug.cgi?id=1552374 [1]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=956411 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=956411
  - jart 6y ago
    My read here is that Debian was asked to upgrade ca-certificates in April 2020 and they rolled out an update within two weeks. That sounds reasonable. I'd be more concerned about Mozilla. Why did they include the Certinomis CA in the first place? I did a Google search and it appears to be a small business in France that's owned by a stamp printing company. They should totally have the power to read my emails. Quick grep of Mozilla root file tells me they grant 138 different companies root access to SSL. Why do we need that many people trying to make a buck selling prime numbers online? It's gonzo.
deepstack 6y ago
In some ways I get why software developers want to roll their own CA, less dependancy on the OS. There are several times when I visit a web site using latest browser, but get an error while viewing a SSL page because OS CA is too old, and there is no way to upgrade it. I'm talking win xp days of course. But yeah why rely on the OS. EDIT: kinda like how node_module thing got started in the first place. But I can see how this can turn into another browser war with Google and Mozilla each starting to bundle their own CA bundles. God, can we just please have DHT type for Root CA already?
m-p-3 6y ago
One benefit from this is that some Android versions aren't receiving updated root certs anymore, and older ones will encounter issues when browsing secured websites using LetsEncrypt.[1] Because Firefox stores its own root store it is unaffected by this, and Chrome will likely be unaffected to when this change happens. [1]: https://www.androidpolice.com/2020/11/07/many-websites-will-stop-working-on-older-android-versions-in-2021/ https://www.androidpolice.com/2020/11/07/many-websites-will-...

diogenesjunior 6y ago

I like Chrome OS, but a lot of things have been lacking lately.

est31 6y ago
Note that this is about Chrome, the browser, not Chrome OS, the operating system. Chrome OS has its own root store which apparently based on the Gentoo root store: > The certificates in this store are installed as a shared library (libnss) by the package chromiumos-overlay/dev-libs/nss. It is a fork of a Gentoo upstream package with the same name, with added/removed certificates based on Google's policy. https://chromium.googlesource.com/chromiumos/docs/+/master/ca_certs.md https://chromium.googlesource.com/chromiumos/docs/+/master/c... Given that Chrome now gets its own root program, I wonder what the future of this separate root store is.
- sleevi 6y ago
  The overlay is only used by Chrome on ChromeOS (the doc goes into some of the details there), so effectively, the “Chrome OS” root store is being sunset. On a practical, technical level, I’ll probably just leave the overlay in place because libnss gets grumpy when there’s no ckbi shared object to load. Chrome itself won’t be consulting ckbi (i.e. it will take the same path as Linux, which will be to consult the Chrome store, even on Chromium OS builds, and only use NSS for user-added roots and client certs). Whether we keep that overlay in sync with the Chrome store, or stop patching it and let it match upstream, it’s neither here nor there, and mostly just about what ends up being easiest and causing the least confusion. I’m lazy, but in practice the contents there won’t matter, since it’ll be baked into Chrome proper. The etc-based store is a little more complicated, but it’s also not used by Chrome on CrOS, nor by the system apps that only talk to Google (they use the Google service set from pki.goog). Any code using that set doesn’t use the Chromium cert verifier, which as the mail and https://g.co/chrome/root-policy https://g.co/chrome/root-policy explain, go sort of hand in hand with the root program, so it may just continue to use upstream as it does today.

strombofulous 6y ago

Anybody who thinks this isn't being done as a way to stop people from MITMing traffic, just like they did on Android, is fooling themselves. A few years ago google started to move into the "extinguish" phase, and it's only becoming more apparent every time they announce a change

tptacek 6y ago
That doesn't even make sense. People who MITM traffic don't get Web PKI certs to do it with; they inject their own CA certs, a feature that continues to exist in the Chrome root program world and, indeed, has to in order for enterprises to deploy Chrome.
sleevi 6y ago
Happy to be up front here: as we call out in https://g.co/chrome/root-policy https://g.co/chrome/root-policy , the goal is to support the things the user installed and authorized. The priority of constituencies for the Web is a little different than on Android. Android gives a lot of deference to app developers, even against user wishes (think things like screenshot detection or making it hard to intercept traffic without rooting). This gives it a more “secure” reputation for some purposes, but certainly isn’t always what users want. The web has a different set of priorities, as spelled out in https://www.w3.org/TR/html-design-principles/#priority-of-constituencies https://www.w3.org/TR/html-design-principles/#priority-of-co... I’m not going to argue which is better, but I will acknowledge there are tradeoffs in both. Chrome, the browser, tries to be the user’s agent to the Web. Sometimes, however, being the users agent isn’t always cut and dry obvious. Extensions are a great way to empower users to mix the Web as they want, but they can also be a really harmful abuse vector. Finding the balance is tricky, and I know folks aren’t always happy with how that balance is struck, but a lot of thought and community engagement definitely goes in to it. Now, how does this relate to what you were speculating about? Well, installing roots is similarly one of those things that’s dual-use. I love tools like Fiddler and CharlesProxy, and have fought hard to make sure they work well (see the token binding discussions, which would have totally broke these tools). These tools use locally installed certs to really empower developers and users, and I don’t want to break that! But there’s also stuff like https://security.googleblog.com/2019/08/protecting-chrome-users-in-kazakhstan.html https://security.googleblog.com/2019/08/protecting-chrome-us... , which can completely destroy users’ privacy and security, using locally installed roots. So, like with extensions, there’s a balance to be had, between enabling great use cases, while also protecting users from abusive things. Obviously, as you note, problems like this could be “solved” by not allowing locally installed roots at all. When Kazakhstan required a root to be installed to compromise traffic, for example, Android and iOS users were largely protected precisely because “install this root” doesn’t really work as KZ intended. That’s great! But it’s also got downsides, like breaking Fiddler and Charles. I can absolutely say that the reason for this new root program is exactly what is said on the policy, and similar to Mozilla’s explanation at https://blog.mozilla.org/security/2019/02/14/why-does-mozilla-maintain-our-own-root-certificate-store/ https://blog.mozilla.org/security/2019/02/14/why-does-mozill... . This isn’t about trying to block all locally installed certs, like Android, but about providing good, safe, cross-platform and interoperable defaults. But I also will acknowledge that there totally are concerns about abusive roots, whether from Unwanted Software, device compromise, or the like, and this will give better tools to help communicate, explain, and warn users as appropriate, so they have a chance to make an informed decision. For example, this can close off quite a few abuse vectors for malware, today, and that’s already a win. However, any big changes to how locally installed roots work are quite a while away, and, like so many of the changes that involve complex tradeoffs, will involve a lot of research to understand users’ needs and desires and use cases. So no, this isn’t some secret plot to prevent users from managing their device as they see fit, or from choosing the CAs they trust.

xfalcox 6y ago

I wonder if this will come before Let's Encrypt cut date for old Androids. Looks like no...

rbirkby 6y ago

What about Edge and other Chromium browsers? For Edgium on Linux, will this likely result in a change of store?

londons_explore 6y ago

This seems like a lot of work by Chrome developers and the community, for little gain. What's wrong with just using the OS trust store and code/logic, perhaps with a whitelist/blacklist for anything particularly problematic where the OS trust store isn't updated quick enough? Is Chrome going to decide that NTFS file operations are too slow next and implement their own ChromeFS in a little Chrome partition? No... Use what the OS provides...

dubcanada 6y ago
This is related to Android. They are doing it so they can use it on Android.
- sleevi 6y ago
  No more related to Android than any other Chrome supported OS. The post linked in the article, https://g.co/chrome/root-policy https://g.co/chrome/root-policy makes it clear: the goal is to provide a consistent, cross-platform experience to the Web. Users, and developers, hate “it doesn’t work on my machine” bugs, which are incredibly common. Sites that work on Chrome on macOS 10, but not macOS 11, or work on Chrome on Windows 10 but not Chrome on Windows 7. Giving developers and site operators predictable guidance, so that it “works in Chrome, works in Firefox” is good, no different than what Web Platform features you can use as https://caniuse.com https://caniuse.com Yes, Android is important, but it’s a bit like saying the forest exists because of this specific tree here, when in fact it’s made up of hundreds of trees, of all sorts of types.