5 ms·

The bad taste this left in my mouth was the glee with which it was conducted. Delisting a CA is a thing you can do. It is a serious thing you can do. But the

by ethbr0 4y ago

The bad taste this left in my mouth was the glee with which it was conducted. Delisting a CA is a thing you can do. It is a serious thing you can do. But the internet drama surrounding this, and some posts on the mailing list itself, were overly emotional. "Ha! We caught you! You damned executive, you!" is not exactly the tone with which I'd want to conduct weighty matters. It was like looking back into the early-90s, when pitchforks-and-torches-because-we-technically-can were the norm. I don't miss those days, and I'd thought the internet had grown up more. Apologies in advance for being the no-fun police, but we're talking about destroying a business. Even if it's a shitty business, run by shitty people, trying to do shitty things... just don't fucking dance. https://m.youtube.com/watch?v=0k5aVLi_yhM https://m.youtube.com/watch?v=0k5aVLi_yhM

akerl_ 4y ago
Can you help clarify where you're seeing glee, either now or in the early 90s?
lolinder 4y ago
If you look at the emails by Kathleen, she was very deliberate and professional in her phrasing. She only called Rachel out by name in a footnote, and never drew attention to the lie directly. I elided a whole lot of body text to put those two lines together, and even when combined Kathleen doesn't explicitly call her out for lying. That shows a lot of decorum and restraint on Kathleen's part, given the nonsense she was confronted with. There were definitely some people on the mailing list who were less than professional (and of course on Reddit and HN), but the main decision makers were very careful and professional throughout. Rachel is the only one out of the core people involved who lapsed into immaturity.
- ethbr0 4y ago
  Agreed. Kathleen and the browser reps, as far as I could tell, were nothing but professional and cordial. But by Rachel's fourth reply(?), the response tone seems to shift. Suddenly it's okay for the peanut gallery to snark. And I get it -- I probably couldn't have resisted a choice remark after being shoveled a load of what read like horsecrap. But when "I’m not sure how exactly you’re involved in the CA community, do you represent a CA? A browser perhaps? Or any of the governing bodies? [...] Or are you concerned citizen consumers?" is responded to with "I think all of us are easily found via Google", that's less than the gravitas I'd expect. And I find it fascinating HN cheered this one, when HN's 2nd favorite topic du jour is complaining about how large companies steamroll smaller entities whenever it suits them, without any standard process.
  - lolinder 4y ago
    Yes, I fully agree about Kurt's replies—they were unprofessional and added no value. The others that chimed in at the same time were more thoughtful (though equally scathing), but were probably colored a bit by proximity to Kurt's. > And I find it fascinating HN cheered this one, when HN's 2nd favorite topic du jour is complaining about how large companies steamroll smaller entities whenever it suits them, without any standard process. I think the main reason it drew so much attention is because the TrustCor rep made such a scene out of it. Evasion and deflection escalated into ad hominems and conspiracy theories. By the time it hit the HN front page, it was pretty obvious that TrustCor couldn't be trusted, so there wasn't room for any sympathy.
    - ethbr0 4y ago
      Call me a pathological contrarian, but I try to step extra carefully when dealing with a matter where there doesn't seem to be room for any sympathy. More often than not, it's those obviously-righteous times that have led to my behaving like the biggest ass. Better to do a happy thing with a smile on your face, and a grim thing with no expression at all.
    - kseifried 4y ago
      So my response: CA's sell trust. Billions of people trust them. They should be above reproach. A CA in the root certificate store has supposedly passed through an arduous set of processes and should be rock solid. While you may not like the tone of my comments, was there anything that was factually incorrect? I saw BS answers and called them out. Specifically as to her question: >Thank you all for chiming in. I’m not sure how exactly you’re involved in the CA community, do you represent a CA? A browser perhaps? Or any of the governing I mean... yeah. I've been doing Open Source security since 1998. https://seifried.org/lasg/ https://seifried.org/lasg/, https://seclists.org/bugtraq/1998/Apr/ https://seclists.org/bugtraq/1998/Apr/. in the last decade I was one the CVE Board (resigned), still on the CWE/CAPEC board, I do the https://opensourcesecuritypodcast.com/ https://opensourcesecuritypodcast.com/ (350+ episodes). I was 1/3 of oss-security mailing traffic for a few years thanks to being "the CVE assigner" and such. If you look at the rest of the dev-security-policy@mozilla.org list posters it's the same 2-3 dozen people posting for almost a decade (myself included, going back to 2010 or so, I have a spreadsheet somewhere but it's easy enough to confirm). Again: a CA should be like a bank, well run, regulated, and trustworthy, they should not be toppled over so easily by a group of random Internet volunteers. I would also point out that Microsoft retroactively removed them: https://groups.google.com/a/mozilla.org/g/dev-security-policy/c/oxX69KFvsm4/m/NxbhuW-4CgAJ https://groups.google.com/a/mozilla.org/g/dev-security-polic... so say what you will about my tone, but the facts are the facts. Also if anyone is interested we're looking at other root CA's, theres a few more that appear to be less then ideal. Heck within the last 3 months we've also had: SERPRO with MASSIVE problems (assigning certificates to URLs, "Bennar" and so on), they actually withdrew from the process because they need time to clean up their organization. https://groups.google.com/a/ccadb.org/g/public/c/Mux855BsRg4 https://groups.google.com/a/ccadb.org/g/public/c/Mux855BsRg4 bjcn.ca with again, spyware concerns and some other potential issues, https://groups.google.com/a/ccadb.org/g/public/c/o9lbCbr92Ug/m/lPkqrHF1DQAJ https://groups.google.com/a/ccadb.org/g/public/c/o9lbCbr92Ug... and there's a nice summary from 2 days ago: ========================================================================================= Summary of Discussion and Action Items Discussion Item #1: A concern was raised about BJCA’s Beijing One Pass software, which apparently facilitates client access to a digital portal or platform. It was noted that BJCA had attempted to address suspicions about the software in Comment #15, that the software was needed to support a USB token and to install another certificate chain, and not the two above-referenced roots. A follow-up question was whether a security report concerning the software would be made publicly available. BJCA Response to Discussion Item #1: “This report is a communication document between our company and the competent government department, and it is not suitable for disclosure or submission to Mozilla because it involves confidential information. And because the security incident does not involve the certificate chain of the root inclusion case submitted to Mozilla this time, we made a clarification in the Mozilla root inclusion case by disclosing the main points of the report.” ========================== Discussion Item #2: Two components were also mentioned: wmControl.exe and zfkeymonitor.exe. BJCA Response to Discussion Item #2: The “suspected spyware behavior indicated in the report was caused by one of the drivers, wmControl.exe. This program is a driver provided by the USB Token manufacturer, Its software behavior is different from spyware and does not have malicious behavior. It is intended to ensure the normal use of this type of [device] in the browser. In addition, the USB Token for digital certificate corresponding to the driver wmControl.exe is an old version device, and its driver has been deleted in the new version of the certificate environment software (version >= 3.6.8)”. Concerning zfkeymonitor.exe, BJCA responded that their software did not include the zfkeymonitor program. ========================== Discussion Item #3: Clarification was requested about root certificate installation by the One Pass software. BJCA Response to Discussion Item #3: BJCA reiterated that their software did not attempt to install the two roots, but stated, “in order to improve the user experience, the BJCA certificate environment software chooses to skip user confirmation during the installation process, which may cause doubts for users. At present, we have plans to adopt advanced options in the new version of the software, allowing users to choose whether to confirm the installation, and support users to choose to add certificates and updates to the current user's personal storage instead of the computer's trusted root or trusted third party storage. No doubt that there is an obvious contradiction between convenience and security, which could improve the software security but degrades the user experience and increase our operation costs.” According to BJCA, it maintains two separate systems: a global, public-trust system that meets international standards (WebTrust, CA/Browser Forum, etc.) and issues and manages SSL/TLS server certificates; and a national system that follows Chinese standards and issues and manages personal certificates, enterprise certificates and equipment certificates (e.g., Beijing One Pass software and certificate). BJCA acknowledges that both systems are under control of the same legal business entity, but for the latter, the software is not part of the global, public-trust system. BJCA says it “will also refer to the recommendations of experts, learn from the best practices of the public trust system, continue to innovate, practice corporate social responsibility, and strive to build a safe and reliable of cyberspace.” ========================== Conclusion We thank community members for their review and consideration during this period. Root Store Programs will make final inclusion decisions independently, on their own timelines, and based on each Root Store Member’s inclusion criteria. Further discussion may take place in the independently managed Root Store community forums (i.e., MDSP). ========================================================================================= If anyone is interested, we're looking into what can be done around the root certificate world, if you're interested feel free to reach out to me at kurt@seifried.org (I'm still in the exploratory phase, e.g. can anything actionable/useful be done, and so on).
- threeseed 4y ago
  > There were definitely some people on the mailing list who were less than professional Especially people like Kurt Seifried who appears to have no relationship to any of the browser vendors or parties involved. Just some random person demanding answers from Rachel as though he is a prosecutor. It doesn't seem to me like a public mailing list is necessarily the best way to handle this given the seriousness of the topic and the business impact to the company involved.
  - pas 4y ago
    it concerns millions of users directly and billions indirectly. yes we can trust Mozilla and the other browser vendors to do the right thing. we already trust them very explicitly when using their software. but the public by default approach is pretty good for these things. IETF, NANOG and other big org lists are also public. yes as all mailing lists there are occasional flareups. (and yes browser vendors or various CA/B forum members can still start a shadow list to conspire) ... but the business impact happened when they did the not-the-right thing, to minimize their impact they could have declared this an incident, suspend issuance, withdraw from the trusted set and apply again later. they again did the not-the-right thing by trying to bullshit their way to minimal impact. it's not exactly virtuous, but quite understandable to throw a few rotten eggs. (and the list could be CA/B members & candidates only).
    - kseifried 4y ago
      It's wild that people think there is some sort of "CA Police" looking out for bad CAs. Sorry but random people like myself demanding answers during the public discussion phase is actually what the "CA Police" looks like. If anyone is interested, we're looking into what can be done around the root certificate world, if you're interested feel free to reach out to me at kurt@seifried.org (I'm still in the exploratory phase, e.g. can anything actionable/useful be done, and so on).
  - rektide 4y ago
    Given the same facts, I come to opposite conclusion. To create trust, this business must be conducted in public & must be open access. Imposing any more barriers or concealing the dealings would be actively harmful to the process. Anyone is capable of presenting good data to inform these decisions, to show how & why trust should or should not be given, and that's great, that's how we let in the sunlight. It is just a mailing list. No one has to take commenters seriously. I found the tone here by the cimmunity to be quite fine. But even if it descended sitnificantly, became much less professional, I wouldnt be happy per se but I would not until a long long descent begin to question the value of the open process. It is up to the reader to figure out what credentials people have; as Kurt said, you've got to do your own searching to find out. That there is noise in the channel (in this case it did not seem to come from the community) is not something we should restrict the processs to avoid; it is something we must allow & be able to be tolerant & resillient to.
woodruffw 4y ago
I didn’t see that much glee in the thread. At most, maybe mild bemusement at how ridiculous the entire affair was, relative to how staid policy enforcement can be.
8note 4y ago
I can't say I've got sympathy for the business and it's owners, but my sympathies to the employees impacted is best focused on why employees must get so screwed over by the business going under, not on preserving the bad business. The business is the unimportant part
x0x0 4y ago
> but we're talking about destroying a business Yes, and that's good. Their business was our trust -- every one of our devices -- and they made it clear they didn't deserve it.
causality0 4y ago
It was like looking back into the early-90s, when pitchforks-and-torches-because-we-technically-can were the norm. I don't miss those days, and I'd thought the internet had grown up more. Yeah, the 90s, back when anti-trust was a thing and big companies occasionally actually got punished in meaningful ways for shitting all over society. Sure, it was just a small fish, but it's still deeply satisfying to see someone fail to get away with it because it's so rare.

by ethbr0 4y ago

akerl_ 4y ago

Can you help clarify where you're seeing glee, either now or in the early 90s?

lolinder 4y ago

If you look at the emails by Kathleen, she was very deliberate and professional in her phrasing. She only called Rachel out by name in a footnote, and never drew attention to the lie directly. I elided a whole lot of body text to put those two lines together, and even when combined Kathleen doesn't explicitly call her out for lying. That shows a lot of decorum and restraint on Kathleen's part, given the nonsense she was confronted with. There were definitely some people on the mailing list who were less than professional (and of course on Reddit and HN), but the main decision makers were very careful and professional throughout. Rachel is the only one out of the core people involved who lapsed into immaturity.

ethbr0 4y ago
Agreed. Kathleen and the browser reps, as far as I could tell, were nothing but professional and cordial. But by Rachel's fourth reply(?), the response tone seems to shift. Suddenly it's okay for the peanut gallery to snark. And I get it -- I probably couldn't have resisted a choice remark after being shoveled a load of what read like horsecrap. But when "I’m not sure how exactly you’re involved in the CA community, do you represent a CA? A browser perhaps? Or any of the governing bodies? [...] Or are you concerned citizen consumers?" is responded to with "I think all of us are easily found via Google", that's less than the gravitas I'd expect. And I find it fascinating HN cheered this one, when HN's 2nd favorite topic du jour is complaining about how large companies steamroll smaller entities whenever it suits them, without any standard process.
- lolinder 4y ago
  Yes, I fully agree about Kurt's replies—they were unprofessional and added no value. The others that chimed in at the same time were more thoughtful (though equally scathing), but were probably colored a bit by proximity to Kurt's. > And I find it fascinating HN cheered this one, when HN's 2nd favorite topic du jour is complaining about how large companies steamroll smaller entities whenever it suits them, without any standard process. I think the main reason it drew so much attention is because the TrustCor rep made such a scene out of it. Evasion and deflection escalated into ad hominems and conspiracy theories. By the time it hit the HN front page, it was pretty obvious that TrustCor couldn't be trusted, so there wasn't room for any sympathy.
  - ethbr0 4y ago
    Call me a pathological contrarian, but I try to step extra carefully when dealing with a matter where there doesn't seem to be room for any sympathy. More often than not, it's those obviously-righteous times that have led to my behaving like the biggest ass. Better to do a happy thing with a smile on your face, and a grim thing with no expression at all.
  - kseifried 4y ago
    So my response: CA's sell trust. Billions of people trust them. They should be above reproach. A CA in the root certificate store has supposedly passed through an arduous set of processes and should be rock solid. While you may not like the tone of my comments, was there anything that was factually incorrect? I saw BS answers and called them out. Specifically as to her question: >Thank you all for chiming in. I’m not sure how exactly you’re involved in the CA community, do you represent a CA? A browser perhaps? Or any of the governing I mean... yeah. I've been doing Open Source security since 1998. https://seifried.org/lasg/ https://seifried.org/lasg/, https://seclists.org/bugtraq/1998/Apr/ https://seclists.org/bugtraq/1998/Apr/. in the last decade I was one the CVE Board (resigned), still on the CWE/CAPEC board, I do the https://opensourcesecuritypodcast.com/ https://opensourcesecuritypodcast.com/ (350+ episodes). I was 1/3 of oss-security mailing traffic for a few years thanks to being "the CVE assigner" and such. If you look at the rest of the dev-security-policy@mozilla.org list posters it's the same 2-3 dozen people posting for almost a decade (myself included, going back to 2010 or so, I have a spreadsheet somewhere but it's easy enough to confirm). Again: a CA should be like a bank, well run, regulated, and trustworthy, they should not be toppled over so easily by a group of random Internet volunteers. I would also point out that Microsoft retroactively removed them: https://groups.google.com/a/mozilla.org/g/dev-security-policy/c/oxX69KFvsm4/m/NxbhuW-4CgAJ https://groups.google.com/a/mozilla.org/g/dev-security-polic... so say what you will about my tone, but the facts are the facts. Also if anyone is interested we're looking at other root CA's, theres a few more that appear to be less then ideal. Heck within the last 3 months we've also had: SERPRO with MASSIVE problems (assigning certificates to URLs, "Bennar" and so on), they actually withdrew from the process because they need time to clean up their organization. https://groups.google.com/a/ccadb.org/g/public/c/Mux855BsRg4 https://groups.google.com/a/ccadb.org/g/public/c/Mux855BsRg4 bjcn.ca with again, spyware concerns and some other potential issues, https://groups.google.com/a/ccadb.org/g/public/c/o9lbCbr92Ug/m/lPkqrHF1DQAJ https://groups.google.com/a/ccadb.org/g/public/c/o9lbCbr92Ug... and there's a nice summary from 2 days ago: ========================================================================================= Summary of Discussion and Action Items Discussion Item #1: A concern was raised about BJCA’s Beijing One Pass software, which apparently facilitates client access to a digital portal or platform. It was noted that BJCA had attempted to address suspicions about the software in Comment #15, that the software was needed to support a USB token and to install another certificate chain, and not the two above-referenced roots. A follow-up question was whether a security report concerning the software would be made publicly available. BJCA Response to Discussion Item #1: “This report is a communication document between our company and the competent government department, and it is not suitable for disclosure or submission to Mozilla because it involves confidential information. And because the security incident does not involve the certificate chain of the root inclusion case submitted to Mozilla this time, we made a clarification in the Mozilla root inclusion case by disclosing the main points of the report.” ========================== Discussion Item #2: Two components were also mentioned: wmControl.exe and zfkeymonitor.exe. BJCA Response to Discussion Item #2: The “suspected spyware behavior indicated in the report was caused by one of the drivers, wmControl.exe. This program is a driver provided by the USB Token manufacturer, Its software behavior is different from spyware and does not have malicious behavior. It is intended to ensure the normal use of this type of [device] in the browser. In addition, the USB Token for digital certificate corresponding to the driver wmControl.exe is an old version device, and its driver has been deleted in the new version of the certificate environment software (version >= 3.6.8)”. Concerning zfkeymonitor.exe, BJCA responded that their software did not include the zfkeymonitor program. ========================== Discussion Item #3: Clarification was requested about root certificate installation by the One Pass software. BJCA Response to Discussion Item #3: BJCA reiterated that their software did not attempt to install the two roots, but stated, “in order to improve the user experience, the BJCA certificate environment software chooses to skip user confirmation during the installation process, which may cause doubts for users. At present, we have plans to adopt advanced options in the new version of the software, allowing users to choose whether to confirm the installation, and support users to choose to add certificates and updates to the current user's personal storage instead of the computer's trusted root or trusted third party storage. No doubt that there is an obvious contradiction between convenience and security, which could improve the software security but degrades the user experience and increase our operation costs.” According to BJCA, it maintains two separate systems: a global, public-trust system that meets international standards (WebTrust, CA/Browser Forum, etc.) and issues and manages SSL/TLS server certificates; and a national system that follows Chinese standards and issues and manages personal certificates, enterprise certificates and equipment certificates (e.g., Beijing One Pass software and certificate). BJCA acknowledges that both systems are under control of the same legal business entity, but for the latter, the software is not part of the global, public-trust system. BJCA says it “will also refer to the recommendations of experts, learn from the best practices of the public trust system, continue to innovate, practice corporate social responsibility, and strive to build a safe and reliable of cyberspace.” ========================== Conclusion We thank community members for their review and consideration during this period. Root Store Programs will make final inclusion decisions independently, on their own timelines, and based on each Root Store Member’s inclusion criteria. Further discussion may take place in the independently managed Root Store community forums (i.e., MDSP). ========================================================================================= If anyone is interested, we're looking into what can be done around the root certificate world, if you're interested feel free to reach out to me at kurt@seifried.org (I'm still in the exploratory phase, e.g. can anything actionable/useful be done, and so on).
threeseed 4y ago
> There were definitely some people on the mailing list who were less than professional Especially people like Kurt Seifried who appears to have no relationship to any of the browser vendors or parties involved. Just some random person demanding answers from Rachel as though he is a prosecutor. It doesn't seem to me like a public mailing list is necessarily the best way to handle this given the seriousness of the topic and the business impact to the company involved.
- pas 4y ago
  it concerns millions of users directly and billions indirectly. yes we can trust Mozilla and the other browser vendors to do the right thing. we already trust them very explicitly when using their software. but the public by default approach is pretty good for these things. IETF, NANOG and other big org lists are also public. yes as all mailing lists there are occasional flareups. (and yes browser vendors or various CA/B forum members can still start a shadow list to conspire) ... but the business impact happened when they did the not-the-right thing, to minimize their impact they could have declared this an incident, suspend issuance, withdraw from the trusted set and apply again later. they again did the not-the-right thing by trying to bullshit their way to minimal impact. it's not exactly virtuous, but quite understandable to throw a few rotten eggs. (and the list could be CA/B members & candidates only).
  - kseifried 4y ago
    It's wild that people think there is some sort of "CA Police" looking out for bad CAs. Sorry but random people like myself demanding answers during the public discussion phase is actually what the "CA Police" looks like. If anyone is interested, we're looking into what can be done around the root certificate world, if you're interested feel free to reach out to me at kurt@seifried.org (I'm still in the exploratory phase, e.g. can anything actionable/useful be done, and so on).
- rektide 4y ago
  Given the same facts, I come to opposite conclusion. To create trust, this business must be conducted in public & must be open access. Imposing any more barriers or concealing the dealings would be actively harmful to the process. Anyone is capable of presenting good data to inform these decisions, to show how & why trust should or should not be given, and that's great, that's how we let in the sunlight. It is just a mailing list. No one has to take commenters seriously. I found the tone here by the cimmunity to be quite fine. But even if it descended sitnificantly, became much less professional, I wouldnt be happy per se but I would not until a long long descent begin to question the value of the open process. It is up to the reader to figure out what credentials people have; as Kurt said, you've got to do your own searching to find out. That there is noise in the channel (in this case it did not seem to come from the community) is not something we should restrict the processs to avoid; it is something we must allow & be able to be tolerant & resillient to.

woodruffw 4y ago

I didn’t see that much glee in the thread. At most, maybe mild bemusement at how ridiculous the entire affair was, relative to how staid policy enforcement can be.

8note 4y ago

I can't say I've got sympathy for the business and it's owners, but my sympathies to the employees impacted is best focused on why employees must get so screwed over by the business going under, not on preserving the bad business. The business is the unimportant part

x0x0 4y ago

> but we're talking about destroying a business Yes, and that's good. Their business was our trust -- every one of our devices -- and they made it clear they didn't deserve it.

causality0 4y ago

It was like looking back into the early-90s, when pitchforks-and-torches-because-we-technically-can were the norm. I don't miss those days, and I'd thought the internet had grown up more. Yeah, the 90s, back when anti-trust was a thing and big companies occasionally actually got punished in meaningful ways for shitting all over society. Sure, it was just a small fish, but it's still deeply satisfying to see someone fail to get away with it because it's so rare.