11 ms·

Zed's new project: Vulnerability Arbitration

238 points by acangiano 16y ago

lordmatty 16y ago
Cool idea - hope he has some good legal cover :)
- tptacek 16y ago
  Huh? Why?
snagg 16y ago
I think this approach exposes a number of problems that are pretty relevant. First off vendors by decrypting the content of the researcher's submission are basically giving up vulnerabilities to the bad guys that want to target unpatched systems (read: what malware most of the time does). Second thing I see no reasons why the vulnerability disclosure process should change and migrate to SSL based encryption: vendors can already expose "lies" if they feel like it with PGP encryption and honestly having a "middle-man" in between that can potentially have sensible data looks like a bad idea. So how about: researchers post on the website the SHA-1 of the POC or just a line saying "product X is vulnerable", then the owner of the website asks the vendor for confirmation. If the researcher has no "karma points" then the submission is hold back until the vendor confirms, if the researcher has "karma points" (already multiple confirmed and valuable submissions) then the advisory gets published immediately regardless of whether the vendor acknowledges it or not. Still does it actually help anybody? how would you convince consumers to actually pick products or similar based on that website? To me it looks like the people interested in this kind of information have already other means (twitter, ml, direct contact with vendors), the others don't care.
- zedshaw 16y ago
  > First off vendors by decrypting the content of the researcher's submission are basically giving up vulnerabilities to the bad guys that want to target unpatched systems They already do this. After a vulnerability is patched and fixed they release what it did. > Second thing I see no reasons why the vulnerability disclosure process should change and migrate to SSL based encryption It's a proposed first step, and makes sure that no vendor can avoid the system by simply not publishing a key. I envision in the future their GPG key would also be on the site and could be used. > vendors can already expose "lies" if they feel like it with PGP encryption and honestly having a "middle-man" in between that can potentially have sensible data looks like a bad idea. Uh, not sure what the "middle-man" is, but if you mean vulnarb.com then no, the point is that it's industry standard asymmetric crypto so I wouldn't know anything. In fact, I'd have incentive to not know anything so that I'm not getting sued. Finally, your proposed karma points system doesn't seem to solve anything. If everything is encrypted so that the receiver is only able to decrypt (not even the sender can do that), then there's no point in preventing people from publishing. Are you afraid that someone would slander a company? Couldn't the company simply decrypt and publish their lies then sue them for slander?
  - snagg 16y ago
    > They already do this. After a vulnerability is patched and fixed they release what it did. Usually researchers' submissions are way more detailed than the advisories you see from vendors. Meaning that you cannot just decrypt the content of what the research submitted. It's true that oftentimes you can find the bug by reading the advisory and using tools to diff the patch but it's a long shot compared to just publishing the original researcher's submission imo. >Uh, not sure what the "middle-man" is, but if you mean vulnarb.com then no, the point is that it's industry standard asymmetric crypto so I wouldn't know anything. In fact, I'd have incentive to not know anything so that I'm not getting sued. yep I meant vulnarb.com. I'm not saying that you have anything decrypted and ready to use, I'm saying that it's pointless to have another recipient for sensible data. Cause in the extreme scenario where the private key is stolen than it's just more attack surface to get to the submissions. If we assume that no leak of this sort happens, still what's the reason for a third party to have this sort of data compared to say sha-1 hash of the Poc? Actually my point about the karma system is that you can achieve the same goal you have in mind without needing any sort of data from the researcher other than "product X is vulnerable" and then the karma points will determine whether the researcher is reliable or not.
    - zedshaw 16y ago
      > Actually my point about the karma system is that you can achieve the same goal you have in mind without needing any sort of data from the researcher other than "product X is vulnerable" and then the karma points will determine whether the researcher is reliable or not. No, that doesn't actually solve this problem because then it just turns into a he-said-she-said. Also, you seem to leave out that the company can decide to decrypt it or not. Hell, they might come to an agreement with the researcher to just have the researcher assert it's fixed and never decrypt it. Although, I think that it should be fully exposed so people can see how it's really done and there's incentive to really fix it, not half-ass fix it.
      - snagg 16y ago
        >No, that doesn't actually solve this problem because then it just turns into a he-said-she-said. then I fail to understand what you plan to do with the encrypted stuff that you get from the researcher since the only one able to decrypt it would be the vendor. At that point the scenario is: researcher says this is vulnerable, vendor denies it, somebody needs to decrypt the content of what the researcher has sent to the vendor. In contrast what happens without having the encrypted content is: researcher says this is vulnerable, vendor denies it, the researcher can publish the un-encrypted original advisory on your website if he feels like it. Regardless the SSL trick is a nice one I just don't see the point of the third-party involved and I somewhat doubt that a website like this can be useful to the end-user or to put pressure on the vendor.
patio11 16y ago
With regards to use of the SSL-is-PGP trick discussed earlier, I think you may run into a problem in the real world of megacorps: the peons in charge of appsec may be several fiefdoms away from the peons who could actually access the private key for the SSL certificate. Suppose you report a SQL injection in X university's website. I would have been responsible for getting that fixed at my old job. I know what campus the guy with root on our web tier works at, but working on memory of our org chart, I can only narrow it down to a thousand people, literally. Extraneous copies of that private key aren't floating around internally. Indeed, I rather suspect that if there exists a process by which I can learn it, we just failed ISOSomething. How am I supposed to figure out what your cypher text telling me what is wrong actually says?
- tptacek 16y ago
  The person in charge of appsec is guaranteed to be many fiefdoms away from the sysadmins. In a lot of very large famous companies, you'd be wise to put money on the notion that the appsec guy hates the guys that manage the websites.
  - omh 16y ago
    And vice versa! As a sysadmin I wouldn't want to give the private key to someone with the knowledge of how to use it for interception of the traffic it's protecting ( and possibly the opportunity!)
- there 16y ago
  and the people in charge of the web servers may not be authorized to be handling the security-related material they will be decrypting with the key and forwarding off.
  - tptacek 16y ago
    Extremely good point.
- px403 16y ago
  security@megacorp : Hey, we just had a bug report, could I get a copy of the private key for our root domain so I can decrypt it? webit@megacorp : lawl
iuguy 16y ago
Zed Shaw is a very clever man. However, I've read the site time and time again and I'm not sure what problem this is supposed to solve. There are already services that pay you to disclose your vulnerabilities to them and project manage the whole thing with the vendor for you, such as Tipping Point, The Zero Day Initiative and iDefense. The whole SSL thing is a clever little hack, but is more of a solution in search of a problem. For a security researcher, secure comms with a vendor is hardly a problem that hasn't already been solved (and if the vendor wants to communicate over unencrypted SMTP and they're aware of the risks that come with it, then it's their own fault if somehow the comms are compromised). I'm sure it'll be another very smart and funky thing from Zed, but I don't see why people would use it over something that pays them for bugs.
- zedshaw 16y ago
  That's a really good point. I'd be competing with companies that pay people to give them defects, which is really hard to go against. I sure don't have the money to do it, so maybe this would just be for open source projects at first.
  - iuguy 16y ago
    Interesting. What if you flip it around? Instead of making it something for researchers to use and competing with tipping point et al, how about making it something for open source projects to use to manage the process of handling security bugs? I'm not aware of anything in that space, but there may be some mileage in exploring it. I could see this working in a SaaS model a la github - free if you keep everything open. Paid for if you don't.
    - zedshaw 16y ago
      That's possible, and also for the SMB market that's producing lots of software but might be just a few devs. Think: iphone, ipad apps, small sites. There's not much for them either.
      - 16y ago
zwp 16y ago
Zed will act as a clearing house? There's no danger scale (CVSS or whatever) for customers and no comp for, ahh, researchers. I don't understand how this will work. Also clearly demonstrated that it's easy to submit reports. So... 1. Spam vulnarb.com with vulns in my competitor's software. 2. Spam vulnarb.com with vulns in my own company (maybe I can spin this... "1239801211 vuln reports against us... vulnarb clearly not to be trusted... our robust and mature security team always available...". Any press is good press right?) 3. Spam anybody I like in there, bury my product's issues. How exactly does Joe Customer realistically compare software using this? ("I want to view PDFs, should I use Acrobat, Foxit, evince, xpdf?"). 3b. Spam vulnarb.com with faux vulnarb.com vulns :-p
- zedshaw 16y ago
  So, block spam and identify people to avoid is what you're saying? Done. Now what?
  - zwp 16y ago
    Faux vulnerability reports might be slightly harder to spot than your average Nigerian Princess. Will you validate my exploit for IBM RACF on OS/390?
    - zedshaw 16y ago
      It's easy. Vendor decrypts it. It gets ignored. Done.
      - zwp 16y ago
        The process isn't clear to me. So the vendor communicates "fixed" to you? Or do I report to the vendor and to you simultaneously using this mechanism, and then I subsequently follow up with you? Does the vendor communicate with me? You? Both of us? How? What happens in disputed cases? (eg I have reported unsafe calls in signal handlers to be told "no working exploit, no fix"). Now what? I still have to publicly disclose vulns in my own infrastructure to get traction? What have I gained by going to vulnarb? Even assuming consumers consult such a service what can they learn if the details (severity) of the issues is not available? I don't think I can sell this to my CSO ("there's a guy on the 'net that says I should send him encrypted text of exploits that I've found in our infrastructure"). And for stuff that doesn't touch my day job: what's your sales pitch over ZDI? I thought the shell script was neat but sorry, I'm not really compelled by your service proposition. EDIT: your _current_ service proposition ;-)
wewyor 16y ago
Well that gives a reasonable use case for this: http://dpaste.de/61O8/ http://dpaste.de/61O8/ (http://news.ycombinator.com/item?id=2402136 http://news.ycombinator.com/item?id=2402136) Makes more sense now.
the-kenny 16y ago
Why not use GnuPG?
- tptacek 16y ago
  Using GPG is a much better idea and is in fact what most vendors, including the Microsoft MSRC, the Adobe and Cisco PSIRTs, and the Apple product security team already do.
- bhickey 16y ago
  I suspect for purposes of key distribution. If <huge company> doesn't feel like playing Zed's game, they could easily forgo issuing a GPG key. By using SSL, he's strong arming them into participating.
  - zedshaw 16y ago
    Yep, exactly. People reporting vulnerability shouldn't have to beg to do it, they should be able to do it and it's up to the corporations to provide more convenient means. That and I thought it was a pretty cool hack to encrypt a payload using just a shell script and openssl. :-)
    - Create 16y ago
      If a vendor bothers to fix a security patch, the ad/disclosure is already signed, most of the time. Harvesting the keys from i.e. the bugtraq archive seems convenient enough. But yes, I also like the SSL idea, now if only verisign and comodo would be more trustworthy than the WOT...
    - bhickey 16y ago
      Zed, could you make your e-mail address a bit easier to find? I don't use twitter and figured you wouldn't want me phoning you up.
  - iuguy 16y ago
    How is he strong arming them by sending an encrypted blob around that only the vendor can see? The vendor can still choose to ignore the issue or sue the researcher.
zacharypinter 16y ago
I wonder how easy it is inside a corporation to get access to the SSL private key? I'm guessing a company like Microsoft doesn't hand that out to their developers easily.
- jerf 16y ago
  You would essentially proxy the request over to ops for decryption. That's how we handle GPG-encrypted vulnerability reports in the company I work for. We don't have out the private key to everybody, I think only two people have it, and they forward back the decrypted result internally to a security mailing list. The first couple of times you tried I would expect a general "WTF?" response from ops (indeed they should ask some pointed questions and not just immediately comply), but once you got over that it wouldn't be hard to put a process in place to deal with this routinely.
  - tptacek 16y ago
    Or, instead of building a process for getting your ops team to do unusual things with your site certificate, you could just put up the web page with your GPG key on it.
    - zedshaw 16y ago
      Yes, ideally GPG would be better, but if you only did GPG then companies could simply not publish a key and avoid the whole system. I'm envisioning some combination whereby, if they are already registered on vulnarb.com then you just get their GPG key (hopefully signed by their SSL key), otherwise you just use the SSL key if that's all you have. After that I haven't explored it further. The advantage of an SSL key is that they usually must have one, it's secure, and they can't prevent people from using it.
      - tptacek 16y ago
        Vendors who don't provide some secure channel to send findings to are already subject to vigilante-style disincentives: if you don't provide a key, researchers will drop zero day on you publicly. Remember, public drops used to be the primary way things got disclosed. You appear to think we need to worry about vendors gaming the system. But on the researcher side, the standard definition of "responsible disclosure" is, "I wait 60 days after I send you something and then I go public whether you say OK or not". That isn't just my take on the situation; it's what actually happens.
itgoon 16y ago
Kudos to Zed, for at least attempting to find a workable middle-ground. It's a change from the usual and useless finger-pointing.
- tptacek 16y ago
  The fingerpointing isn't for want of a better system of disclosure. It's because fixing bugs costs money. Very few vulnerability researchers have any notion of how much it costs to roll out a dot release; the idea that it could cost tens of thousands of dollars strikes them as nonsensical. Meanwhile, very few software vendors have any notion of how big a deal (say) memory corruption is; the idea that it cost the public hundreds of thousands of dollars strikes them as nonsensical. All the tools in the world aren't going to get researchers to care about why it's taking 6 months for a vendor to publish their findings. To them, 60 days is an extremely generous window. And all the tools in the world aren't going to educate vendors on the true costs of software vulnerabilities.
  - zedshaw 16y ago
    I mostly disagree with this. Right now the corporations have pretty much stacked the deck against consumers when it comes to security. A simple site where you can go and at least see the amount of vulnerability in a product, and also any past ones could do a lot to change how things are handled currently.
    - m0nastic 16y ago
      How would this differ from a site like the Zero Day Initiative? (aside from that it looks like there's much less hassle to use this system; or maybe that's actually the point). I think this is interesting, but the two "types of vendors" I see this being useful for are: 1.) Vendors who don't have an already established method for communicating vulnerabilities (and therefore the researcher doesn't have a good way to interact with them). and 2.) Vendors who chose not to work with researchers (of which there used to be a lot, and are still a few). I see type 1 quickly instating a policy to deal with vulnerabilities after something of there's get posted to the site (so they can more quickly deal with future issues), and I see type 2 continuing to ignore vulnerabilities (and possibly being hostile to the researcher and the site for posting things). So maybe the benefit is shaming "type 2" vendors?
trotsky 16y ago
Doesn't this mean that anyone who administers (or breaks in to) a webserver or ssl reverse proxy belonging to the vulnerable party would be able to decrypt the report+POC? Isn't that a pretty serious issue given that such reports can be extremely sensitive and at least in most mid to large size vendors like cisco, microsoft, google etc. they would normally only be available to the security teams + the responsible engineering team? Certainly not to ops.
- bonzoesc 16y ago
  Yes, but it's better than the most common scenario now, which is you generally don't have a public key for the security team at all. Besides, if you can't trust your SSL administrators, why do you have them administrating your SSL?
  - trotsky 16y ago
    It seems to me that trust is a more subtle metric than on and off. If nothing else, if you're trying to keep a secret it behooves you to tell as few people as possible, even if you trust them all. There are lots and lots of security team pgp keys out there. Clearly not every vendor has a secure process, but certainly the people who make up the volume of the reported incidents do. Vendors who do have them might be worse off if people started using this scheme. cisco: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xA29130D0 http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xA291... google: https://services.google.com/corporate/publickey.txt https://services.google.com/corporate/publickey.txt microsoft: http://www.microsoft.com/technet/security/bulletin/pgp.mspx http://www.microsoft.com/technet/security/bulletin/pgp.mspx
    - tptacek 16y ago
      I am trying to imagine the response you'd get from, say, a message oriented middleware vendor without a dedicated security contact, upon sending them a message encrypted in such a way that only their SSL certificate could be used to decrypt it. It's funny to think about it as "mass hysteria!"; the people who care about these reports are on the dev team and can safely be expected not to really understand what an SSL cert is, let alone have access to it or the patience to use OpenSSL's command line. But more realistically what's going to happen is that the message will get chucked entirely.
deleted 16y ago
[deleted]
tptacek 16y ago
This is a much more complicated answer than the industry standard, which is to post a SHA-1 hash of a summary of your exploit, along with (sometimes) the vendor name. I'm not sure how important the extra problems that this solves are. Yes, it allows a vendor to "prove" they fixed a vulnerability. Perhaps if you're looking at vulnerability research as an outsider you think this is a significant issue. But it really isn't. If Chris Valasek or Tavis Ormandy say that (say) Adobe hasn't fixed their finding, everybody (including Adobe) will believe them. On the flip side, when Adobe fixes Valasek or Ormandy's bug, they're going to say that right away. Believe it or not, vulnerability researchers like it when vendors fix their bugs; it's part of how you keep score. And even if you think proving vulnerabilities is a real issue, all this does is allow vendors to decide autonomously to prove something. But vendors don't need tools to do this, and neither do researchers. The real problem is that vendors sit on vulnerabilities for months or years; giving them one more (complicated) way to publish doesn't really help much. Meanwhile, the researcher can disclose any time she wants; she justs posts the exploit to Pastie or F-D. Done and done. And yes, this creates a central location for customers to see outstanding security issues with products. But Zed can do that right now without getting researchers to do anything differently. He can just follow security researchers on Twitter and RSS their blogs and wait for them to post things. Then he can create entries on his site. Charlie Miller even posts numbers, like, "I have 193882 binned crashes in Quicktime and Apple has fixed none of them so far". Some visualization tools might come in handy. It's less fun than crypto schemes, but probably more useful. Also worth mentioning: there already are "vulnerability arbitration centers" that do exactly this. Also, they take over the project management with the vendor. Also, they publish formal advisories to alert the public. Also, they pay the researchers. Sometimes a lot. One of them is TippingPoint's Zero Day Initiative, which also runs the annual Pwn2Own contest at CanSecWest. Another is iDefense. Finally, you can consider whether this addresses the problem of "how do I communicate a vulnerability securely to a vendor". That is indeed a real problem and this is indeed a viable answer to that problem. But every vendor that has a vulnerability response process already has a secure channel for receiving reports. From spending the last 5+ years of my life communicating findings to vendors who don't have that process in place, let me assure you that "secure channel" is the least of your problems. Forget getting your email intercepted; worry more that your finding is going to end up in the vendor's public bug database tagged as a "feature" (seriously, go pick a slightly non-mainstream vendor with a public database and do searches for "segmentation fault" or even "nessus", as in "bug: product crashes and reboots when nessus is run against it"). --- If, as a vendor, you want to do something to streamline vulnerability reports, run - do - not - walk to create this web page on your site: http://news.ycombinator.com/item?id=804257 http://news.ycombinator.com/item?id=804257 I highly recommend you just crib from what 37signals did. If you don't, and rely instead on the notion that a researcher could in theory use the RSA key in your SSL cert to send you a secure message, please bear in mind that many --- perhaps most --- researchers will interpret your lack of guidance on this as a license to simply publish your flaws directly on a mailing list. You didn't, after all, tell them not to, or tell them where they should instead send their findings.
HerraBRE 16y ago
I think it could be useful to encrypt the same report to multiple recipients - just generate multiple pass.enc files. This allows the researcher to publicly send the same report to multiple vendors (and it will verifiably be the same report) and adds the option of introducing "neutral" third party arbitrators as well. For example, the researcher might want to publicly CC all vulnerabilities to CERT (to pick a random example), to put a little more pressure on the vendor and share the burden of following up and keeping the vendor honest.
- zedshaw 16y ago
  That shell script does that already. Go ahead and run it for multiple sites with the same file: sh encrypt.sh www.google.com mysecret.txt sh encrypt.sh www.apple.com mysecret.txt sh encrypt.sh www.microsoft.com mysecret.txt It then will make a directory for each in results using different passwords and their respective keys.
  - HerraBRE 16y ago
    That is exactly what I don't want. :-) I want the same random key sent to multiple recipients. That way they can all verify that they are seeing the same thing. Edit: Just to clarify, without using the same random key for all recipients, the reports are not verifiably the same until everyone shows their random keys. If you use the same random key for all recipients, you don't have this problem.
    - zedshaw 16y ago
      Could be an option, although I'm not sure what's better given that most vendors wouldn't want to have other vendors know they're vulnerable until they fix it. Also, I'm pretty sure that if adobe and microsoft both have a vulnerability then they'll talk to each other.
      - HerraBRE 16y ago
        I'm mostly thinking about the arbitrator scenario, or key escrow for each report. Your current scheme suffers from the fact that once the report has been generated, the only person that can verify that a given document is the same as the submitted report is the vendor - who may not be inclined to cooperate. It's admittedly a little far-fetched, but if there is a dispute as to the contents of the report and the vendor refuses to disclose the secret key, it ends up being the vendor's word against that of the researcher. This can be solved by keeping the random key around or by sending it to multiple recipients. Or both. Your current strategy of immediately deleting it is pointless while the original plain-text report still exists but leads to the conundrum above.
rmoriz 16y ago
I wonder if we'll use this in a couple of years with DNSSEC keys instead of certificates used for https.
ChuckMcM 16y ago
I think its an excellent idea, because if the company's secret keys are compromised already then that is a meta-vulnerability that needs immediate attention :-). And only the company should be able to get access to their secret keys. I would include in the encrypted message the length of the encrypted message. While its unlikely that an attacker would take a previously encrypted message and try to add confounding/confusing text to it you would want to avoid having the message compromised between sender and receiver. There is no way (as yet) in your stream to know if the company has picked up a copy of the vulnerability. But that may remain true as there is a question as to negligence if you know about a vuln and don't fix it, are you negligent for consequential damages it causes? There is also no way to know if two or more reports are the same vulnerability or not. It would be useful if there was a way to somehow indicate this but that too is a hard problem. The effect might be that when a vulnerability is discovered the manufacturer starts seeing a bunch of reports but they are all the same one, and then after the patch the reports won't go down until the patched version is distributed widely.
dgl 16y ago
Seems like a neat hack to find a certificate for an entity that doesn't necessarily publish a PGP key. However unless the company's primary website is on SSL how is it going to work? Choosing a random SSL secured site under a domain might lead to something outsourced which the original company would find it hard to obtain the private key for. (Even the primary website could be on Akamai and then the company may not have access to the private key.) If you start requiring a company to provide a specific domain for the key you might as well ask for a PGP key instead and store the PGP encrypted messages.
- zedshaw 16y ago
  Can you find instances of this practice? I'd like to know so I can investigate.
  - dgl 16y ago
    For the Akamai thing, compare what certs you get for say www.oracle.com and www.akamai.com (I see the same cert). The CN doesn't match there so that case is a bit different to a third party hosting a valid cert. You probably would want to check for things like this somehow though.
spc476 16y ago
It sounds like to me there's an excellent opportunity here for some clever security researchers to set up a company like Underwriters Laboratories (http://en.wikipedia.org/wiki/Underwriters_Laboratories http://en.wikipedia.org/wiki/Underwriters_Laboratories).
tlrobinson 16y ago
Using the public SSL cert is an interesting idea, but I foresee problems. Dev: "Hey some dude on the Internet posted a vulnerability for our product but it's encrypted with our webserver's public SSL key. Can I have our private key to decrypt it?" IT: "Fuck off." Perhaps provide a secure way for the security team of large companies to submit another public key they control?
blantonl 16y ago
This is a fantastic idea, however, how will it be sustained if the idea takes off? Non-profit? Consortium funding (scary), other? Has zed taking this into account and documented?

238 points by acangiano 16y ago

lordmatty 16y ago

Cool idea - hope he has some good legal cover :)

tptacek 16y ago
Huh? Why?

snagg 16y ago

I think this approach exposes a number of problems that are pretty relevant. First off vendors by decrypting the content of the researcher's submission are basically giving up vulnerabilities to the bad guys that want to target unpatched systems (read: what malware most of the time does). Second thing I see no reasons why the vulnerability disclosure process should change and migrate to SSL based encryption: vendors can already expose "lies" if they feel like it with PGP encryption and honestly having a "middle-man" in between that can potentially have sensible data looks like a bad idea. So how about: researchers post on the website the SHA-1 of the POC or just a line saying "product X is vulnerable", then the owner of the website asks the vendor for confirmation. If the researcher has no "karma points" then the submission is hold back until the vendor confirms, if the researcher has "karma points" (already multiple confirmed and valuable submissions) then the advisory gets published immediately regardless of whether the vendor acknowledges it or not. Still does it actually help anybody? how would you convince consumers to actually pick products or similar based on that website? To me it looks like the people interested in this kind of information have already other means (twitter, ml, direct contact with vendors), the others don't care.

zedshaw 16y ago
> First off vendors by decrypting the content of the researcher's submission are basically giving up vulnerabilities to the bad guys that want to target unpatched systems They already do this. After a vulnerability is patched and fixed they release what it did. > Second thing I see no reasons why the vulnerability disclosure process should change and migrate to SSL based encryption It's a proposed first step, and makes sure that no vendor can avoid the system by simply not publishing a key. I envision in the future their GPG key would also be on the site and could be used. > vendors can already expose "lies" if they feel like it with PGP encryption and honestly having a "middle-man" in between that can potentially have sensible data looks like a bad idea. Uh, not sure what the "middle-man" is, but if you mean vulnarb.com then no, the point is that it's industry standard asymmetric crypto so I wouldn't know anything. In fact, I'd have incentive to not know anything so that I'm not getting sued. Finally, your proposed karma points system doesn't seem to solve anything. If everything is encrypted so that the receiver is only able to decrypt (not even the sender can do that), then there's no point in preventing people from publishing. Are you afraid that someone would slander a company? Couldn't the company simply decrypt and publish their lies then sue them for slander?
- snagg 16y ago
  > They already do this. After a vulnerability is patched and fixed they release what it did. Usually researchers' submissions are way more detailed than the advisories you see from vendors. Meaning that you cannot just decrypt the content of what the research submitted. It's true that oftentimes you can find the bug by reading the advisory and using tools to diff the patch but it's a long shot compared to just publishing the original researcher's submission imo. >Uh, not sure what the "middle-man" is, but if you mean vulnarb.com then no, the point is that it's industry standard asymmetric crypto so I wouldn't know anything. In fact, I'd have incentive to not know anything so that I'm not getting sued. yep I meant vulnarb.com. I'm not saying that you have anything decrypted and ready to use, I'm saying that it's pointless to have another recipient for sensible data. Cause in the extreme scenario where the private key is stolen than it's just more attack surface to get to the submissions. If we assume that no leak of this sort happens, still what's the reason for a third party to have this sort of data compared to say sha-1 hash of the Poc? Actually my point about the karma system is that you can achieve the same goal you have in mind without needing any sort of data from the researcher other than "product X is vulnerable" and then the karma points will determine whether the researcher is reliable or not.
  - zedshaw 16y ago
    > Actually my point about the karma system is that you can achieve the same goal you have in mind without needing any sort of data from the researcher other than "product X is vulnerable" and then the karma points will determine whether the researcher is reliable or not. No, that doesn't actually solve this problem because then it just turns into a he-said-she-said. Also, you seem to leave out that the company can decide to decrypt it or not. Hell, they might come to an agreement with the researcher to just have the researcher assert it's fixed and never decrypt it. Although, I think that it should be fully exposed so people can see how it's really done and there's incentive to really fix it, not half-ass fix it.
    - snagg 16y ago
      >No, that doesn't actually solve this problem because then it just turns into a he-said-she-said. then I fail to understand what you plan to do with the encrypted stuff that you get from the researcher since the only one able to decrypt it would be the vendor. At that point the scenario is: researcher says this is vulnerable, vendor denies it, somebody needs to decrypt the content of what the researcher has sent to the vendor. In contrast what happens without having the encrypted content is: researcher says this is vulnerable, vendor denies it, the researcher can publish the un-encrypted original advisory on your website if he feels like it. Regardless the SSL trick is a nice one I just don't see the point of the third-party involved and I somewhat doubt that a website like this can be useful to the end-user or to put pressure on the vendor.

patio11 16y ago

With regards to use of the SSL-is-PGP trick discussed earlier, I think you may run into a problem in the real world of megacorps: the peons in charge of appsec may be several fiefdoms away from the peons who could actually access the private key for the SSL certificate. Suppose you report a SQL injection in X university's website. I would have been responsible for getting that fixed at my old job. I know what campus the guy with root on our web tier works at, but working on memory of our org chart, I can only narrow it down to a thousand people, literally. Extraneous copies of that private key aren't floating around internally. Indeed, I rather suspect that if there exists a process by which I can learn it, we just failed ISOSomething. How am I supposed to figure out what your cypher text telling me what is wrong actually says?

tptacek 16y ago
The person in charge of appsec is guaranteed to be many fiefdoms away from the sysadmins. In a lot of very large famous companies, you'd be wise to put money on the notion that the appsec guy hates the guys that manage the websites.
- omh 16y ago
  And vice versa! As a sysadmin I wouldn't want to give the private key to someone with the knowledge of how to use it for interception of the traffic it's protecting ( and possibly the opportunity!)
there 16y ago
and the people in charge of the web servers may not be authorized to be handling the security-related material they will be decrypting with the key and forwarding off.
- tptacek 16y ago
  Extremely good point.
px403 16y ago
security@megacorp : Hey, we just had a bug report, could I get a copy of the private key for our root domain so I can decrypt it? webit@megacorp : lawl

iuguy 16y ago

Zed Shaw is a very clever man. However, I've read the site time and time again and I'm not sure what problem this is supposed to solve. There are already services that pay you to disclose your vulnerabilities to them and project manage the whole thing with the vendor for you, such as Tipping Point, The Zero Day Initiative and iDefense. The whole SSL thing is a clever little hack, but is more of a solution in search of a problem. For a security researcher, secure comms with a vendor is hardly a problem that hasn't already been solved (and if the vendor wants to communicate over unencrypted SMTP and they're aware of the risks that come with it, then it's their own fault if somehow the comms are compromised). I'm sure it'll be another very smart and funky thing from Zed, but I don't see why people would use it over something that pays them for bugs.

zedshaw 16y ago
That's a really good point. I'd be competing with companies that pay people to give them defects, which is really hard to go against. I sure don't have the money to do it, so maybe this would just be for open source projects at first.
- iuguy 16y ago
  Interesting. What if you flip it around? Instead of making it something for researchers to use and competing with tipping point et al, how about making it something for open source projects to use to manage the process of handling security bugs? I'm not aware of anything in that space, but there may be some mileage in exploring it. I could see this working in a SaaS model a la github - free if you keep everything open. Paid for if you don't.
  - zedshaw 16y ago
    That's possible, and also for the SMB market that's producing lots of software but might be just a few devs. Think: iphone, ipad apps, small sites. There's not much for them either.
    - 16y ago

zwp 16y ago

Zed will act as a clearing house? There's no danger scale (CVSS or whatever) for customers and no comp for, ahh, researchers. I don't understand how this will work. Also clearly demonstrated that it's easy to submit reports. So... 1. Spam vulnarb.com with vulns in my competitor's software. 2. Spam vulnarb.com with vulns in my own company (maybe I can spin this... "1239801211 vuln reports against us... vulnarb clearly not to be trusted... our robust and mature security team always available...". Any press is good press right?) 3. Spam anybody I like in there, bury my product's issues. How exactly does Joe Customer realistically compare software using this? ("I want to view PDFs, should I use Acrobat, Foxit, evince, xpdf?"). 3b. Spam vulnarb.com with faux vulnarb.com vulns :-p

zedshaw 16y ago
So, block spam and identify people to avoid is what you're saying? Done. Now what?
- zwp 16y ago
  Faux vulnerability reports might be slightly harder to spot than your average Nigerian Princess. Will you validate my exploit for IBM RACF on OS/390?
  - zedshaw 16y ago
    It's easy. Vendor decrypts it. It gets ignored. Done.
    - zwp 16y ago
      The process isn't clear to me. So the vendor communicates "fixed" to you? Or do I report to the vendor and to you simultaneously using this mechanism, and then I subsequently follow up with you? Does the vendor communicate with me? You? Both of us? How? What happens in disputed cases? (eg I have reported unsafe calls in signal handlers to be told "no working exploit, no fix"). Now what? I still have to publicly disclose vulns in my own infrastructure to get traction? What have I gained by going to vulnarb? Even assuming consumers consult such a service what can they learn if the details (severity) of the issues is not available? I don't think I can sell this to my CSO ("there's a guy on the 'net that says I should send him encrypted text of exploits that I've found in our infrastructure"). And for stuff that doesn't touch my day job: what's your sales pitch over ZDI? I thought the shell script was neat but sorry, I'm not really compelled by your service proposition. EDIT: your _current_ service proposition ;-)

wewyor 16y ago

Well that gives a reasonable use case for this: http://dpaste.de/61O8/ http://dpaste.de/61O8/ (http://news.ycombinator.com/item?id=2402136 http://news.ycombinator.com/item?id=2402136) Makes more sense now.

the-kenny 16y ago

Why not use GnuPG?

tptacek 16y ago
Using GPG is a much better idea and is in fact what most vendors, including the Microsoft MSRC, the Adobe and Cisco PSIRTs, and the Apple product security team already do.
bhickey 16y ago
I suspect for purposes of key distribution. If <huge company> doesn't feel like playing Zed's game, they could easily forgo issuing a GPG key. By using SSL, he's strong arming them into participating.
- zedshaw 16y ago
  Yep, exactly. People reporting vulnerability shouldn't have to beg to do it, they should be able to do it and it's up to the corporations to provide more convenient means. That and I thought it was a pretty cool hack to encrypt a payload using just a shell script and openssl. :-)
  - Create 16y ago
    If a vendor bothers to fix a security patch, the ad/disclosure is already signed, most of the time. Harvesting the keys from i.e. the bugtraq archive seems convenient enough. But yes, I also like the SSL idea, now if only verisign and comodo would be more trustworthy than the WOT...
  - bhickey 16y ago
    Zed, could you make your e-mail address a bit easier to find? I don't use twitter and figured you wouldn't want me phoning you up.
- iuguy 16y ago
  How is he strong arming them by sending an encrypted blob around that only the vendor can see? The vendor can still choose to ignore the issue or sue the researcher.

zacharypinter 16y ago

I wonder how easy it is inside a corporation to get access to the SSL private key? I'm guessing a company like Microsoft doesn't hand that out to their developers easily.

jerf 16y ago
You would essentially proxy the request over to ops for decryption. That's how we handle GPG-encrypted vulnerability reports in the company I work for. We don't have out the private key to everybody, I think only two people have it, and they forward back the decrypted result internally to a security mailing list. The first couple of times you tried I would expect a general "WTF?" response from ops (indeed they should ask some pointed questions and not just immediately comply), but once you got over that it wouldn't be hard to put a process in place to deal with this routinely.
- tptacek 16y ago
  Or, instead of building a process for getting your ops team to do unusual things with your site certificate, you could just put up the web page with your GPG key on it.
  - zedshaw 16y ago
    Yes, ideally GPG would be better, but if you only did GPG then companies could simply not publish a key and avoid the whole system. I'm envisioning some combination whereby, if they are already registered on vulnarb.com then you just get their GPG key (hopefully signed by their SSL key), otherwise you just use the SSL key if that's all you have. After that I haven't explored it further. The advantage of an SSL key is that they usually must have one, it's secure, and they can't prevent people from using it.
    - tptacek 16y ago
      Vendors who don't provide some secure channel to send findings to are already subject to vigilante-style disincentives: if you don't provide a key, researchers will drop zero day on you publicly. Remember, public drops used to be the primary way things got disclosed. You appear to think we need to worry about vendors gaming the system. But on the researcher side, the standard definition of "responsible disclosure" is, "I wait 60 days after I send you something and then I go public whether you say OK or not". That isn't just my take on the situation; it's what actually happens.

itgoon 16y ago

Kudos to Zed, for at least attempting to find a workable middle-ground. It's a change from the usual and useless finger-pointing.

tptacek 16y ago
The fingerpointing isn't for want of a better system of disclosure. It's because fixing bugs costs money. Very few vulnerability researchers have any notion of how much it costs to roll out a dot release; the idea that it could cost tens of thousands of dollars strikes them as nonsensical. Meanwhile, very few software vendors have any notion of how big a deal (say) memory corruption is; the idea that it cost the public hundreds of thousands of dollars strikes them as nonsensical. All the tools in the world aren't going to get researchers to care about why it's taking 6 months for a vendor to publish their findings. To them, 60 days is an extremely generous window. And all the tools in the world aren't going to educate vendors on the true costs of software vulnerabilities.
- zedshaw 16y ago
  I mostly disagree with this. Right now the corporations have pretty much stacked the deck against consumers when it comes to security. A simple site where you can go and at least see the amount of vulnerability in a product, and also any past ones could do a lot to change how things are handled currently.
  - m0nastic 16y ago
    How would this differ from a site like the Zero Day Initiative? (aside from that it looks like there's much less hassle to use this system; or maybe that's actually the point). I think this is interesting, but the two "types of vendors" I see this being useful for are: 1.) Vendors who don't have an already established method for communicating vulnerabilities (and therefore the researcher doesn't have a good way to interact with them). and 2.) Vendors who chose not to work with researchers (of which there used to be a lot, and are still a few). I see type 1 quickly instating a policy to deal with vulnerabilities after something of there's get posted to the site (so they can more quickly deal with future issues), and I see type 2 continuing to ignore vulnerabilities (and possibly being hostile to the researcher and the site for posting things). So maybe the benefit is shaming "type 2" vendors?

trotsky 16y ago

Doesn't this mean that anyone who administers (or breaks in to) a webserver or ssl reverse proxy belonging to the vulnerable party would be able to decrypt the report+POC? Isn't that a pretty serious issue given that such reports can be extremely sensitive and at least in most mid to large size vendors like cisco, microsoft, google etc. they would normally only be available to the security teams + the responsible engineering team? Certainly not to ops.

bonzoesc 16y ago
Yes, but it's better than the most common scenario now, which is you generally don't have a public key for the security team at all. Besides, if you can't trust your SSL administrators, why do you have them administrating your SSL?
- trotsky 16y ago
  It seems to me that trust is a more subtle metric than on and off. If nothing else, if you're trying to keep a secret it behooves you to tell as few people as possible, even if you trust them all. There are lots and lots of security team pgp keys out there. Clearly not every vendor has a secure process, but certainly the people who make up the volume of the reported incidents do. Vendors who do have them might be worse off if people started using this scheme. cisco: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xA29130D0 http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xA291... google: https://services.google.com/corporate/publickey.txt https://services.google.com/corporate/publickey.txt microsoft: http://www.microsoft.com/technet/security/bulletin/pgp.mspx http://www.microsoft.com/technet/security/bulletin/pgp.mspx
  - tptacek 16y ago
    I am trying to imagine the response you'd get from, say, a message oriented middleware vendor without a dedicated security contact, upon sending them a message encrypted in such a way that only their SSL certificate could be used to decrypt it. It's funny to think about it as "mass hysteria!"; the people who care about these reports are on the dev team and can safely be expected not to really understand what an SSL cert is, let alone have access to it or the patience to use OpenSSL's command line. But more realistically what's going to happen is that the message will get chucked entirely.

deleted 16y ago

[deleted]

tptacek 16y ago

This is a much more complicated answer than the industry standard, which is to post a SHA-1 hash of a summary of your exploit, along with (sometimes) the vendor name. I'm not sure how important the extra problems that this solves are. Yes, it allows a vendor to "prove" they fixed a vulnerability. Perhaps if you're looking at vulnerability research as an outsider you think this is a significant issue. But it really isn't. If Chris Valasek or Tavis Ormandy say that (say) Adobe hasn't fixed their finding, everybody (including Adobe) will believe them. On the flip side, when Adobe fixes Valasek or Ormandy's bug, they're going to say that right away. Believe it or not, vulnerability researchers like it when vendors fix their bugs; it's part of how you keep score. And even if you think proving vulnerabilities is a real issue, all this does is allow vendors to decide autonomously to prove something. But vendors don't need tools to do this, and neither do researchers. The real problem is that vendors sit on vulnerabilities for months or years; giving them one more (complicated) way to publish doesn't really help much. Meanwhile, the researcher can disclose any time she wants; she justs posts the exploit to Pastie or F-D. Done and done. And yes, this creates a central location for customers to see outstanding security issues with products. But Zed can do that right now without getting researchers to do anything differently. He can just follow security researchers on Twitter and RSS their blogs and wait for them to post things. Then he can create entries on his site. Charlie Miller even posts numbers, like, "I have 193882 binned crashes in Quicktime and Apple has fixed none of them so far". Some visualization tools might come in handy. It's less fun than crypto schemes, but probably more useful. Also worth mentioning: there already are "vulnerability arbitration centers" that do exactly this. Also, they take over the project management with the vendor. Also, they publish formal advisories to alert the public. Also, they pay the researchers. Sometimes a lot. One of them is TippingPoint's Zero Day Initiative, which also runs the annual Pwn2Own contest at CanSecWest. Another is iDefense. Finally, you can consider whether this addresses the problem of "how do I communicate a vulnerability securely to a vendor". That is indeed a real problem and this is indeed a viable answer to that problem. But every vendor that has a vulnerability response process already has a secure channel for receiving reports. From spending the last 5+ years of my life communicating findings to vendors who don't have that process in place, let me assure you that "secure channel" is the least of your problems. Forget getting your email intercepted; worry more that your finding is going to end up in the vendor's public bug database tagged as a "feature" (seriously, go pick a slightly non-mainstream vendor with a public database and do searches for "segmentation fault" or even "nessus", as in "bug: product crashes and reboots when nessus is run against it"). --- If, as a vendor, you want to do something to streamline vulnerability reports, run - do - not - walk to create this web page on your site: http://news.ycombinator.com/item?id=804257 http://news.ycombinator.com/item?id=804257 I highly recommend you just crib from what 37signals did. If you don't, and rely instead on the notion that a researcher could in theory use the RSA key in your SSL cert to send you a secure message, please bear in mind that many --- perhaps most --- researchers will interpret your lack of guidance on this as a license to simply publish your flaws directly on a mailing list. You didn't, after all, tell them not to, or tell them where they should instead send their findings.

HerraBRE 16y ago

I think it could be useful to encrypt the same report to multiple recipients - just generate multiple pass.enc files. This allows the researcher to publicly send the same report to multiple vendors (and it will verifiably be the same report) and adds the option of introducing "neutral" third party arbitrators as well. For example, the researcher might want to publicly CC all vulnerabilities to CERT (to pick a random example), to put a little more pressure on the vendor and share the burden of following up and keeping the vendor honest.

zedshaw 16y ago
That shell script does that already. Go ahead and run it for multiple sites with the same file: sh encrypt.sh www.google.com mysecret.txt sh encrypt.sh www.apple.com mysecret.txt sh encrypt.sh www.microsoft.com mysecret.txt It then will make a directory for each in results using different passwords and their respective keys.
- HerraBRE 16y ago
  That is exactly what I don't want. :-) I want the same random key sent to multiple recipients. That way they can all verify that they are seeing the same thing. Edit: Just to clarify, without using the same random key for all recipients, the reports are not verifiably the same until everyone shows their random keys. If you use the same random key for all recipients, you don't have this problem.
  - zedshaw 16y ago
    Could be an option, although I'm not sure what's better given that most vendors wouldn't want to have other vendors know they're vulnerable until they fix it. Also, I'm pretty sure that if adobe and microsoft both have a vulnerability then they'll talk to each other.
    - HerraBRE 16y ago
      I'm mostly thinking about the arbitrator scenario, or key escrow for each report. Your current scheme suffers from the fact that once the report has been generated, the only person that can verify that a given document is the same as the submitted report is the vendor - who may not be inclined to cooperate. It's admittedly a little far-fetched, but if there is a dispute as to the contents of the report and the vendor refuses to disclose the secret key, it ends up being the vendor's word against that of the researcher. This can be solved by keeping the random key around or by sending it to multiple recipients. Or both. Your current strategy of immediately deleting it is pointless while the original plain-text report still exists but leads to the conundrum above.

rmoriz 16y ago

I wonder if we'll use this in a couple of years with DNSSEC keys instead of certificates used for https.

ChuckMcM 16y ago

I think its an excellent idea, because if the company's secret keys are compromised already then that is a meta-vulnerability that needs immediate attention :-). And only the company should be able to get access to their secret keys. I would include in the encrypted message the length of the encrypted message. While its unlikely that an attacker would take a previously encrypted message and try to add confounding/confusing text to it you would want to avoid having the message compromised between sender and receiver. There is no way (as yet) in your stream to know if the company has picked up a copy of the vulnerability. But that may remain true as there is a question as to negligence if you know about a vuln and don't fix it, are you negligent for consequential damages it causes? There is also no way to know if two or more reports are the same vulnerability or not. It would be useful if there was a way to somehow indicate this but that too is a hard problem. The effect might be that when a vulnerability is discovered the manufacturer starts seeing a bunch of reports but they are all the same one, and then after the patch the reports won't go down until the patched version is distributed widely.

dgl 16y ago

Seems like a neat hack to find a certificate for an entity that doesn't necessarily publish a PGP key. However unless the company's primary website is on SSL how is it going to work? Choosing a random SSL secured site under a domain might lead to something outsourced which the original company would find it hard to obtain the private key for. (Even the primary website could be on Akamai and then the company may not have access to the private key.) If you start requiring a company to provide a specific domain for the key you might as well ask for a PGP key instead and store the PGP encrypted messages.

zedshaw 16y ago
Can you find instances of this practice? I'd like to know so I can investigate.
- dgl 16y ago
  For the Akamai thing, compare what certs you get for say www.oracle.com and www.akamai.com (I see the same cert). The CN doesn't match there so that case is a bit different to a third party hosting a valid cert. You probably would want to check for things like this somehow though.

spc476 16y ago

It sounds like to me there's an excellent opportunity here for some clever security researchers to set up a company like Underwriters Laboratories (http://en.wikipedia.org/wiki/Underwriters_Laboratories http://en.wikipedia.org/wiki/Underwriters_Laboratories).

tlrobinson 16y ago

Using the public SSL cert is an interesting idea, but I foresee problems. Dev: "Hey some dude on the Internet posted a vulnerability for our product but it's encrypted with our webserver's public SSL key. Can I have our private key to decrypt it?" IT: "Fuck off." Perhaps provide a secure way for the security team of large companies to submit another public key they control?

blantonl 16y ago

This is a fantastic idea, however, how will it be sustained if the idea takes off? Non-profit? Consortium funding (scary), other? Has zed taking this into account and documented?