8 ms·

Bug bounty platforms buy researcher silence, violate labor laws, critics say

119 points by keydutch 7y ago

tomlockwood 7y ago
> "I've seen some quote unquote valid vulnerability reports," Laurens ("lvh") Van Houtven, principal at Latacora, a secops and cryptography expert, tells CSO. "If someone asked me 'should I put this in my appsec report?', I'd say 'you can put it in there, but I will never let you live it down.'" Is there any reason that someone who works for a company that wants to be perpetually hired by startups to do security would poo-poo a bug bounty program? This comment seems pretty motivated. Additionally, a vulnerability is a vulnerability. Plenty of companies get owned due to pretty trivial vulnerabilities. > Moussouris, now founder and CEO of bug bounty consultancy Luta Security, questions how much of HackerOne is real. Again, could be motivated. > The bug bounty platforms' NDAs prohibit even mentioning the existence of a private bug bounty. Tweeting something like "Company X has a private bounty program over at Bugcrowd" would be enough to get a hacker kicked off their platform. Sure. How many traditional pentesters are signing NDAs? > Consider a finder who spends weeks or months of unpaid work to discover and document a security flaw. Someone else independently discovers, documents and submits that same bug five minutes before the first finder. Under the rules of most HackerOne and Bugcrowd bounty programs, the first submitter gets all the money, the second finder gets nothing. How are we proposing this alternately works in a way that isn't gameable by having control of multiple accounts? > The lack of vetting of bug bounty hunters, where anyone, including this reporter, can sign up for a HackerOne or Bugcrowd account with any email address, is the key sticking point, Antokol says. > Bugcrowd declined to elaborate on what the process of pre-vetting researchers actually looks like. So, this reporter signed up for an account and had immediate access to all public programs without any additional steps. What is really the point of this so-called journalism? In the preceding paragraphs we were complaining about NDAs stopping people talking about private programs, then complaining about the lack of vetting, then saying we gain access to the PUBLIC programs when we sign up, then saying the bug bounty companies refuse to talk about their vetting methods??? What is this? I can see why Bugcrowd and Project Zero refused to talk to this person. > Mature organizations can and should run their own VDP in house. If they are ready for an avalanche of dubious bug reports, they might optionally choose to run a bug bounty. I have a feeling that this journalist is pretty in bed with the traditional pen-testing industry. His history of articles is pretty focused. Bug bounty programs use a customer's real money to pay out bugs. Bugcrowd, at least, triages those bugs before any payment is made. It is in the interest of nobody involved to submit "dubious" bug bounty reports. The only way I can explain that opinion is to think it comes from the mouth of someone with a grudge.
- tptacek 7y ago
  Latacora is paid in part to manage bug bounty programs, so the motivation is not in the direction you suggest it is.
  - gyanchawdhary 7y ago
    In part.
    - lvh 7y ago
      We bootstrap security teams in a long-term engagement. We staff bug bounty programs, but do not bill for that service separately. If a client uses H1 and pays them for triage, it fairly straightforwardly removes a cost for us. The bug bounty nonsense spray I'm talking about is "sev crit: DMARC is set to quarantine instead of reject" and literal descriptions of how session cookies work (that inexplicably get paid out for: we've seen it happen). I assume we're in agreement I should not be fine with people putting that in pentest reports; if that's the case, the quote is not really controversial: all it says is that H1 has a lot of nonsense reports. The mechanism GP describes would require Latacora to be a proxy good for H1. Do you believe that is the case?
  - tomlockwood 7y ago
    I'd find it hard to believe a company that was positioning itself as experts in security wouldn't be motivated to talk down the output of other security vendors.
    - tptacek 7y ago
      This is a superficial argument of a form the guidelines for the site ask you not to make. (I'm affiliated with Latacora, for what it's worth; you're just dead wrong about the commercial motivations here).
      - tomlockwood 7y ago
        In good faith I'm addressing your comment about a security vendor's motivations. I don't really mind who it is or what industry it is. I expect a database company to talk down the capabilities of other database companies, even if they sometimes use or provide that company's products.
tptacek 7y ago
I think bounty programs are mostly a bad idea for startups and medium-sized tech companies. That said, the critiques in this piece do not make a lot of sense to me. Take transparency. The claim this article makes is that commercial bounty programs work against transparency by paying researchers only when they agree to NDAs. But transparency isn't a norm in software security to begin with; most vulnerability researchers work for labs and consultancies that rarely if ever disclose vulnerabilities. Disclosure of findings is not a norm on commercial software security assessments. Meanwhile, for any target an independent researcher can lawfully assess, the researcher retains the ability to ignore the bounty program and publish straight to Twitter. My sense of it is that HackerOne has probably increased transparency, in the sense that I've read a lot of published reports on H1 that I don't expect I would have seen if the platform didn't exist, and, in my commercial work, haven't seen a lot of private reports that cut the other way. Or this "Safe Harbor" argument, that commercial bounties force researchers to sign NDAs to be immunized from CFAA suits and prosecutions. Sure, but in the absence of H1, most of those CFAA immunizations weren't available on any terms. H1 doesn't enforce CFAA liability; CFAA liability is a natural default under US law. If anything, H1 is mitigating CFAA concerns, not amplifying them. I don't know what to say about the labor law concerns here. I know that the people offering legal opinions here are lawyers and are versed in California labor law. But knowing what I know about how bug bounty people work: there is no way the median bug bounty "participant" would qualify for minimum wage and benefits at the various companies they interact with. The modal bounty participant has a grab bag of a couple dozen scripts they spam against hundreds of different companies with bounties. Is the claim here that H1 owes them a wage? Is the argument here simply that H1 needs to move out of California? The minimum wage thing doesn't ring true either, since it implies that all project-based consulting --- that is, non-T&M consulting where clients pay an agreed-upon rate for an outcome regardless of the time a project takes to complete --- is susceptible to the "ABC" test as well. But project contracts are very common in California. I'm sure there's a subtlety I'm missing. The "ISO compliance" thing is just silly. By the logic in this article, practically none of the thousands of commercial application pentests performed in 2020 to date will be "ISO compliant". Ultimately: I think if you have to ask, you shouldn't run a bounty program. But that's mostly because I think bounty programs don't work very well, and generate an avalanche of noise. I don't think many of the reasons in this article matter.
gyanchawdhary 7y ago
This article is a) very long and written mostly as a whitepaper/opinion piece and less of a journalistic report b) it's very likely a paid hit piece against bug bounty companies. However, I'm not surprised as the timing makes sense. The COVID-19 situ has impacted a lot of pen-testing firms [canceled testing contracts/security staff augmentation, longterm security audits] and it's expensive AF for these companies to keep their consultants on bench. Bug bounties are by no means perfect, but whatever this article is trying to point out isn't going to stop CISO's from doubling down on bug bounties. In fact, CISO's who've been skeptic about BBs will be now be pressed to explore this option even more. About Me: Sold a few security startups, before that worked as the head of app sec testing for a bailed out british investment bank.
- tptacek 7y ago
  Who would pay to put a hit piece on bug bounty providers? What pentesting firm do you think has their shit together to the extent that they run a media relations team? Who among them don't have revenue streams from triage consulting? It's a bad article. CSO publishes a lot of bad articles. But there's no way this is a paid piece. The people quoted in this article believe what they're saying.
  - gyanchawdhary 7y ago
    > has their shit together to the extent that they run a media relations team. Any company that drops cool research (ATM / Automotive hacking / SCADA etc) use PR firms to drum up their work (which is fair IMHO for SEO/showcasing your new capabilities and service offerings). I did it and I know loads of boutique firms in the UK who do it. I also know of education companies working closely with PR firms to publish articles about "cybersecurity skills gap" or "cybersecurity shortage" to manufacture and invent demand and they are pretty effective too. So it's not that outlandish to think that security companies aren't tactical enough to have PR companies write such articles either. My suspicion about this specific article was given its timing. > Who among them don't have revenue streams from triage consulting? Trigale Consulting is good money but it's mostly getting crumbs from the master's table. BB's are at the apex of this pyramid and will continue to do so. Sharp boutique firms that specialize in stuff will obviously always thrive but that's a diff topic >It's a bad article. CSO publishes a lot of bad articles. Agree > But there's no way this is a paid piece. Maybe but I'm still doubtful. > The people quoted in this article believe what they're saying. Fair point.
anonytrary 7y ago
Bug bounty programs don't make much sense to me. Why would you find bugs and hire external people to fix them when you can hire external people to find bugs and pay trusted employees to fix them? Just hire pentesters.
- mic47 7y ago
  Bug bounty programs are not supposed to replace you other security activities, but it's a way for you to have additional source of vulnerabilities. Advantages of these programs is that security researcher will get rewarded when they find a bug, and that there is clear process for disclosing bugs. You still should hire pentesters, you still should have trusted employees to find bugs and fix them, and more... If you are relying just on bug bounties, your security will suck. That being said, NDA's sound sketchy, if you disclose bug, than after it is fixed, you should be able to blog about it (or when they do not fix it for looong time).
  - PappaPatat 7y ago
    > Bug bounty programs are not supposed to replace you other security activities, but it's a way for you to have additional source of vulnerabilities. Exactly the way we position our own Bug Bounty Program. Where the pentesters can be hired to also confirm things done well, the hunters are only paid for failures they found. In our case there is an added bonus with the Bug Bounty Program: we've come to REALLY apriciate the technical level of reports. Since they only get paid for triagable findings, the details we get reported are so much better then what we used to get from our pentesters. Of course we now require the same quality of reporting from them. What also helps is that the pentesters are motivated more to deliver higher quality findings since they are aware the service will enter the Bug Bounty Program after their findings are resolved. Again, BBP should NOT replace your other security activities, they are an additional source with possible unforeseen benefits.
- pdubs1 7y ago
  one word: crowdsource.
- detaro 7y ago
keydutch 7y ago
Commercial bug bounty companies like Hackerone and Bugcrowd will suffer the most from the crisis for sure. Even more then pentesters. When there are such cool sites like Openbugbounty, all they have to do with their abnormal pricing is to organize their own funerals.

119 points by keydutch 7y ago

> "I've seen some quote unquote valid vulnerability reports," Laurens ("lvh") Van Houtven, principal at Latacora, a secops and cryptography expert, tells CSO. "If someone asked me 'should I put this in my appsec report?', I'd say 'you can put it in there, but I will never let you live it down.'" Is there any reason that someone who works for a company that wants to be perpetually hired by startups to do security would poo-poo a bug bounty program? This comment seems pretty motivated. Additionally, a vulnerability is a vulnerability. Plenty of companies get owned due to pretty trivial vulnerabilities. > Moussouris, now founder and CEO of bug bounty consultancy Luta Security, questions how much of HackerOne is real. Again, could be motivated. > The bug bounty platforms' NDAs prohibit even mentioning the existence of a private bug bounty. Tweeting something like "Company X has a private bounty program over at Bugcrowd" would be enough to get a hacker kicked off their platform. Sure. How many traditional pentesters are signing NDAs? > Consider a finder who spends weeks or months of unpaid work to discover and document a security flaw. Someone else independently discovers, documents and submits that same bug five minutes before the first finder. Under the rules of most HackerOne and Bugcrowd bounty programs, the first submitter gets all the money, the second finder gets nothing. How are we proposing this alternately works in a way that isn't gameable by having control of multiple accounts? > The lack of vetting of bug bounty hunters, where anyone, including this reporter, can sign up for a HackerOne or Bugcrowd account with any email address, is the key sticking point, Antokol says. > Bugcrowd declined to elaborate on what the process of pre-vetting researchers actually looks like. So, this reporter signed up for an account and had immediate access to all public programs without any additional steps. What is really the point of this so-called journalism? In the preceding paragraphs we were complaining about NDAs stopping people talking about private programs, then complaining about the lack of vetting, then saying we gain access to the PUBLIC programs when we sign up, then saying the bug bounty companies refuse to talk about their vetting methods??? What is this? I can see why Bugcrowd and Project Zero refused to talk to this person. > Mature organizations can and should run their own VDP in house. If they are ready for an avalanche of dubious bug reports, they might optionally choose to run a bug bounty. I have a feeling that this journalist is pretty in bed with the traditional pen-testing industry. His history of articles is pretty focused. Bug bounty programs use a customer's real money to pay out bugs. Bugcrowd, at least, triages those bugs before any payment is made. It is in the interest of nobody involved to submit "dubious" bug bounty reports. The only way I can explain that opinion is to think it comes from the mouth of someone with a grudge.

tptacek 7y ago
Latacora is paid in part to manage bug bounty programs, so the motivation is not in the direction you suggest it is.
- gyanchawdhary 7y ago
  In part.
  - lvh 7y ago
    We bootstrap security teams in a long-term engagement. We staff bug bounty programs, but do not bill for that service separately. If a client uses H1 and pays them for triage, it fairly straightforwardly removes a cost for us. The bug bounty nonsense spray I'm talking about is "sev crit: DMARC is set to quarantine instead of reject" and literal descriptions of how session cookies work (that inexplicably get paid out for: we've seen it happen). I assume we're in agreement I should not be fine with people putting that in pentest reports; if that's the case, the quote is not really controversial: all it says is that H1 has a lot of nonsense reports. The mechanism GP describes would require Latacora to be a proxy good for H1. Do you believe that is the case?
- tomlockwood 7y ago
  I'd find it hard to believe a company that was positioning itself as experts in security wouldn't be motivated to talk down the output of other security vendors.
  - tptacek 7y ago
    This is a superficial argument of a form the guidelines for the site ask you not to make. (I'm affiliated with Latacora, for what it's worth; you're just dead wrong about the commercial motivations here).
    - tomlockwood 7y ago
      In good faith I'm addressing your comment about a security vendor's motivations. I don't really mind who it is or what industry it is. I expect a database company to talk down the capabilities of other database companies, even if they sometimes use or provide that company's products.

tptacek 7y ago

I think bounty programs are mostly a bad idea for startups and medium-sized tech companies. That said, the critiques in this piece do not make a lot of sense to me. Take transparency. The claim this article makes is that commercial bounty programs work against transparency by paying researchers only when they agree to NDAs. But transparency isn't a norm in software security to begin with; most vulnerability researchers work for labs and consultancies that rarely if ever disclose vulnerabilities. Disclosure of findings is not a norm on commercial software security assessments. Meanwhile, for any target an independent researcher can lawfully assess, the researcher retains the ability to ignore the bounty program and publish straight to Twitter. My sense of it is that HackerOne has probably increased transparency, in the sense that I've read a lot of published reports on H1 that I don't expect I would have seen if the platform didn't exist, and, in my commercial work, haven't seen a lot of private reports that cut the other way. Or this "Safe Harbor" argument, that commercial bounties force researchers to sign NDAs to be immunized from CFAA suits and prosecutions. Sure, but in the absence of H1, most of those CFAA immunizations weren't available on any terms. H1 doesn't enforce CFAA liability; CFAA liability is a natural default under US law. If anything, H1 is mitigating CFAA concerns, not amplifying them. I don't know what to say about the labor law concerns here. I know that the people offering legal opinions here are lawyers and are versed in California labor law. But knowing what I know about how bug bounty people work: there is no way the median bug bounty "participant" would qualify for minimum wage and benefits at the various companies they interact with. The modal bounty participant has a grab bag of a couple dozen scripts they spam against hundreds of different companies with bounties. Is the claim here that H1 owes them a wage? Is the argument here simply that H1 needs to move out of California? The minimum wage thing doesn't ring true either, since it implies that all project-based consulting --- that is, non-T&M consulting where clients pay an agreed-upon rate for an outcome regardless of the time a project takes to complete --- is susceptible to the "ABC" test as well. But project contracts are very common in California. I'm sure there's a subtlety I'm missing. The "ISO compliance" thing is just silly. By the logic in this article, practically none of the thousands of commercial application pentests performed in 2020 to date will be "ISO compliant". Ultimately: I think if you have to ask, you shouldn't run a bounty program. But that's mostly because I think bounty programs don't work very well, and generate an avalanche of noise. I don't think many of the reasons in this article matter.

gyanchawdhary 7y ago

This article is a) very long and written mostly as a whitepaper/opinion piece and less of a journalistic report b) it's very likely a paid hit piece against bug bounty companies. However, I'm not surprised as the timing makes sense. The COVID-19 situ has impacted a lot of pen-testing firms [canceled testing contracts/security staff augmentation, longterm security audits] and it's expensive AF for these companies to keep their consultants on bench. Bug bounties are by no means perfect, but whatever this article is trying to point out isn't going to stop CISO's from doubling down on bug bounties. In fact, CISO's who've been skeptic about BBs will be now be pressed to explore this option even more. About Me: Sold a few security startups, before that worked as the head of app sec testing for a bailed out british investment bank.

tptacek 7y ago
Who would pay to put a hit piece on bug bounty providers? What pentesting firm do you think has their shit together to the extent that they run a media relations team? Who among them don't have revenue streams from triage consulting? It's a bad article. CSO publishes a lot of bad articles. But there's no way this is a paid piece. The people quoted in this article believe what they're saying.
- gyanchawdhary 7y ago
  > has their shit together to the extent that they run a media relations team. Any company that drops cool research (ATM / Automotive hacking / SCADA etc) use PR firms to drum up their work (which is fair IMHO for SEO/showcasing your new capabilities and service offerings). I did it and I know loads of boutique firms in the UK who do it. I also know of education companies working closely with PR firms to publish articles about "cybersecurity skills gap" or "cybersecurity shortage" to manufacture and invent demand and they are pretty effective too. So it's not that outlandish to think that security companies aren't tactical enough to have PR companies write such articles either. My suspicion about this specific article was given its timing. > Who among them don't have revenue streams from triage consulting? Trigale Consulting is good money but it's mostly getting crumbs from the master's table. BB's are at the apex of this pyramid and will continue to do so. Sharp boutique firms that specialize in stuff will obviously always thrive but that's a diff topic >It's a bad article. CSO publishes a lot of bad articles. Agree > But there's no way this is a paid piece. Maybe but I'm still doubtful. > The people quoted in this article believe what they're saying. Fair point.

anonytrary 7y ago

Bug bounty programs don't make much sense to me. Why would you find bugs and hire external people to fix them when you can hire external people to find bugs and pay trusted employees to fix them? Just hire pentesters.

mic47 7y ago
Bug bounty programs are not supposed to replace you other security activities, but it's a way for you to have additional source of vulnerabilities. Advantages of these programs is that security researcher will get rewarded when they find a bug, and that there is clear process for disclosing bugs. You still should hire pentesters, you still should have trusted employees to find bugs and fix them, and more... If you are relying just on bug bounties, your security will suck. That being said, NDA's sound sketchy, if you disclose bug, than after it is fixed, you should be able to blog about it (or when they do not fix it for looong time).
- PappaPatat 7y ago
  > Bug bounty programs are not supposed to replace you other security activities, but it's a way for you to have additional source of vulnerabilities. Exactly the way we position our own Bug Bounty Program. Where the pentesters can be hired to also confirm things done well, the hunters are only paid for failures they found. In our case there is an added bonus with the Bug Bounty Program: we've come to REALLY apriciate the technical level of reports. Since they only get paid for triagable findings, the details we get reported are so much better then what we used to get from our pentesters. Of course we now require the same quality of reporting from them. What also helps is that the pentesters are motivated more to deliver higher quality findings since they are aware the service will enter the Bug Bounty Program after their findings are resolved. Again, BBP should NOT replace your other security activities, they are an additional source with possible unforeseen benefits.
pdubs1 7y ago
one word: crowdsource.
detaro 7y ago

keydutch 7y ago

Commercial bug bounty companies like Hackerone and Bugcrowd will suffer the most from the crisis for sure. Even more then pentesters. When there are such cool sites like Openbugbounty, all they have to do with their abnormal pricing is to organize their own funerals.