5 ms·

This wasn't unexpected outside of the extend of the bounties. What you have to realize is how important Security is to Shopify. We are a trust based business t

by xal 10y ago

This wasn't unexpected outside of the extend of the bounties. What you have to realize is how important Security is to Shopify. We are a trust based business to an extreme extend. We host the livelihoods of hundreds of thousands of other businesses. If we are down or compromised all of them can't make money ( as some of you saw during Black Friday, to the tune of $300k+ a minute at times ). One of the best ways for us to augment our internal security team is to work with the white hat community. This was a pain before Hacker One but now is significantly easier. One challenge is that Shopify (still) hasn't really got the profile in the tech industry that a lot of Silicon Valley local companies have. This is totally fine by me, but it's means that if a top white hat sits down and decide what to work on, we are not automatically top of mind. So we decided to overspent as a kind of "marketing" investment. Hacker one is a classical two sided market place. There is plenty of supply of skilled researchers but also a lot of demand for their services. We want to be known for being one of the most responsive companies and also pay top dollars for top findings. So the basic idea is that when we launch something new, we 10x the payouts to bootstrap the process of familiarization. We also provide a very convenient local environment for doing the work in. It should be more fun and more lucrative to make Shopify related discoveries then other companies. After this initial period we then reduce the payouts somewhere slightly above community standards. Its all just business 101. Internally we are actually thrilled how the shopify-scripts/mruby program went. Most (all?) of what was found would have been caught by our sandboxing but we don't want to rely on this. As everyone who does security knows - lots of exploits, even if superficially contained, can sometimes combine into "the big one".

raesene6 10y ago
So I'd be interested to know your thoughts on bug bounties as against "traditional" security reviews. Have these areas of your application been through external reviews before being opened up to bug bounty or did you decide to start there? I was thinking that for the amount you've paid out in bounties you could've engaged a reasonable team for several man-months, so was interested in what led you more down the bug bounty line for this.
- user5994461 10y ago
  > you could've engaged a reasonable team for several man-months Doubt that. 1) The daily rate for this kind of work is high. 2) It's hard to find people who can execute. (read: security as in pen testing, not security as in filling a PCI compliance checklist). 3) Multiply 1 and 2 by the number of people you want in the team.
  - raesene6 10y ago
    Sorry, but got to disagree with you there. I'm a security tester and have been in the industry for 15+ years either as a buyer or provider of services, for small and large companies, I've been involved in procurement of multi hundred $k tests and involved in the delivery similar sized engagements. You absolutely can get good security testing consultants for $2k/day for example, and probably less depending on the region and exact speciality. for $368k at a $2k/day rate that would be 184 person-days or several person months (as stated). It's well known there are teams who specialize in specific technologies, and with a sophisticated customer, not too tricky to interview and make sure you get the right people.
    - user5994461 10y ago
      Sorry. Should have clarified. They could get pen testers and it would cost as much as they paid so far. Managing a pen test or a bounty program are very different things. I don't think that they planned to spend $300k in exploits when they first rolled out the bounties. A company serious about security should have both anyway: Security audits + bounty programs. They fundamentally cover different things, with some overlap.
      - raesene6 10y ago
        Ahh indeed, that was kind of my initial question for the top-commentor. I'm interested in whether Shopify explored getting security consultants in to review this area before going for a bug bounty on it, or went straight to the bug bounty. My personal feeling is that the order of play should be Internal Security Review --> External Security Review --> Bug Bounty as you can use the first two stages to catch all the basic stuff and some of the advanced stuff then leave the bug bounty to pay out for things the first two elements missed, but that you still want to know about. So I'm interested in data that suggests that companies are either going for that route, or have decided to cut out the external consultant review and go straight to bug bounty. In this case I'd guess part of that would be whether Shopify did indeed expect a $300k+ bug bounty programme or whether that was a surprise to them.
fjdlwlv 10y ago
Paying one engineer's worth of salary to bug bounties isn't overspending, it's freelancing
simonbarker87 10y ago
Not really related but as a Shopify customer of 5 years, we are unbelievably happy with the platform. The thing I love the most is the customer service, we are UK based and it doesn't matter what time of day I call, I get through to someone and they are always incredibly well informed and helpful. I've never had to be transferred, it doesn't matter if I'm calling to talk through a weird DNS issue, ask what the current best way to work on our theme locally is or to simply add a specific feature from the top tier (real-time carrier rates) to our plan, the first person I speak to is the one that helps the whole way through. This is so rare with a tech, and frankly, any company and is the single biggest reason why we never look anywhere else or consider moving. As far as stability goes we've never had an outage and never had a slow down due to high traffic even after national radio appearances. And you're right about livelihoods being on the line - our Shopify store isn't our biggest revenue stream but it's the one that generates most of the profit as we make higher margins selling direct, if our store were to go down it would be a nightmare.

by xal 10y ago

raesene6 10y ago

So I'd be interested to know your thoughts on bug bounties as against "traditional" security reviews. Have these areas of your application been through external reviews before being opened up to bug bounty or did you decide to start there? I was thinking that for the amount you've paid out in bounties you could've engaged a reasonable team for several man-months, so was interested in what led you more down the bug bounty line for this.

user5994461 10y ago
> you could've engaged a reasonable team for several man-months Doubt that. 1) The daily rate for this kind of work is high. 2) It's hard to find people who can execute. (read: security as in pen testing, not security as in filling a PCI compliance checklist). 3) Multiply 1 and 2 by the number of people you want in the team.
- raesene6 10y ago
  Sorry, but got to disagree with you there. I'm a security tester and have been in the industry for 15+ years either as a buyer or provider of services, for small and large companies, I've been involved in procurement of multi hundred $k tests and involved in the delivery similar sized engagements. You absolutely can get good security testing consultants for $2k/day for example, and probably less depending on the region and exact speciality. for $368k at a $2k/day rate that would be 184 person-days or several person months (as stated). It's well known there are teams who specialize in specific technologies, and with a sophisticated customer, not too tricky to interview and make sure you get the right people.
  - user5994461 10y ago
    Sorry. Should have clarified. They could get pen testers and it would cost as much as they paid so far. Managing a pen test or a bounty program are very different things. I don't think that they planned to spend $300k in exploits when they first rolled out the bounties. A company serious about security should have both anyway: Security audits + bounty programs. They fundamentally cover different things, with some overlap.
    - raesene6 10y ago
      Ahh indeed, that was kind of my initial question for the top-commentor. I'm interested in whether Shopify explored getting security consultants in to review this area before going for a bug bounty on it, or went straight to the bug bounty. My personal feeling is that the order of play should be Internal Security Review --> External Security Review --> Bug Bounty as you can use the first two stages to catch all the basic stuff and some of the advanced stuff then leave the bug bounty to pay out for things the first two elements missed, but that you still want to know about. So I'm interested in data that suggests that companies are either going for that route, or have decided to cut out the external consultant review and go straight to bug bounty. In this case I'd guess part of that would be whether Shopify did indeed expect a $300k+ bug bounty programme or whether that was a surprise to them.

fjdlwlv 10y ago

Paying one engineer's worth of salary to bug bounties isn't overspending, it's freelancing

simonbarker87 10y ago

Not really related but as a Shopify customer of 5 years, we are unbelievably happy with the platform. The thing I love the most is the customer service, we are UK based and it doesn't matter what time of day I call, I get through to someone and they are always incredibly well informed and helpful. I've never had to be transferred, it doesn't matter if I'm calling to talk through a weird DNS issue, ask what the current best way to work on our theme locally is or to simply add a specific feature from the top tier (real-time carrier rates) to our plan, the first person I speak to is the one that helps the whole way through. This is so rare with a tech, and frankly, any company and is the single biggest reason why we never look anywhere else or consider moving. As far as stability goes we've never had an outage and never had a slow down due to high traffic even after national radio appearances. And you're right about livelihoods being on the line - our Shopify store isn't our biggest revenue stream but it's the one that generates most of the profit as we make higher margins selling direct, if our store were to go down it would be a nightmare.