4 ms·

My commentary was precisely about the state-of-the-practice. That $10k is "an extraordinarily high sum for" what was likely weeks of work on this bug, and prob

by blagie 2y ago

My commentary was precisely about the state-of-the-practice. That $10k is "an extraordinarily high sum for" what was likely weeks of work on this bug, and probably months of work poking in other places, reflects the very, very low focus on security industry-wide. This is why we need significant civil -- or possibly occasionally criminal -- liability. Civil if it's simple negligence, and criminal if it's gross negligence leading to harm. If Google were to pay me $200 if it leaked my data, that would: - Be worth much less than my privacy - Amount to damages of $400B worldwide if there were a compromise impacting all $2B users (although, realistically, damages would be lower in middle and low income countries) This would represent a 20% fall in Google's market cap, which feels about right. At that point, I expect the bug bounties would be set many orders of magnitude higher. Security bugs should be rare. They're common. This is a problem, and one created by our market incentive structures. You are correct that Apple is an exception, and seems to mind security.

tptacek 2y ago
Google pays a piece rate. They pay the rate the market will bear, unless you impress them, like these people did, and then they pay a bit more. They do not compensate you for your working hours. Google is not going to pay you $200 if they leak your email address. Google pays as much attention to security as Apple does. If you want a world in which these kinds of security bugs create multimillion-dollar liabilities, you can advocate for the new statutes that will create that world; just be aware that only companies like Google will be able to afford to operate in that world.
- blagie 2y ago
  I am very much advocating for new statutes. That's precisely what my post was doing. Companies should not be allowed to externalize costs of bad security on users. I disagree with the claim that "only companies like Google will be able to afford to operate in that world." That's not how markets work. 1) The impact would be that frameworks would develop with better security. This would result in a slowdown of software engineering. Perhaps it would start to look like any other engineering discipline, where things are analyzed for safety. 2) Every other industry shows that in situations like this, big players are disadvantaged. The analysis here is pretty basic: - If I'm running a small $10M startup making a little iPhone app for some obscure task, the risk of legal liability from this is among the smallest of my risks of going under, so I'm incentivized to ignore it. If I were faced with a $400B liability, I declare bankruptcy, so in effect, that's a $10M liability. The expected cost is 5% times 10M = $500k, so it makes sense to spend up to $500k to mitigate a 5% risk. - If Google has a team working on that same app, and doesn't manage security properly, the $400B liability stays a $400B liability. There is no ROI analysis where it makes sense to build a little app which has a 5% chance of leaking data. Do it right, or don't do it at all. The expected cost to Google here is 5% times $400B = $20B. This is why, in virtually every other industry, big players are (1) more trusted (2) more expensive, and phrases like "small, fly-by-night operation" exist (and make business sense to run).
  - tptacek 2y ago
    Well, as long as we agree that none of this is how the world works right now, I don't think we have to litigate this.
edanm 2y ago
I think you're looking at this wrong. Security is hard. Incredibly hard. Unlike most things in business which are positive-sum, security isn't - it's adversarial. If we make companies pay huge civil fines for things that are so hard to protect against, we're stifling a ton of innovation. I usually analogize a large company to a bank. A bank is supposed to keep your money secure, and for sure you'd have a legitimate beef if a bank robber could waltz in and steal your money easily because it's not kept in a vault. But what if it is kept in a vault? What if the bank isn't attacked by a random group of bank robbers, but rather by the armed forces of a hostile nation? We don't expect banks to protect against armies - that's what we have states for! They provide centralized protection against threats that are far too large for any individual entity to take on by themselves. This is the same, albeit out of sight, situation with large companies. You can have thousands, tens of thousands of people around the world poking at everything your company does for years, looking for any vulnerability. No company can truly withstand that kind of scrutiny - and I don't think making civil penalties higher will change that. And on top of criminal or opportunistic actors, companies also have to be worried about state actors too. The only way is for the state to take on an active role in security. I don't see any other way that gets real security for anyone.
vineyardmike 2y ago
> If Google were to pay me $200 if it leaked my data, that would: Be worth much less than my privacy I think you need to do a lot to justify a non-zero value for this, frankly. How is your “privacy” worth $200? What data is valuable and what data isn’t? Under what context? If your privacy is worth $200 per leak (by some definition of leak), you surely take steps to anonymize your data already and wouldn’t use a service like Google (or name your untrustworthy party). I’m not saying leaks are good but trying to price it in seems fraught.
paulddraper 2y ago
I think you are vastly overestimating the damage of knowing an email address. I used to get books dropped off at my door with the names, addresses, and phone numbers of thousands of people. The first two are often public record.

by blagie 2y ago

tptacek 2y ago

Google pays a piece rate. They pay the rate the market will bear, unless you impress them, like these people did, and then they pay a bit more. They do not compensate you for your working hours. Google is not going to pay you $200 if they leak your email address. Google pays as much attention to security as Apple does. If you want a world in which these kinds of security bugs create multimillion-dollar liabilities, you can advocate for the new statutes that will create that world; just be aware that only companies like Google will be able to afford to operate in that world.

blagie 2y ago
I am very much advocating for new statutes. That's precisely what my post was doing. Companies should not be allowed to externalize costs of bad security on users. I disagree with the claim that "only companies like Google will be able to afford to operate in that world." That's not how markets work. 1) The impact would be that frameworks would develop with better security. This would result in a slowdown of software engineering. Perhaps it would start to look like any other engineering discipline, where things are analyzed for safety. 2) Every other industry shows that in situations like this, big players are disadvantaged. The analysis here is pretty basic: - If I'm running a small $10M startup making a little iPhone app for some obscure task, the risk of legal liability from this is among the smallest of my risks of going under, so I'm incentivized to ignore it. If I were faced with a $400B liability, I declare bankruptcy, so in effect, that's a $10M liability. The expected cost is 5% times 10M = $500k, so it makes sense to spend up to $500k to mitigate a 5% risk. - If Google has a team working on that same app, and doesn't manage security properly, the $400B liability stays a $400B liability. There is no ROI analysis where it makes sense to build a little app which has a 5% chance of leaking data. Do it right, or don't do it at all. The expected cost to Google here is 5% times $400B = $20B. This is why, in virtually every other industry, big players are (1) more trusted (2) more expensive, and phrases like "small, fly-by-night operation" exist (and make business sense to run).
- tptacek 2y ago
  Well, as long as we agree that none of this is how the world works right now, I don't think we have to litigate this.

edanm 2y ago

I think you're looking at this wrong. Security is hard. Incredibly hard. Unlike most things in business which are positive-sum, security isn't - it's adversarial. If we make companies pay huge civil fines for things that are so hard to protect against, we're stifling a ton of innovation. I usually analogize a large company to a bank. A bank is supposed to keep your money secure, and for sure you'd have a legitimate beef if a bank robber could waltz in and steal your money easily because it's not kept in a vault. But what if it is kept in a vault? What if the bank isn't attacked by a random group of bank robbers, but rather by the armed forces of a hostile nation? We don't expect banks to protect against armies - that's what we have states for! They provide centralized protection against threats that are far too large for any individual entity to take on by themselves. This is the same, albeit out of sight, situation with large companies. You can have thousands, tens of thousands of people around the world poking at everything your company does for years, looking for any vulnerability. No company can truly withstand that kind of scrutiny - and I don't think making civil penalties higher will change that. And on top of criminal or opportunistic actors, companies also have to be worried about state actors too. The only way is for the state to take on an active role in security. I don't see any other way that gets real security for anyone.

vineyardmike 2y ago

> If Google were to pay me $200 if it leaked my data, that would: Be worth much less than my privacy I think you need to do a lot to justify a non-zero value for this, frankly. How is your “privacy” worth $200? What data is valuable and what data isn’t? Under what context? If your privacy is worth $200 per leak (by some definition of leak), you surely take steps to anonymize your data already and wouldn’t use a service like Google (or name your untrustworthy party). I’m not saying leaks are good but trying to price it in seems fraught.

paulddraper 2y ago

I think you are vastly overestimating the damage of knowing an email address. I used to get books dropped off at my door with the names, addresses, and phone numbers of thousands of people. The first two are often public record.