4 ms·

Do you want a solution that makes such an attack unprofitable to execute (i.e. it raises the cost of doing a similar attack above $4.5M)? Buy insurance and pra

by Veserv 6y ago

Do you want a solution that makes such an attack unprofitable to execute (i.e. it raises the cost of doing a similar attack above $4.5M)? Buy insurance and pray. There is not a single readily available enterprise solution that would even dare to put that in writing, let alone deliver. If there is one that does dare, ask to test their claim by having the deal conditional on them putting out an open $4.5M bug bounty and nobody collecting on it (you should also be the one to determine if they have to pay). Every company will either back out or get collected on, no question. This is a good test because if it costs more than $4.5M to do such an attack, then it would be unprofitable to collect on the bug bounty. It is also unlikely to give a false positive (their system is good when it is not) since $4.5M is more than than almost every other bug bounty and it is totally above board, so you will get the best of the best trying to break into the system.

jfoster 6y ago
Not a good test. If they expect a decent ROI, the attack can't cost any amount up to $4.5M to execute. The $4.5M isn't a guaranteed outcome from such an attack. Some targets may not be willing to pay, or may only be willing to pay a substantially smaller amount. Also, part of the execution cost is the non-monetary "breaking the law" factor. For instance, would you rather make a legal $1M or an illegal $2M?
- Veserv 6y ago
  Are you saying it is a bad test because it overestimates the difficulty? If so, I agree that it overestimates the difficulty. My argument is that nobody would dare to accept such a test, which is clearly an overestimate of difficulty, therefore nobody is even close to achieving the actual number. If you wanted a more accurate estimate of security you would probably need to bump it up by a factor of 3-5x to account for execution risk. If you are not, I am not sure what you are arguing since all of your statements show how the test overestimates the difficulty. The test is designed to identify, with reasonably high confidence, whether any attack is profitable given a specific upside. Therefore it should be as easy as possible for a legitimate bug that could result in at least a $4.5M upside to be paid. A 100% guaranteed legal payout clearly minimizes the risk and thus allows more attacks below a $4.5M cost to execute to be profitable. Any other form of payout means the cost to execute must be lower to be profitable. Put another way, if it is unprofitable to do it totally legally for some amount of money, it is probably even more unprofitable to do it illegally for the same amount of money (obviously this excludes cases where you might be able to gain a higher upside, but then we are not talking about mitigating attacks with a certain upside), therefore this estimate should be no lower than the true cost to execute (on average). To use your examples: If they want a decent ROI, the cost of attack must be significantly less than $4.5M to execute to be a good investment. ROI of illegal actions usually needs to be higher to make up for the risk since you would almost always choose a legal action with the same ROI. If the probability of payout is less than 100%, then the cost of attack must be less than $4.5M to make up for the reduced probability of success. The probability of payout for a bug bounty is usually higher than the highly variable payout of an attack. Also since the buyer is using this to make a quality decision and has authority to force the vendor to pay out with the stated scheme, it is in the buyers best interest to pay out credible attacks. If the non-legal nature is a serious cost, then the cost of attack must be less than $4.5M for the illegal case to make up for the extra risk and cost. Non-legal actions usually come with extra costs compared to legal actions and thus have to be even more profitable to be worth doing. Therefore, if a $4.5M bug bounty (where payout is decided by the product buyer) is not claimed after some reasonable amount of time we can conclude, with some reasonable amount of confidence, that the lowest risk option is likely unprofitable. Therefore, higher risk illegal options with the same upside are even less likely to be profitable. Thus, the test is a relatively good lower-bound for identifying if attacks with a $4.5M upside are actually being mitigated. If you can not even institute this lower bound, then you are nowhere near the necessary level.

by Veserv 6y ago

jfoster 6y ago

Not a good test. If they expect a decent ROI, the attack can't cost any amount up to $4.5M to execute. The $4.5M isn't a guaranteed outcome from such an attack. Some targets may not be willing to pay, or may only be willing to pay a substantially smaller amount. Also, part of the execution cost is the non-monetary "breaking the law" factor. For instance, would you rather make a legal $1M or an illegal $2M?

Veserv 6y ago
Are you saying it is a bad test because it overestimates the difficulty? If so, I agree that it overestimates the difficulty. My argument is that nobody would dare to accept such a test, which is clearly an overestimate of difficulty, therefore nobody is even close to achieving the actual number. If you wanted a more accurate estimate of security you would probably need to bump it up by a factor of 3-5x to account for execution risk. If you are not, I am not sure what you are arguing since all of your statements show how the test overestimates the difficulty. The test is designed to identify, with reasonably high confidence, whether any attack is profitable given a specific upside. Therefore it should be as easy as possible for a legitimate bug that could result in at least a $4.5M upside to be paid. A 100% guaranteed legal payout clearly minimizes the risk and thus allows more attacks below a $4.5M cost to execute to be profitable. Any other form of payout means the cost to execute must be lower to be profitable. Put another way, if it is unprofitable to do it totally legally for some amount of money, it is probably even more unprofitable to do it illegally for the same amount of money (obviously this excludes cases where you might be able to gain a higher upside, but then we are not talking about mitigating attacks with a certain upside), therefore this estimate should be no lower than the true cost to execute (on average). To use your examples: If they want a decent ROI, the cost of attack must be significantly less than $4.5M to execute to be a good investment. ROI of illegal actions usually needs to be higher to make up for the risk since you would almost always choose a legal action with the same ROI. If the probability of payout is less than 100%, then the cost of attack must be less than $4.5M to make up for the reduced probability of success. The probability of payout for a bug bounty is usually higher than the highly variable payout of an attack. Also since the buyer is using this to make a quality decision and has authority to force the vendor to pay out with the stated scheme, it is in the buyers best interest to pay out credible attacks. If the non-legal nature is a serious cost, then the cost of attack must be less than $4.5M for the illegal case to make up for the extra risk and cost. Non-legal actions usually come with extra costs compared to legal actions and thus have to be even more profitable to be worth doing. Therefore, if a $4.5M bug bounty (where payout is decided by the product buyer) is not claimed after some reasonable amount of time we can conclude, with some reasonable amount of confidence, that the lowest risk option is likely unprofitable. Therefore, higher risk illegal options with the same upside are even less likely to be profitable. Thus, the test is a relatively good lower-bound for identifying if attacks with a $4.5M upside are actually being mitigated. If you can not even institute this lower bound, then you are nowhere near the necessary level.