5 ms·

> As long as you will still delete accounts that you suspect of fraud, based on some opaque definition of suspecting fraud, "we don't want this to happen" is th

> As long as you will still delete accounts that you suspect of fraud, based on some opaque definition of suspecting fraud, "we don't want this to happen" is the best you can do, and that's not very reassuring. Having worked in fraud detection, this is true for every provider you use. The adversarial nature of fraud detection means making _your definition of fraud_ public is not an option because people will use it as a loss function for their fraud. I would just make sure I don't rely on a single provider and minimize single points of failure.

notatoad 8y ago
i understand why fraud detection mechanisms need to be opaque. i don't understand why the consequence has to be "we will delete your account"
- kajecounterhack 8y ago
  In the context of cloud instances, accounts allocate resources. If they never deleted fraudulent accounts, they couldn't release the allocated resources, which is a huge problem. Imagine bad guy x who steals 100k credit cards, then uses a botnet to spin up VMs on 100k fraudulent accounts; if each VM has 10gb of data on it, you've just frozen 1 petabyte. Say there's a 90 day appeals process; those resources can't be deleted or reclaimed for 90 days, which is already tremendously expensive. If that expanded to forever, you would just have to close up shop. In reality bad guys operate at even larger scales than this, doing all kinds of things. Depending on what company you are, you might not even be able to pull the compute resources away from them right away; so in addition to burning disk alloc they're now burning compute units. It depends on the product too -- what if they spin up pubsub instances and leave messages in the queue? You might have to freeze those queues so that if they're valid customers they don't lose data. The more offerings you have, the more potential loss. But yeah, they don't delete accounts right away -- there is indeed a grace period, and they do keep all your stuff (I suspect legislation may have something to do with it too). Last I remember it goes to appeal process (human review), and your data can't be deleted / resources reclaimed for quite a while. So it's not zero recourse for the user. Sounds like they're adding even more humans into the loop to ensure they don't unfortunately flag and then delete valid users. But it's non-trivial.
  - fjsolwmv 8y ago
    In your extreme example of 100k valid stolen credit cards being burned simultaneously in one attack, you've got 1PB which is only $25k/mo online, $7k/mo coldline, retail, so even less wholesale or on tape backup. That's hardly noticeable on the bottom line.
    - kajecounterhack 8y ago
      Fwiw speaking from experience 100k is a toy problem and the "fake accounts freezing out resources" problem is _very_ noticeable to the bottom line. The scale of attacks they face are orders of magnitude higher. The 10gb figure is also contrived. If you don't freeze the _bandwidth and compute_ resources you're even more fucked, because those have far worse scaling characteristics. So big company X taking immediate action to take down services make a lot of sense in that light. They don't do it in every situation though e.g borderline cases that require human review. Fraud fighting is costly.
  - SXX 8y ago
    Sorry, but if a company has such a big trouble with fraud it's absolutely doesn't mean it has any right to destoy someone else business. Especially considering that as you noted criminals usually tend to work on scale and they usually won't bother with registering any kind of legal entity or doing some manual confirmation, phone talk, etc. And what more important: there is plenty of services that will do lot's of check in advance and ask your passport / card / face photo and such, but they never ever stop your servers or destoy your data unless you do something extremely bad. And if it's come to company account it's even less likely to get in any trougle since most of services take work with companies serious. Yet Google just like everyone around have extremely easy sign up process that all criminals are happy about, but can blow you in a face at any time. So they first create a problem to make marketing easier and then try to compensate for it by increasing risk of fault for thrir legit customers.
    - kajecounterhack 8y ago
      > Yet Google just like everyone around have extremely easy sign up process that all criminals are happy about, but can blow you in a face at any time. So they first create a problem to make marketing easier and then try to compensate for it by increasing risk of fault for thrir legit customers. That's a good point. What's interesting is that Google's capabilities mean that they are able to allocate you the resources you ask for post-signup pretty much instantly, whereas AWS doesn't (they actually are just unable to allocate resources so quickly). This turns out to be a nice property if that n day spinup time window is enough to detect some bad guys and take them down in the interim, before they've caused massive damage to your infra / cost to your org. It's an O(1) improvement though. Bad guys will still just wait you out and then cause damage, so it's not like you're actually less likely to have your account frozen at AWS/Azure vs Google.
tomjen3 8y ago
It is a violation of the gdpr to not the user why he was suspended, sharing any data about why with the user, not allowing him to correct the data if they are wrong or not allowing decisions made by algorithms to be appealed for human review. Missing any of those and you might be subject to fines, although this only applies to human customers.
- comex 8y ago
  Unlikely: https://academic.oup.com/idpl/article/7/2/76/3860948 https://academic.oup.com/idpl/article/7/2/76/3860948
- TheIronYuppie 8y ago
  The customer has been informed.

notatoad 8y ago

i understand why fraud detection mechanisms need to be opaque. i don't understand why the consequence has to be "we will delete your account"

kajecounterhack 8y ago
In the context of cloud instances, accounts allocate resources. If they never deleted fraudulent accounts, they couldn't release the allocated resources, which is a huge problem. Imagine bad guy x who steals 100k credit cards, then uses a botnet to spin up VMs on 100k fraudulent accounts; if each VM has 10gb of data on it, you've just frozen 1 petabyte. Say there's a 90 day appeals process; those resources can't be deleted or reclaimed for 90 days, which is already tremendously expensive. If that expanded to forever, you would just have to close up shop. In reality bad guys operate at even larger scales than this, doing all kinds of things. Depending on what company you are, you might not even be able to pull the compute resources away from them right away; so in addition to burning disk alloc they're now burning compute units. It depends on the product too -- what if they spin up pubsub instances and leave messages in the queue? You might have to freeze those queues so that if they're valid customers they don't lose data. The more offerings you have, the more potential loss. But yeah, they don't delete accounts right away -- there is indeed a grace period, and they do keep all your stuff (I suspect legislation may have something to do with it too). Last I remember it goes to appeal process (human review), and your data can't be deleted / resources reclaimed for quite a while. So it's not zero recourse for the user. Sounds like they're adding even more humans into the loop to ensure they don't unfortunately flag and then delete valid users. But it's non-trivial.
- fjsolwmv 8y ago
  In your extreme example of 100k valid stolen credit cards being burned simultaneously in one attack, you've got 1PB which is only $25k/mo online, $7k/mo coldline, retail, so even less wholesale or on tape backup. That's hardly noticeable on the bottom line.
  - kajecounterhack 8y ago
    Fwiw speaking from experience 100k is a toy problem and the "fake accounts freezing out resources" problem is _very_ noticeable to the bottom line. The scale of attacks they face are orders of magnitude higher. The 10gb figure is also contrived. If you don't freeze the _bandwidth and compute_ resources you're even more fucked, because those have far worse scaling characteristics. So big company X taking immediate action to take down services make a lot of sense in that light. They don't do it in every situation though e.g borderline cases that require human review. Fraud fighting is costly.
- SXX 8y ago
  Sorry, but if a company has such a big trouble with fraud it's absolutely doesn't mean it has any right to destoy someone else business. Especially considering that as you noted criminals usually tend to work on scale and they usually won't bother with registering any kind of legal entity or doing some manual confirmation, phone talk, etc. And what more important: there is plenty of services that will do lot's of check in advance and ask your passport / card / face photo and such, but they never ever stop your servers or destoy your data unless you do something extremely bad. And if it's come to company account it's even less likely to get in any trougle since most of services take work with companies serious. Yet Google just like everyone around have extremely easy sign up process that all criminals are happy about, but can blow you in a face at any time. So they first create a problem to make marketing easier and then try to compensate for it by increasing risk of fault for thrir legit customers.
  - kajecounterhack 8y ago
    > Yet Google just like everyone around have extremely easy sign up process that all criminals are happy about, but can blow you in a face at any time. So they first create a problem to make marketing easier and then try to compensate for it by increasing risk of fault for thrir legit customers. That's a good point. What's interesting is that Google's capabilities mean that they are able to allocate you the resources you ask for post-signup pretty much instantly, whereas AWS doesn't (they actually are just unable to allocate resources so quickly). This turns out to be a nice property if that n day spinup time window is enough to detect some bad guys and take them down in the interim, before they've caused massive damage to your infra / cost to your org. It's an O(1) improvement though. Bad guys will still just wait you out and then cause damage, so it's not like you're actually less likely to have your account frozen at AWS/Azure vs Google.

tomjen3 8y ago

It is a violation of the gdpr to not the user why he was suspended, sharing any data about why with the user, not allowing him to correct the data if they are wrong or not allowing decisions made by algorithms to be appealed for human review. Missing any of those and you might be subject to fines, although this only applies to human customers.

comex 8y ago
Unlikely: https://academic.oup.com/idpl/article/7/2/76/3860948 https://academic.oup.com/idpl/article/7/2/76/3860948
TheIronYuppie 8y ago
The customer has been informed.