3 ms·

Yeah we need strong anti-Kafka laws so that individuals can always find out what they've been "charged" with and that there always need to be an "appeal" proces

Yeah we need strong anti-Kafka laws so that individuals can always find out what they've been "charged" with and that there always need to be an "appeal" process and businesses are strictly denied the ability to claim that they can't reveal their methods.

jimmydddd 3y ago
I had my Venmo account frozen. I'd only made about three payments. Two to a food bank and one to a school organization. Via email, they said the decision was permanent and their policy was to not discuss why the decision was made. Luckily for me it was only Venmo, which I don't really need to use.
- oblio 3y ago
  Similar story with Uber UK. Account banned but just in the UK... After tens of rides across the world.
nine_k 3y ago
Listing the specific problem that led to a service denial could be helpful, but could be not. I bet that it's very rarely something like "You did <this one forbidden thing>, so we ban you according to the rule 3.6.4.2 of ToS". It could be something like: "The pattern of your activity is very similar to patterns of activity of 9823 known scam attacks". Or even more detailed: "8 out of 11 statistical models analyzing 982 parameters of your activity during last 48 days showed the probability of your account being used for illicit fund transfers above 0.92, and the remaining 3, above 0.57. In particular, our 100M-paremeter AI model (link to a PDF with technical details) has classified your activity thusly: (a 20-row table follows). According to the law, we use a publicly available model, widely accepted as a standard in the industry. You can inspect the model's weights at huggingface (link follows)". Interpreting such results is a job. Disputing such results in a court, or with an arbiter, would be really hard, unless you're a practitioner of the pattern-recognition craft yourself.
- oblio 3y ago
  Doesn't matter. Just forcing them to disclose that info would force them to clean up their act.
- lamontcg 3y ago
  There's no need for the model weights and parameters. The probabilities output from a multilabel classification model would be entirely sufficient to tell what the model thought the user had done. It was trained on a category and it is claiming that the user is part of that category. That category should have a human description of what the corporate entity was looking for and that should be enough for some kind of arbiter to determine if the 'accused' was actually doing that or not. The question isn't what the model is doing, the question is if the person is a false positive or not, which should be able to be determined without any reference to the model at all. If the model is doing too many false positives then determining why the model is broken becomes the job of the corporation (ideally with some punitive financial damages as incentive). Although I do think that the model weights and parameters should in some way be subpoenable so that the model can be 'compelled to testify' if someone has enough legal, technical and fiscal resources to sue the company.
eunos 3y ago
This is one of the consequence of strong Anti Money Laundering law. Bank much prefers to avoid False Negative (letting money launderers operate).
- seanmcdirmid 3y ago
  It isn’t the bank’s preferences. Rather it is forced by the liability bestowed by them by the government. They don’t get the benefit of safe harbor like ISPs.
kylebenzle 3y ago
They can't even get the legal system to function, how are they going to fix ALL corporations? Some of best examples are the juvenile and and family court systems in a state like Ohio. Not only do they not even TRY to do what is fair or right, they have one simple mandate, pay for their own existence. As soon as the system set up to punish children is made to profit off those same children you have a sick and broken system but thats where we are today. So, VERY little chance we are going to convince a company to make less money for the social good when even out children's courts are run like a business.

jimmydddd 3y ago

I had my Venmo account frozen. I'd only made about three payments. Two to a food bank and one to a school organization. Via email, they said the decision was permanent and their policy was to not discuss why the decision was made. Luckily for me it was only Venmo, which I don't really need to use.

oblio 3y ago
Similar story with Uber UK. Account banned but just in the UK... After tens of rides across the world.

nine_k 3y ago

Listing the specific problem that led to a service denial could be helpful, but could be not. I bet that it's very rarely something like "You did <this one forbidden thing>, so we ban you according to the rule 3.6.4.2 of ToS". It could be something like: "The pattern of your activity is very similar to patterns of activity of 9823 known scam attacks". Or even more detailed: "8 out of 11 statistical models analyzing 982 parameters of your activity during last 48 days showed the probability of your account being used for illicit fund transfers above 0.92, and the remaining 3, above 0.57. In particular, our 100M-paremeter AI model (link to a PDF with technical details) has classified your activity thusly: (a 20-row table follows). According to the law, we use a publicly available model, widely accepted as a standard in the industry. You can inspect the model's weights at huggingface (link follows)". Interpreting such results is a job. Disputing such results in a court, or with an arbiter, would be really hard, unless you're a practitioner of the pattern-recognition craft yourself.

oblio 3y ago
Doesn't matter. Just forcing them to disclose that info would force them to clean up their act.
lamontcg 3y ago
There's no need for the model weights and parameters. The probabilities output from a multilabel classification model would be entirely sufficient to tell what the model thought the user had done. It was trained on a category and it is claiming that the user is part of that category. That category should have a human description of what the corporate entity was looking for and that should be enough for some kind of arbiter to determine if the 'accused' was actually doing that or not. The question isn't what the model is doing, the question is if the person is a false positive or not, which should be able to be determined without any reference to the model at all. If the model is doing too many false positives then determining why the model is broken becomes the job of the corporation (ideally with some punitive financial damages as incentive). Although I do think that the model weights and parameters should in some way be subpoenable so that the model can be 'compelled to testify' if someone has enough legal, technical and fiscal resources to sue the company.

eunos 3y ago

This is one of the consequence of strong Anti Money Laundering law. Bank much prefers to avoid False Negative (letting money launderers operate).

seanmcdirmid 3y ago
It isn’t the bank’s preferences. Rather it is forced by the liability bestowed by them by the government. They don’t get the benefit of safe harbor like ISPs.

kylebenzle 3y ago

They can't even get the legal system to function, how are they going to fix ALL corporations? Some of best examples are the juvenile and and family court systems in a state like Ohio. Not only do they not even TRY to do what is fair or right, they have one simple mandate, pay for their own existence. As soon as the system set up to punish children is made to profit off those same children you have a sick and broken system but thats where we are today. So, VERY little chance we are going to convince a company to make less money for the social good when even out children's courts are run like a business.