8 ms·

I feel like the Chinese style social score system won't have enough support in government for most countries, but we'll receive a decentralized version anyway t

by Eyght 6y ago

I feel like the Chinese style social score system won't have enough support in government for most countries, but we'll receive a decentralized version anyway through corporate overreach. Also, it's always amusing to arrive at an article about data collection and be greeted by this: https://i.imgur.com/d4Z4sdd.png https://i.imgur.com/d4Z4sdd.png

RealStickman_ 6y ago
That's actually one of the best cookie banners I have seen. You don't even have to deselect everything, just 3 boxes.
icedistilled 6y ago
I mean, it's already standard practice (All states except CA) to use credit score in things like pricing your auto insurance. It's only a tiny leap to incorporate a similar magic number some company comes up with. Actually due to competition, if it correlates to risk, all companies will literally be forced to use the magic numbers or go into an adverse selection death spiral. Unless there is regulation against it, like in CA. Not all regulation is bad.
- white-flame 6y ago
  One difference there is that people do have the right to view and challenge credit reports, while all this other social scoring is hidden from the targeted people.
- TeMPOraL 6y ago
  I wonder at which point it becomes a self-fulfilling property - at which point decisions based on data-driven pigeonholing actually lock people on the paths "discovered" in the numbers? E.g. if a young adult gets classified as "disorderly, drunk, unsuitable for reproduction, suitable only for low-skill work" based on their history of college partying, and then consequently denied work and social opportunities (as everyone doing background checks sees that summary), the prediction essentially becomes a sentence. (The third season of Westworld, despite bad writing and even worse gunfights, was very good at bringing this point up.)
  - api 6y ago
    This is kind of the meta plot of Gattaca. You get what you measure, so if we try to measure "potential" we end up getting only the potential our measurements say we have.
  - joejerryronnie 6y ago
    A Brave New World did it easier, simply sort each person into a genetically pre-determined path prior to birth. No reason to go through all that messy data collection and analysis.
  - zozbot234 6y ago
    Yup, this is what "structural" social prejudice and discrimination are all about, once you strip away all the pointless and meaningless rhetoric that's somehow supposed to be "about" these issues. It's a self-perpetuating equilibrium of basic social conditions and superstructure (viz. discourse, supporting ideas, commonly-held worldviews etc.) that create a nearly unescapable "trap" of invisible oppression.
  - lmkg 6y ago
    Read Weapons of Math Destruction by Cathy O'Neil. The book explores several ways that's already happening. Her main premise is that there's a feedback loop in many data-driven policies. You only get success results for the things that you try, and you only try the things you already think are likely to receive. As a result, algorithmic policies tend to reinforce the status quo. Loan risk algorithms will favor people "similar to" those who have paid back loans before, a sample group biased towards people that banks have already loaned to before. As a result, a lot of the factors are biased towards "from a white upper-middle-class suburban background." And recidivism estimators, which are used as jail sentencing guidelines in some places. Screening algorithms for job resumes, and college applications. Algorithms send police to where crimes are reported. Crimes are reported because the police are there to witness them. The area gets designated a high-crime area. Regular people are arrested more often because regular activity is suspicious in a high-crime area, affecting their future prospects. The higher arrest rate is used to justify this. It's a continuous spectrum rather than a single point. But if I were to pick a single "point" where it became a self-fulling prophecy? 1994, due to the widespread passage of three-strikes laws.
- jacques_chester 6y ago
  > Actually due to competition, if it correlates to risk, all companies will literally be forced to use the magic numbers or go into an adverse selection death spiral. To expand on this: adverse selection is where a consumer has hidden information about the cost they can inflict on a provider. Usually this is talked about in terms of insurance (especially health insurance), but risk is risk and so the principles are the same. My recollection of what economists predict is that there are two stable equilibria that achieve Pareto efficiency. The first is that discrimination (in the general yes/no sense) is completely forbidden and risks are totally pooled. The second is that complete discrimination is possible without limitation. The worst outcomes are all found in attempted compromises. Not only do you have the costs of whatever tradeoff you chose between pooling and discrimination being less efficient than the Pareto points, but you also introduce a great deal of dead weight due to complex regulation and oversight, plus efforts made to evade regulation and oversight. Collectively we are worse off, even if individuals think otherwise. I don't think allowing total discrimination is a viable option in this day and age. Which means banning it wholesale and encouraging the formation of universal risk pools.
Spooky23 6y ago
It’s already happening. Georgia had a anti-opioid program where they could score every pregnant woman in the state for likelihood to become dependent on opioids during pregnancy. Babies born with addiction cost the state about $1M each. There were a few similar programs a few years ago when federal grants were made available. Iirc, they were modeled on the stuff built to identify people vulnerable to becoming extremist terrorist types. Insurance companies have databases of way more lifestyle and other behavior data than people realize. (Everything from sports, politics, to porn and gambling habits — anything for sale) I’m pretty sure Georgia mashed that against Medicaid claim data to build the model. It’s a reason why we all need to watch “pre-existing condition” debates closely. If insurers know that 45 year old divorced father of 3 who moves every 3 months is a smoker, gambler and drinker, they don’t want to write a policy — the guy is a trainwreck with no support system.
- harlanji 6y ago
  As a healthy homeless former professional working as a grocery bagger now, I can feel management watching me for signs of drug abuse to explain my ratty clothes and spotty hygeine. Nobody can guess why I’m not working a good job but seem very smart and thorough (slander from boss destroyed reputation). I’m glad to have a job and sorry to show up looking trashy, but it’s the best I can do right now. I am just thankful that they don’t have any data or scores for me yet to help them see a drug addict or boozer, because I know that I’d raise a lot of flags there and it’d be enough to color the suspicion. Metrics are scary, I know my current scores will make things harder for years to come, starting with credit having bombed right down to these corporate/insurer metrics whose systems I was in as a $200K/yr professional. Lots of explaining the data to do if I make it back to pro life.
  - erikerikson 6y ago
    Glad you're doing what sounds to be healthier. I suspect many think differently here but I don't think the problem lies in the knowledge but in how people with power are currently using the knowledge. Restricting the knowledge is simply or current best mitigation of the current conventions. Another world might use that knowledge to better support you and others who have seen hard times and restore what sounds like a lack of justice and help you find an environment where you could thrive and perhaps even that boss of yours. [edit: so that they could thrive but also have reduced negative impact] In other words, rather than focusing on mitigating risk a sufficiently high quality system could help us maximize our lives. Unfortunately, the probability of something so pro-social being the outcome seems low.
Hoasi 6y ago
> Also, it's always amusing to arrive at an article about data collection and be greeted by this: https://i.imgur.com/d4Z4sdd.png https://i.imgur.com/d4Z4sdd.png Meta amusement: https://i.imgur.com/E5bFeb3.png https://i.imgur.com/E5bFeb3.png
cactus2093 6y ago
The credit system in the US is already a pretty horrific version of this, and nobody really seems to care and it’s been like this for decades. And it’s not even just used for credit anymore, renting an apartment now usually requires a credit check. I had thought credit karma and similar services were at least adding transparency to the industry but was recently in for a rude awakening when trying to get preapproval for our first mortgage. It turns out, the whole idea of a credit score is kind of a lie. Your credit report can be pulled by creditors, and creditors can interpret it in different ways as they see fit. Services like credit karma are just making up their own score based on the report which can be drastically different than what the creditor decides it is. And credit pulled for different uses somehow ends up with different scores, e.g. for a credit card approval your score will often show up higher than for a mortgage. You can check your credit report for free, as required by law, but nobody is required to tell you your credit score. You can also basically just pay to remove many things from your credit report. It’s insane and the fact that this fairly old and low-tech institution has not managed to be regulated successfully or made fair to the average person, bodes very poorly IMO for regulation of privacy and fairness based on future technology.
- lotsofpulp 6y ago
  > You can check your credit report for free, as required by law, but nobody is required to tell you your credit score. You can also basically just pay to remove many things from your credit report. There is no “the credit score”. Any lender is free to use whatever scoring algorithm they like. And they do. Some lenders may choose to use certain scores for certain things, but underwriting has no obligation to use one specific credit score. Therefore there is no reason for people to care about their credit score, whether it be from FICO or credit karma or whoever. The only thing you should do is make sure information on the credit report is accurate. And what is your source for claiming you can pay to get things removed from credit score? It doesn’t even make sense, as it just shows the status of your lines of credit. You can’t just make a line of credit disappear as no credit reporting agency or financial institution is going to want to commit fraud for any amount of money that an average person might offer.
  - 6y ago
trustmeimdrunk 6y ago
last i checked data collection from western-block companies is pretty centralized. Extensive records of your online behavior are being used (hence the money being made in it's collection and sale) by HR dpt, insurance companies, and loan desks to determine whether you qualify for a top tier salary for the same work, affordable coverage, and approval for a loan.
simonebrunozzi 6y ago
It would not be "decentralized". It will still be centralized, however the "master" will not be a government, but a private company.

by Eyght 6y ago

RealStickman_ 6y ago

That's actually one of the best cookie banners I have seen. You don't even have to deselect everything, just 3 boxes.

icedistilled 6y ago

I mean, it's already standard practice (All states except CA) to use credit score in things like pricing your auto insurance. It's only a tiny leap to incorporate a similar magic number some company comes up with. Actually due to competition, if it correlates to risk, all companies will literally be forced to use the magic numbers or go into an adverse selection death spiral. Unless there is regulation against it, like in CA. Not all regulation is bad.

white-flame 6y ago
One difference there is that people do have the right to view and challenge credit reports, while all this other social scoring is hidden from the targeted people.
TeMPOraL 6y ago
I wonder at which point it becomes a self-fulfilling property - at which point decisions based on data-driven pigeonholing actually lock people on the paths "discovered" in the numbers? E.g. if a young adult gets classified as "disorderly, drunk, unsuitable for reproduction, suitable only for low-skill work" based on their history of college partying, and then consequently denied work and social opportunities (as everyone doing background checks sees that summary), the prediction essentially becomes a sentence. (The third season of Westworld, despite bad writing and even worse gunfights, was very good at bringing this point up.)
- api 6y ago
  This is kind of the meta plot of Gattaca. You get what you measure, so if we try to measure "potential" we end up getting only the potential our measurements say we have.
- joejerryronnie 6y ago
  A Brave New World did it easier, simply sort each person into a genetically pre-determined path prior to birth. No reason to go through all that messy data collection and analysis.
- zozbot234 6y ago
  Yup, this is what "structural" social prejudice and discrimination are all about, once you strip away all the pointless and meaningless rhetoric that's somehow supposed to be "about" these issues. It's a self-perpetuating equilibrium of basic social conditions and superstructure (viz. discourse, supporting ideas, commonly-held worldviews etc.) that create a nearly unescapable "trap" of invisible oppression.
- lmkg 6y ago
  Read Weapons of Math Destruction by Cathy O'Neil. The book explores several ways that's already happening. Her main premise is that there's a feedback loop in many data-driven policies. You only get success results for the things that you try, and you only try the things you already think are likely to receive. As a result, algorithmic policies tend to reinforce the status quo. Loan risk algorithms will favor people "similar to" those who have paid back loans before, a sample group biased towards people that banks have already loaned to before. As a result, a lot of the factors are biased towards "from a white upper-middle-class suburban background." And recidivism estimators, which are used as jail sentencing guidelines in some places. Screening algorithms for job resumes, and college applications. Algorithms send police to where crimes are reported. Crimes are reported because the police are there to witness them. The area gets designated a high-crime area. Regular people are arrested more often because regular activity is suspicious in a high-crime area, affecting their future prospects. The higher arrest rate is used to justify this. It's a continuous spectrum rather than a single point. But if I were to pick a single "point" where it became a self-fulling prophecy? 1994, due to the widespread passage of three-strikes laws.
jacques_chester 6y ago
> Actually due to competition, if it correlates to risk, all companies will literally be forced to use the magic numbers or go into an adverse selection death spiral. To expand on this: adverse selection is where a consumer has hidden information about the cost they can inflict on a provider. Usually this is talked about in terms of insurance (especially health insurance), but risk is risk and so the principles are the same. My recollection of what economists predict is that there are two stable equilibria that achieve Pareto efficiency. The first is that discrimination (in the general yes/no sense) is completely forbidden and risks are totally pooled. The second is that complete discrimination is possible without limitation. The worst outcomes are all found in attempted compromises. Not only do you have the costs of whatever tradeoff you chose between pooling and discrimination being less efficient than the Pareto points, but you also introduce a great deal of dead weight due to complex regulation and oversight, plus efforts made to evade regulation and oversight. Collectively we are worse off, even if individuals think otherwise. I don't think allowing total discrimination is a viable option in this day and age. Which means banning it wholesale and encouraging the formation of universal risk pools.

Spooky23 6y ago

It’s already happening. Georgia had a anti-opioid program where they could score every pregnant woman in the state for likelihood to become dependent on opioids during pregnancy. Babies born with addiction cost the state about $1M each. There were a few similar programs a few years ago when federal grants were made available. Iirc, they were modeled on the stuff built to identify people vulnerable to becoming extremist terrorist types. Insurance companies have databases of way more lifestyle and other behavior data than people realize. (Everything from sports, politics, to porn and gambling habits — anything for sale) I’m pretty sure Georgia mashed that against Medicaid claim data to build the model. It’s a reason why we all need to watch “pre-existing condition” debates closely. If insurers know that 45 year old divorced father of 3 who moves every 3 months is a smoker, gambler and drinker, they don’t want to write a policy — the guy is a trainwreck with no support system.

harlanji 6y ago
As a healthy homeless former professional working as a grocery bagger now, I can feel management watching me for signs of drug abuse to explain my ratty clothes and spotty hygeine. Nobody can guess why I’m not working a good job but seem very smart and thorough (slander from boss destroyed reputation). I’m glad to have a job and sorry to show up looking trashy, but it’s the best I can do right now. I am just thankful that they don’t have any data or scores for me yet to help them see a drug addict or boozer, because I know that I’d raise a lot of flags there and it’d be enough to color the suspicion. Metrics are scary, I know my current scores will make things harder for years to come, starting with credit having bombed right down to these corporate/insurer metrics whose systems I was in as a $200K/yr professional. Lots of explaining the data to do if I make it back to pro life.
- erikerikson 6y ago
  Glad you're doing what sounds to be healthier. I suspect many think differently here but I don't think the problem lies in the knowledge but in how people with power are currently using the knowledge. Restricting the knowledge is simply or current best mitigation of the current conventions. Another world might use that knowledge to better support you and others who have seen hard times and restore what sounds like a lack of justice and help you find an environment where you could thrive and perhaps even that boss of yours. [edit: so that they could thrive but also have reduced negative impact] In other words, rather than focusing on mitigating risk a sufficiently high quality system could help us maximize our lives. Unfortunately, the probability of something so pro-social being the outcome seems low.

Hoasi 6y ago

> Also, it's always amusing to arrive at an article about data collection and be greeted by this: https://i.imgur.com/d4Z4sdd.png https://i.imgur.com/d4Z4sdd.png Meta amusement: https://i.imgur.com/E5bFeb3.png https://i.imgur.com/E5bFeb3.png

cactus2093 6y ago

The credit system in the US is already a pretty horrific version of this, and nobody really seems to care and it’s been like this for decades. And it’s not even just used for credit anymore, renting an apartment now usually requires a credit check. I had thought credit karma and similar services were at least adding transparency to the industry but was recently in for a rude awakening when trying to get preapproval for our first mortgage. It turns out, the whole idea of a credit score is kind of a lie. Your credit report can be pulled by creditors, and creditors can interpret it in different ways as they see fit. Services like credit karma are just making up their own score based on the report which can be drastically different than what the creditor decides it is. And credit pulled for different uses somehow ends up with different scores, e.g. for a credit card approval your score will often show up higher than for a mortgage. You can check your credit report for free, as required by law, but nobody is required to tell you your credit score. You can also basically just pay to remove many things from your credit report. It’s insane and the fact that this fairly old and low-tech institution has not managed to be regulated successfully or made fair to the average person, bodes very poorly IMO for regulation of privacy and fairness based on future technology.

lotsofpulp 6y ago
> You can check your credit report for free, as required by law, but nobody is required to tell you your credit score. You can also basically just pay to remove many things from your credit report. There is no “the credit score”. Any lender is free to use whatever scoring algorithm they like. And they do. Some lenders may choose to use certain scores for certain things, but underwriting has no obligation to use one specific credit score. Therefore there is no reason for people to care about their credit score, whether it be from FICO or credit karma or whoever. The only thing you should do is make sure information on the credit report is accurate. And what is your source for claiming you can pay to get things removed from credit score? It doesn’t even make sense, as it just shows the status of your lines of credit. You can’t just make a line of credit disappear as no credit reporting agency or financial institution is going to want to commit fraud for any amount of money that an average person might offer.
- 6y ago

trustmeimdrunk 6y ago

last i checked data collection from western-block companies is pretty centralized. Extensive records of your online behavior are being used (hence the money being made in it's collection and sale) by HR dpt, insurance companies, and loan desks to determine whether you qualify for a top tier salary for the same work, affordable coverage, and approval for a loan.

simonebrunozzi 6y ago

It would not be "decentralized". It will still be centralized, however the "master" will not be a government, but a private company.