6 ms·

Some every day scenario, with an impact, * You apply for a loan, it is refused * Your admission, for you or your children, to a school or university is refuse

by seren 9y ago

Some every day scenario, with an impact, * You apply for a loan, it is refused * Your admission, for you or your children, to a school or university is refused because more worthy people have applied * Your job application is refused, All because of a fuzzy social score, that is mined from your contacts, activities on social network, meta data gathered from your smartphone in an opaque way, CCTV cameras, etc I mean if we have a way to rate people, why not use it everywhere ! This is already partially implemented in China, I remember reading an example where with a low social score you have to pay for a deposit, and you don't have to with a high social score. In a way, that sound efficient and meritocratic, but also ruthless if you end up with a low score. edit : add a reference with some every day life examples https://www.wired.com/story/age-of-social-credit/ https://www.wired.com/story/age-of-social-credit/

tudelo 9y ago
Doesn't the vast amount of data we have allow for things like getting a loan without knowing a guy? As for point 2, if someone is more worthy... why is that bad? And why should an employer not use information easily accessible to them to determine the character of a potential employee when they only can have brief correspondence via interviews? I'm not sure if you're talking about future implications or what but it seems like this is the system that we exist in today. I don't think most of these decisions are based on a "fuzzy social score" though. At least not the topics you brought up.
- seren 9y ago
  I am talking about a future that is already being built today, piece by piece. I am well aware we are already judged with multiple bias, in a an opaque way. But this is not systematic, hopefully Employer A can have different bias than Employer B. For me, the dystopic part would be to have a single, automated source of "truth", like the Sesame score, used everywhere. It means that everything becomes easy or difficult because of that score. Suddenly you cannot rent or at higher price, your insurance rate will increase, it is harder to get a job... Optimistically, in theory, it could make people "better", but who decides what "better" or "worthy" means ?
  - tudelo 9y ago
    Hah, I can agree with a single source of truth. A credit score is essentially what you are referring to. I know for sure you cannot rent without a large deposit with a terrible score, and some jobs will not hire you (especially GOVT jobs) when you have large debts and bad credit. Not sure about the insurance rates, but it certainly would not surprise me.
- tree_of_item 9y ago
  > As for point 2, if someone is more worthy... why is that bad? Someone is "more worthy" according to an opaque machine learning algorithm. Your line is exactly the danger: people are going to confuse "the computer says X" with "X is true".
  - tudelo 9y ago
    I agree that it could be frustrating, and maybe someone will not make that distinction. But either way, these are decisions that have to be made. What is your alternative to how the decisions are made? In the end, you are either leaving it to a human decision who is going to make a choice based on some heuristics and prejudice or an algorithm which is going to make a choice based on heuristics and prejudice. I'm not sure if you have a problem with the lack of distinction between a human and an algorithm or the idea of a lack of fairness. The decisions that require something like a machine learning algorithm were probably opaque to begin with. Fairness doesn't seem so different in either case...
    - anfilt 9y ago
      However, there is a differnce. If I dont get a job offer becuase I dont fit a humans biases of what they want. I can go somewhere else with a human that has differnt biases. The problem is with AI is how many googles are there? We will liekly end up with a handful of companies that rate you. Suddenly it does not matter where you apply for a job. If you did one thing that ruins/lowers these ratings you may not get a job. The problem is only a handful of entities decide what good is. If everyone is using these systems and you fail to meet their definition of good person/hire you like may suddenly become difficult. Where as before people would have decide that themselves even if its a shallow cursory judgement. At least people are makeing their own decisions instead of blindy following a number/evaluation with criteria they do not know.
      - tudelo 9y ago
        Well, that's just part of the problem with the connected world. If you do something wrong it doesn't take machine learning or AI to find out, you just have to google it. Someone hiring you for a job will most likely do this anyways... I have had recruiters even call me out on such things. It has nothing to do with an algorithm. It could be problematic if there was only one source of subjective information, especially if any perceived past transgressions were irredeemable. But if they are factual, do you have a problem with that? E.G, you go to hire somebody and find out that it's not recommended by hiring_hueristic_x based predominately on factors X,Y,Z (X being he stole from his past 10 employers, Y being he never held a job for more than 6 months, Z being he commonly posts serious threats about people he doesn't like online). Also, do you have any idea what actually occurs when you make a decision? I don't. I would love to know someone who does. I still follow it.
deleted 9y ago
[deleted]
Aunche 9y ago
Nothing's changed. People have always been unfairly refused loans or denied jobs. Presumably, these metrics wouldn't be used if they weren't better than the status quo.

by seren 9y ago

tudelo 9y ago

Doesn't the vast amount of data we have allow for things like getting a loan without knowing a guy? As for point 2, if someone is more worthy... why is that bad? And why should an employer not use information easily accessible to them to determine the character of a potential employee when they only can have brief correspondence via interviews? I'm not sure if you're talking about future implications or what but it seems like this is the system that we exist in today. I don't think most of these decisions are based on a "fuzzy social score" though. At least not the topics you brought up.

seren 9y ago
I am talking about a future that is already being built today, piece by piece. I am well aware we are already judged with multiple bias, in a an opaque way. But this is not systematic, hopefully Employer A can have different bias than Employer B. For me, the dystopic part would be to have a single, automated source of "truth", like the Sesame score, used everywhere. It means that everything becomes easy or difficult because of that score. Suddenly you cannot rent or at higher price, your insurance rate will increase, it is harder to get a job... Optimistically, in theory, it could make people "better", but who decides what "better" or "worthy" means ?
- tudelo 9y ago
  Hah, I can agree with a single source of truth. A credit score is essentially what you are referring to. I know for sure you cannot rent without a large deposit with a terrible score, and some jobs will not hire you (especially GOVT jobs) when you have large debts and bad credit. Not sure about the insurance rates, but it certainly would not surprise me.
tree_of_item 9y ago
> As for point 2, if someone is more worthy... why is that bad? Someone is "more worthy" according to an opaque machine learning algorithm. Your line is exactly the danger: people are going to confuse "the computer says X" with "X is true".
- tudelo 9y ago
  I agree that it could be frustrating, and maybe someone will not make that distinction. But either way, these are decisions that have to be made. What is your alternative to how the decisions are made? In the end, you are either leaving it to a human decision who is going to make a choice based on some heuristics and prejudice or an algorithm which is going to make a choice based on heuristics and prejudice. I'm not sure if you have a problem with the lack of distinction between a human and an algorithm or the idea of a lack of fairness. The decisions that require something like a machine learning algorithm were probably opaque to begin with. Fairness doesn't seem so different in either case...
  - anfilt 9y ago
    However, there is a differnce. If I dont get a job offer becuase I dont fit a humans biases of what they want. I can go somewhere else with a human that has differnt biases. The problem is with AI is how many googles are there? We will liekly end up with a handful of companies that rate you. Suddenly it does not matter where you apply for a job. If you did one thing that ruins/lowers these ratings you may not get a job. The problem is only a handful of entities decide what good is. If everyone is using these systems and you fail to meet their definition of good person/hire you like may suddenly become difficult. Where as before people would have decide that themselves even if its a shallow cursory judgement. At least people are makeing their own decisions instead of blindy following a number/evaluation with criteria they do not know.
    - tudelo 9y ago
      Well, that's just part of the problem with the connected world. If you do something wrong it doesn't take machine learning or AI to find out, you just have to google it. Someone hiring you for a job will most likely do this anyways... I have had recruiters even call me out on such things. It has nothing to do with an algorithm. It could be problematic if there was only one source of subjective information, especially if any perceived past transgressions were irredeemable. But if they are factual, do you have a problem with that? E.G, you go to hire somebody and find out that it's not recommended by hiring_hueristic_x based predominately on factors X,Y,Z (X being he stole from his past 10 employers, Y being he never held a job for more than 6 months, Z being he commonly posts serious threats about people he doesn't like online). Also, do you have any idea what actually occurs when you make a decision? I don't. I would love to know someone who does. I still follow it.

deleted 9y ago

[deleted]

Aunche 9y ago

Nothing's changed. People have always been unfairly refused loans or denied jobs. Presumably, these metrics wouldn't be used if they weren't better than the status quo.