7 ms·

Haven't loan decisions always been black box? Sometimes they'll attribute decisions to credit scores, but those scoring systems have always been proprietary, an

• gopher_space 4y ago
  > Haven't loan decisions always been black box? Not to the people making those decisions. That's brand new.
  • dogma1138 4y ago
    People haven’t been making loan decisions for decades in the EU as well. You put numbers into a computer and what comes out is either an approval with a rate or a decline. No bank EU or not has people manually approving loans.
    • ClumsyPilot 4y ago
      You can file a lawsuit 'this bank does not give me loan because I am not white' and check. You can inspect the excell formula in the computer. It does not matter who personally made that decisi9n, the bank is operating this code and paid someone to write it. With 'AI' there is no way to know.
      • dogma1138 4y ago
        No, what you are entitled in the EU is to have the decision reviewed by a human, I fail to see how an AI changes that.
        • ClumsyPilot 4y ago
          No what? Let's imagine my bank is sexist, the code literally says "if WOMAN, Reject' You can sue my bank for discrimination, and in process of discovery you can request information and documents from the bank, including the code / formulas. If you have a solid argument, let's say my bank never approved a mortgage for a woman, the judge will grant your request. You, or someone appointed on your behalf, will look at the code / excel formula, see that it's discriminatory by design, and you can request massive punitive fines, because this demonstrates contempt for the law. Now let's say I instruct my developers to create a neural network that is as convoluted as possible but is unlikely to approves mortgage for a woman. That can be easily done. There will be no proof that we ever did this on purpose, and even proving that it is discriminatory will be challenging. At best, my bank will get a slap on the wrist. That is why they are banning AI from some areas of life.
          • dogma1138 4y ago
            That has nothing to do with entitlements… That isn’t how it would work in reality. It would be up to the bank to prove that their practices are non discriminatory if there will be a discrepancy between the number of applications issued. The plaintiffs will be able to sue and win based on the outcome alone. As far as actual rights go then you have the right to appeal any automated decision and have a human review it without any bias. Which again it doesn’t matter if it’s AI or a an algorithm based on flipping a coin.
    • bigDinosaur 4y ago
      'Black box' doesn't mean 'numbers in a computer', it means 'nobody can explain to me precisely why I was rejected'. A score calculated from an obvious formula is fine (regulations permitting) since you (or a regulator or court or the institution) can establish precisely why you were rejected. A black box system is one in which this cannot be established by any person. This is obviously bad for both legal and moral reasons. Legally you may be (unintentionally) violating anti-discrimination laws. Morally you may be setting up a system that arbitrarily denies people who are otherwise deserving with little to no recourse.
      • sokoloff 4y ago
        By that definition, hiring and dating appear to be black boxes.
        • bigDinosaur 4y ago
          In both of those you can receive plausible explanations from anyone honest or introspective enough. If you think that all decisions are actually black box emotions that can't be introspected or something then yes, they would be total black boxes, and I suppose the goal would be to minimise that in any system we build that has to discriminate.