3 ms·

What we really need is the ability to force organizations to expose their models.

by xenihn 7y ago

toomuchtodo 7y ago
To regulators, sure. Entirely reasonable. To your average Joe? No. This is already done to prevent redlining when originating mortgages. Disclaimer: Work in financial services in risk management, interface with regulators. Opinions are my own.
- xenihn 7y ago
  What's your argument for opposing full exposure, other than the potential impact on your personal livelihood?
  - toomuchtodo 7y ago
    The model is not your property. Do you plan on releasing all of your company’s code on Github gratis? Unlikely. As long as the model doesn’t violate the law, you have no right to it, unless you’re a regulator. Regulators and oversight bodies get full (audited and logged) access. And while it’s none of your business, I don’t need my job, or a job at all. I work for projects I enjoy.
    - xenihn 7y ago
      If you're really financially independent, that's even more incentive for you to support exposure and transparency. People are being harmed by these models. You would lose nothing, and others would benefit. >As long as the model doesn’t violate the law, you have no right to it, unless you’re a regulator. If you have no realistic alternative to being judged by a model, you absolutely should have a right to it. Maybe not the complete implementation, but a verified summary that could be understood by the average person.
      - deleted 7y ago
        [deleted]
    - eropple 7y ago
      In the modern economy it’s impossible to not be constantly and pervasively impacted by these data models. I think it’s reasonable to assert that it is in society’s best interest to make sure that the effects and impacts of these models are communicated as perfectly as possible to those who are forced to live under their purview. What way is this done besides viewing the model? (And given the state of regulatory capture, I’m not sure you can rely on regulators as proxies, no?) It’s totally possible that those models could then be “gamed”, sure—but it’s more important that people are able to live than for lenders and the like to make yet another buck, isn’t it?
      - toomuchtodo 7y ago
        Lenders can still realize a reasonable profit while ensuring people are not improperly discriminated against by machine intelligence. Effective regulation is possible. I would even be willing to serve a tour of duty standing up such a regulatory body, perhaps as a division of the CFPB. Throwing up your hands that government is ineffective entirely is unreasonable, just as asking for all models to be public is.
        eropple 7y ago
        Effective regulation is possible, sure. Is effective regulation likely? And is it made more or less likely by transparency? Heck, as I think about this further--wouldn't transparent models help realize those reasonable profits? If everybody's cards are on the table, doesn't that encourage the market to move towards perfect competition?
        toomuchtodo 7y ago
        If you’re open to a conversation about what legislation would look like, I am receptive to collaboration. The best people for public policy are the ones who don’t want the job, but do it because it’s necessary.
resonantjacket5 7y ago
Problem is once a model is exposed it can be easily gamed
- xenihn 7y ago
  You don't need to expose the model to figure out how to game it. It's going to be gamed anyways.
- maxerickson 7y ago
  You can't really fake some of the inputs. Income, number of loans, stuff like that. You can manipulate them of course, by taking out a cheap loan that you don't necessarily need and things like that. Plenty of people use a credit card for a few expenses (say fuel) just to establish a history.

What we really need is the ability to force organizations to expose their models.

by xenihn 7y ago

What we really need is the ability to force organizations to expose their models.

To regulators, sure. Entirely reasonable. To your average Joe? No. This is already done to prevent redlining when originating mortgages. Disclaimer: Work in financial services in risk management, interface with regulators. Opinions are my own.

xenihn 7y ago
What's your argument for opposing full exposure, other than the potential impact on your personal livelihood?
- toomuchtodo 7y ago
  The model is not your property. Do you plan on releasing all of your company’s code on Github gratis? Unlikely. As long as the model doesn’t violate the law, you have no right to it, unless you’re a regulator. Regulators and oversight bodies get full (audited and logged) access. And while it’s none of your business, I don’t need my job, or a job at all. I work for projects I enjoy.
  - xenihn 7y ago
    If you're really financially independent, that's even more incentive for you to support exposure and transparency. People are being harmed by these models. You would lose nothing, and others would benefit. >As long as the model doesn’t violate the law, you have no right to it, unless you’re a regulator. If you have no realistic alternative to being judged by a model, you absolutely should have a right to it. Maybe not the complete implementation, but a verified summary that could be understood by the average person.
    - deleted 7y ago
      [deleted]
  - eropple 7y ago
    In the modern economy it’s impossible to not be constantly and pervasively impacted by these data models. I think it’s reasonable to assert that it is in society’s best interest to make sure that the effects and impacts of these models are communicated as perfectly as possible to those who are forced to live under their purview. What way is this done besides viewing the model? (And given the state of regulatory capture, I’m not sure you can rely on regulators as proxies, no?) It’s totally possible that those models could then be “gamed”, sure—but it’s more important that people are able to live than for lenders and the like to make yet another buck, isn’t it?
    - toomuchtodo 7y ago
      Lenders can still realize a reasonable profit while ensuring people are not improperly discriminated against by machine intelligence. Effective regulation is possible. I would even be willing to serve a tour of duty standing up such a regulatory body, perhaps as a division of the CFPB. Throwing up your hands that government is ineffective entirely is unreasonable, just as asking for all models to be public is.
      - eropple 7y ago
        Effective regulation is possible, sure. Is effective regulation likely? And is it made more or less likely by transparency? Heck, as I think about this further--wouldn't transparent models help realize those reasonable profits? If everybody's cards are on the table, doesn't that encourage the market to move towards perfect competition?
        toomuchtodo 7y ago
        If you’re open to a conversation about what legislation would look like, I am receptive to collaboration. The best people for public policy are the ones who don’t want the job, but do it because it’s necessary.

resonantjacket5 7y ago

Problem is once a model is exposed it can be easily gamed

xenihn 7y ago
You don't need to expose the model to figure out how to game it. It's going to be gamed anyways.
maxerickson 7y ago
You can't really fake some of the inputs. Income, number of loans, stuff like that. You can manipulate them of course, by taking out a cheap loan that you don't necessarily need and things like that. Plenty of people use a credit card for a few expenses (say fuel) just to establish a history.