5 ms·

People tend to think of this as horrible discriminating decisions made by some horrible people, but most likely it is a very cheap machine developed by an inter

People tend to think of this as horrible discriminating decisions made by some horrible people, but most likely it is a very cheap machine developed by an intern who had 3 days to make something work...

sidpatil 7y ago
Sounds like Hanlon's razor.
gshdg 7y ago
If that's the case, then the decision to assign it to an intern and give them 3 days, and then not to test it with a diverse set of subjects was somebody's fault and failure. If they were not an intern (or even if they were, frankly), the engineer who built it also had a responsibility to think beyond their own demographic; and if the timeline was too short to develop a properly robust system, to at least point that out to management.
- gregoriol 7y ago
  As a side point, it could also be like: would someone pay 100$ for a picture in such a machine? This is sad, but that's how you get fastfoods too
  - StanislavPetrov 7y ago
    The better question is, why replace a simple photo booth that has worked flawlessly for decades for a few quarters with a machine that requires a massive investment to work properly and do the same thing? It reminds me very much of "smart toasters" and other gadgets that are extremely expensive and arguably don't work nearly as well and aren't nearly as reliable as the old appliances they are supposed to replace.
  - micheljansen 7y ago
    This is why we need regulation and can’t leave this kind of stuff up to the market. If it costs too much to make a machine that is trained on not just a set of white faces so it is not discriminatory, that machine should not be allowed to exist. The only alternative is that if you happen to be part of the privileged majority, you can get a cheap picture from an automated booth, whereas everyone else will have no choice but to pay for a more expensive alternative. This will work to amplify any differences that are already ingrained in society (that probably led to the homogenous training set to begin with).
  - bingtop 7y ago
    Passport photos have been issued for over a century without needing this crap. Yes, it meant more rejections, and it meant inspectors allowing photos that had minor rule violations, but convenience isn’t the paramount goal of everything.
bingtop 7y ago
That’s much worse. The number of people out there who will do this deliberately is much smaller than the number of people who will do it accidentally, but the harms are the same. We need to avoid carelessly applying technology to areas where carelessness can harm people severely.
rolltiide 7y ago
It doesn’t matter how it’s rationalized, it wouldn’t have shipped with a more observant set of people. Nobody consciously thinks infrared faucets and hand dryers discriminate, it doesn’t matter about how “hard” contrast on darker people must be for a machine, its just that the edge case when their sensors start failing should have not been considered an edge case at all and never shipped. Nobody should be in a position of playing devils advocate to dismiss minority experiences with technology explaining why companies are tone deaf, it just shouldn't have shipped.
Waterluvian 7y ago
Trivially fixable even in that fairly inexcusable case: add an override button. "by overriding you accept that this photo may ultimately be rejected for passport use."
- Crosseye_Jack 7y ago
  There is an option to override, it’s just not visible in the picture. The booth is asking the user to “touch an option” but only 1 option is visible, the option to retake the photo has a “attempts remaining” counter and that the company logo is in the middle of the photo preview (the photo preview is in the middle of two options, which will make the company logo centred on the screen).
djaque 7y ago
No they don't. This is without a doubt a product of badly designed software. That badly designed software is making life difficult for people based on assumptions about race and gender. It's a problem which is worth fixing and deserves attention.

sidpatil 7y ago

Sounds like Hanlon's razor.

gshdg 7y ago

If that's the case, then the decision to assign it to an intern and give them 3 days, and then not to test it with a diverse set of subjects was somebody's fault and failure. If they were not an intern (or even if they were, frankly), the engineer who built it also had a responsibility to think beyond their own demographic; and if the timeline was too short to develop a properly robust system, to at least point that out to management.

gregoriol 7y ago
As a side point, it could also be like: would someone pay 100$ for a picture in such a machine? This is sad, but that's how you get fastfoods too
- StanislavPetrov 7y ago
  The better question is, why replace a simple photo booth that has worked flawlessly for decades for a few quarters with a machine that requires a massive investment to work properly and do the same thing? It reminds me very much of "smart toasters" and other gadgets that are extremely expensive and arguably don't work nearly as well and aren't nearly as reliable as the old appliances they are supposed to replace.
- micheljansen 7y ago
  This is why we need regulation and can’t leave this kind of stuff up to the market. If it costs too much to make a machine that is trained on not just a set of white faces so it is not discriminatory, that machine should not be allowed to exist. The only alternative is that if you happen to be part of the privileged majority, you can get a cheap picture from an automated booth, whereas everyone else will have no choice but to pay for a more expensive alternative. This will work to amplify any differences that are already ingrained in society (that probably led to the homogenous training set to begin with).
- bingtop 7y ago
  Passport photos have been issued for over a century without needing this crap. Yes, it meant more rejections, and it meant inspectors allowing photos that had minor rule violations, but convenience isn’t the paramount goal of everything.

bingtop 7y ago

That’s much worse. The number of people out there who will do this deliberately is much smaller than the number of people who will do it accidentally, but the harms are the same. We need to avoid carelessly applying technology to areas where carelessness can harm people severely.

rolltiide 7y ago

It doesn’t matter how it’s rationalized, it wouldn’t have shipped with a more observant set of people. Nobody consciously thinks infrared faucets and hand dryers discriminate, it doesn’t matter about how “hard” contrast on darker people must be for a machine, its just that the edge case when their sensors start failing should have not been considered an edge case at all and never shipped. Nobody should be in a position of playing devils advocate to dismiss minority experiences with technology explaining why companies are tone deaf, it just shouldn't have shipped.

Waterluvian 7y ago

Trivially fixable even in that fairly inexcusable case: add an override button. "by overriding you accept that this photo may ultimately be rejected for passport use."

Crosseye_Jack 7y ago
There is an option to override, it’s just not visible in the picture. The booth is asking the user to “touch an option” but only 1 option is visible, the option to retake the photo has a “attempts remaining” counter and that the company logo is in the middle of the photo preview (the photo preview is in the middle of two options, which will make the company logo centred on the screen).

djaque 7y ago

No they don't. This is without a doubt a product of badly designed software. That badly designed software is making life difficult for people based on assumptions about race and gender. It's a problem which is worth fixing and deserves attention.