4 ms·

And darker coloured textiles are less visible on people with darker skin. Grass is green and bananas tend to be curved. I get that it is interesting to researc

And darker coloured textiles are less visible on people with darker skin. Grass is green and bananas tend to be curved. I get that it is interesting to research optically measuring systems to 'search for bias' and I'm sure there are going to be biases in varying degrees, but that doesn't mean it's always for the same reason or with the same result. There are very real problems of course, take the example with the photographic film development from the article: that is a problem because a picture that is supposed to reflect the subject you photographed no longer does what it was suppsoed to do. And someone here posted the example of the soap dispenser that couldn't "see" people that didn't have light skin. Also a problem, because now you can't get soap. Both of those are mostly examples of errors made during engineering (apply Hanlon's razor when in doubt - it's not always malice like people often assume), and might be preventable if there were more different samples and interactions available during development. That doesn't mean it's the only type of problem we have with devices for humans, or that keeping a color palette of people around is always going to fix it. Take the soap dispenser example: you could have made one that uses radar or ultrasonic detection, solves multiple problems because you now don't have to consider lighting conditions which always vary due to differences in bathrooms (light fixtures, positioning, positioning of the dispenser itself etc). That doesn't relate to skin color, but can just as easily be a similar problem that could have been prevented all the same by doing proper engineering. But it's likely that in cases like this (soap dispensers) the detection system was just a module that some other third party company imports and sells, and they bought it from a module integrator somewhere else and they bought the parts as generic motion detectors from a company 4 layers down the chain that specializes in measuring daytime movements of rabbits or something silly like that. On the other hand: if you have a government that requires certain controls to be in place and applied for medical equipment to be used in a medical setting you would expect some of the requirements to be drawn up in a way that reflects the use cases. It appears that either the devices are not scoped within those controls, or the controls failed. Or it's all good and the difference in measurement is not significant enough to matter for the use case and thus the devices are used as-is and this research doesn't relate to the real world as well as people think.

function_seven 6y ago
> Both of those are mostly examples of errors made during engineering (apply Hanlon's razor when in doubt - it's not always malice like people often assume), and might be preventable if there were more different samples and interactions available during development. When I see these types of examples brought up, I don't see malice being assumed. Rather, these are good illustrations of structural bias. "Structural" (or "systemic") is often used in contrast to overt bigotry or malice. It's sometimes the result of a certain workforce or industry being substantially skewed with respect to the wider population. You're right, Hanlon and his razor do work here to show that it's a naive error rather than intentional harm. But the result is still traceable to the disparities between the designers and the population at large. As for the soap dispenser: If the team of engineers designing the module had a mix of skin colors, then they would have quickly seen that their product failed sometimes. They would, at the outset, have known that they needed to tune the photoreceptor better, or switch to a different method of detection. Though it is funny that Nikon, a Japanese company, made cameras with an blink detector that falsely triggered on Asian eyes. That either refutes my own point above, or it indicates they imported that functionality.
- oneplane 6y ago
  The issue is that we assume that there was a building somewhere, where a group of people sat down and designed and produced a device. (the soap dispenser) But we don't actually know that. If there is a single spot where this was done, and also where the testing was done, and there was an abundance of skin tones available, then this was caught. But that's a lot of ifs. It could be that the device as designed and completed for manufacturing 15 years ago, the manufacturing was shifted around between 2 companies and the supply chain has 10 levels deep of participants that all did their own tests on their part in their own company in their own country on their own continent. And between that and the consumer it turned out there were 4 companies that just trade and don't actually look at, touch or use the product. This is probably not what happened at Nikon, I'd guess that they have a few departments that may not work together as well as they could. Would be interesting to see how this actually came to be, I imagine there are many ways to create a (partially (bad)) product, and equally many ways to try to prevent that.
6y ago

function_seven 6y ago

> Both of those are mostly examples of errors made during engineering (apply Hanlon's razor when in doubt - it's not always malice like people often assume), and might be preventable if there were more different samples and interactions available during development. When I see these types of examples brought up, I don't see malice being assumed. Rather, these are good illustrations of structural bias. "Structural" (or "systemic") is often used in contrast to overt bigotry or malice. It's sometimes the result of a certain workforce or industry being substantially skewed with respect to the wider population. You're right, Hanlon and his razor do work here to show that it's a naive error rather than intentional harm. But the result is still traceable to the disparities between the designers and the population at large. As for the soap dispenser: If the team of engineers designing the module had a mix of skin colors, then they would have quickly seen that their product failed sometimes. They would, at the outset, have known that they needed to tune the photoreceptor better, or switch to a different method of detection. Though it is funny that Nikon, a Japanese company, made cameras with an blink detector that falsely triggered on Asian eyes. That either refutes my own point above, or it indicates they imported that functionality.

oneplane 6y ago
The issue is that we assume that there was a building somewhere, where a group of people sat down and designed and produced a device. (the soap dispenser) But we don't actually know that. If there is a single spot where this was done, and also where the testing was done, and there was an abundance of skin tones available, then this was caught. But that's a lot of ifs. It could be that the device as designed and completed for manufacturing 15 years ago, the manufacturing was shifted around between 2 companies and the supply chain has 10 levels deep of participants that all did their own tests on their part in their own company in their own country on their own continent. And between that and the consumer it turned out there were 4 companies that just trade and don't actually look at, touch or use the product. This is probably not what happened at Nikon, I'd guess that they have a few departments that may not work together as well as they could. Would be interesting to see how this actually came to be, I imagine there are many ways to create a (partially (bad)) product, and equally many ways to try to prevent that.

6y ago