3 ms·

Where does Google Vision Cloud sit in your categorization? https://algorithmwatch.org/en/google-vision-racism/ https://algorithmwatch.org/en/google-vision-raci

by 3gg 5y ago

Where does Google Vision Cloud sit in your categorization? https://algorithmwatch.org/en/google-vision-racism/ https://algorithmwatch.org/en/google-vision-racism/

foobiekr 5y ago
First group. You’re observing that they aren’t doing a perfect job, which is true, but my grouping isn’t related to perfection of results.
- 3gg 5y ago
  > The deepmind team etc type of group who actually know what they’re doing and the boundaries of what they are working with. You claim that they "know what they are doing and the boundaries of what they are working with" -- and yet they recklessly make public a racist vision product?
  - foobiekr 5y ago
    Your argument is that knowing what you are doing means error free output.
    - 3gg 5y ago
      It's more like applying the technology with caution and accountability when you already know beforehand that the output is not error-free.
      - username90 5y ago
        They never promised that the output would be error free, having output with errors is still useful for many applications. And the issues you are talking about got fixed as soon as it was discovered and since then Google has made sure to always diversify their datasets by race. Nowadays that is common knowledge that you need to do it, but back then it wasn't obvious that a model wouldn't generalize across human races and it is much thanks to that mistake that everyone now knows it is an issue.
        3gg 5y ago
        It was discovered by others, not them; they fixed the issue only retroactively when it was called out in public. This lack of oversight is part of what I mean with applying things with caution. And why would they have assumed in the first place that the model _would_ generalize across human races, or any other factor for that matter?
  - spacedcowboy 5y ago
    I have a PhD in neural networks, haven't used it in many a year, but some of the knowledge is still there. Some of the memories of racking my brains to understand what the hell is going on are still there, too. It is easy to have a theory of what is going on, to model the processes of how things are playing out inside the system, to make external predictions of the system, and to be utterly wrong. Not because your model is wrong, but because either the boundary conditions were unexpected, or there was an anti-pattern in the data, or because the underlying assumptions of the model were violated by the data (in my case, this happened once when all the data was taken in the Southern Hemisphere...) In all these cases, you can know what you're doing, you can know the boundaries of what what you're working with, and you can get results that surprise you. It's called "research" for a reason. The model can also be ridiculously complex. Some of the equations I was dealing with took several lines to write down, and then only because I was substituting in other, complicated expressions to reduce the apparent complexity. It's easy to make mistakes - and so you can know what you're doing, and the boundaries that you're working with, and still have a mistake in the model that leads to a mistake in the data ... garbage in, garbage out. In short, this shit is hard, yo!
    - SamoyedFurFluff 5y ago
      Forgive me because I myself do not have a PhD in ML, but if this is hard (making a non-racist system) why are there not serious guardrails you prevent releasing racist systems to the public?
      - spacedcowboy 5y ago
        Is it your contention that Google intentionally devised a racist system and imposed it on the public ? That would be quite the claim. If instead it was a fuckup, well that seems adequately covered by “this shit is hard”. If you are instead complaining about a lack of oversight, I don’t have a horse in that race. Ask someone else, I don’t care about the politics, I’m here for the technology.
deleted 5y ago
[deleted]

by 3gg 5y ago

Where does Google Vision Cloud sit in your categorization? https://algorithmwatch.org/en/google-vision-racism/ https://algorithmwatch.org/en/google-vision-racism/

foobiekr 5y ago

First group. You’re observing that they aren’t doing a perfect job, which is true, but my grouping isn’t related to perfection of results.

3gg 5y ago
> The deepmind team etc type of group who actually know what they’re doing and the boundaries of what they are working with. You claim that they "know what they are doing and the boundaries of what they are working with" -- and yet they recklessly make public a racist vision product?
- foobiekr 5y ago
  Your argument is that knowing what you are doing means error free output.
  - 3gg 5y ago
    It's more like applying the technology with caution and accountability when you already know beforehand that the output is not error-free.
    - username90 5y ago
      They never promised that the output would be error free, having output with errors is still useful for many applications. And the issues you are talking about got fixed as soon as it was discovered and since then Google has made sure to always diversify their datasets by race. Nowadays that is common knowledge that you need to do it, but back then it wasn't obvious that a model wouldn't generalize across human races and it is much thanks to that mistake that everyone now knows it is an issue.
      - 3gg 5y ago
        It was discovered by others, not them; they fixed the issue only retroactively when it was called out in public. This lack of oversight is part of what I mean with applying things with caution. And why would they have assumed in the first place that the model _would_ generalize across human races, or any other factor for that matter?
- spacedcowboy 5y ago
  I have a PhD in neural networks, haven't used it in many a year, but some of the knowledge is still there. Some of the memories of racking my brains to understand what the hell is going on are still there, too. It is easy to have a theory of what is going on, to model the processes of how things are playing out inside the system, to make external predictions of the system, and to be utterly wrong. Not because your model is wrong, but because either the boundary conditions were unexpected, or there was an anti-pattern in the data, or because the underlying assumptions of the model were violated by the data (in my case, this happened once when all the data was taken in the Southern Hemisphere...) In all these cases, you can know what you're doing, you can know the boundaries of what what you're working with, and you can get results that surprise you. It's called "research" for a reason. The model can also be ridiculously complex. Some of the equations I was dealing with took several lines to write down, and then only because I was substituting in other, complicated expressions to reduce the apparent complexity. It's easy to make mistakes - and so you can know what you're doing, and the boundaries that you're working with, and still have a mistake in the model that leads to a mistake in the data ... garbage in, garbage out. In short, this shit is hard, yo!
  - SamoyedFurFluff 5y ago
    Forgive me because I myself do not have a PhD in ML, but if this is hard (making a non-racist system) why are there not serious guardrails you prevent releasing racist systems to the public?
    - spacedcowboy 5y ago
      Is it your contention that Google intentionally devised a racist system and imposed it on the public ? That would be quite the claim. If instead it was a fuckup, well that seems adequately covered by “this shit is hard”. If you are instead complaining about a lack of oversight, I don’t have a horse in that race. Ask someone else, I don’t care about the politics, I’m here for the technology.

deleted 5y ago

[deleted]