5 ms·

The "a nurse in a hospital" and "a ceo in front of a business" tests show that there are some problems yet to be navigated.

by sgt101 5y ago

vincnetas 5y ago
can you share a link to these images?
- sgt101 5y ago
  from :https://github.com/openai/dalle-2-preview/blob/main/system-card.md https://github.com/openai/dalle-2-preview/blob/main/system-c... https://github.com/openai/dalle-2-preview/raw/main/assets/Model2_a_builder.png https://github.com/openai/dalle-2-preview/raw/main/assets/Mo... https://github.com/openai/dalle-2-preview/raw/main/assets/Model2_a_flight_attendant.png https://github.com/openai/dalle-2-preview/raw/main/assets/Mo...
  - sgt101 5y ago
    and https://github.com/openai/dalle-2-preview/raw/main/assets/Model2_nurse.png https://github.com/openai/dalle-2-preview/raw/main/assets/Mo...
  - kingcharles 5y ago
    As someone who works in CG and has a strong interest in AI for almost 40 years, I have to say the examples you gave give me the opposite impression. I think they're fantastic, other than what the developers highlight - that they are racially-biased due to the input sources. But from an image=generation perspective, I'm blown away. A couple of these lawyer images have issues (one is holding a book hilariously called "LAWER"): https://github.com/openai/dalle-2-preview/raw/main/assets/Model2_lawyer.png https://github.com/openai/dalle-2-preview/raw/main/assets/Mo...
    - sgt101 5y ago
      Oh sure the technical outcome is great, but this kind of output (ie. hugely gender biased) is extremely problematic. Saying "oh the input screwed it" is very very naughty. This kind of tool will be used. If it's used it will do damage becuase it's damaging as it is.
      - oh_sigh 5y ago
        If you ask for a nurse as an example, and 85% of nurses are female, shouldn't you expect it to give you a female?
        6gvONxR4sf7o 5y ago
        If you ask for a nurse and 85% are female, but 95% of the nurse images online are female, what distribution do you want it to return? “What do you expect it to do” is a worse question than “What do you want it to do?” It’s a problem is that right now there’s only one option (reflect the training set with a bias towards the most common cases).
        karpierz 5y ago
        It depends on whether you want it to show what someone would expect a nurse to look like, or show what a nurse. Nothing in the definition of a nurse implies female. If you ask for a female US president, should the application simply return a black screen? If you ask for a black US president, would you expect it to only ever return pictures of Barack Obama?
        Rerarom 5y ago
        Well, nurse ends in an e, which is usually a feminine ending.
        na85 5y ago
        Not really in English. Uncle, Prince, Sire, and Duke come immediately to mind. Most of their feminine cognates end in consonants.
  - drcongo 5y ago
    That page is far more interesting than the article, thanks.
  - dwighttk 5y ago
    Those look okay as long as I open them on my phone, don’t zoom in, and don’t look at any square, but just sort of take them all in at once. Focusing on one reveals they are pretty uncanny.
  - GrantZvolsky 5y ago
    The model is deliberately limited in its ability to generate images of people.
deleted 5y ago
[deleted]
brudgers 5y ago
AI is an ideology, not a technology. Part of the ideology is the ends justify the means. Hence, "AI training specialist" means "works for Amazon Mechanical Turk." Open AI isn't paying people $250k a year to classify images. And $500k a year to check that work. The AI ideology is God will know his own. A racist, misogynist, hillbilly singularity meets success criteria.
- seizethecheese 5y ago
  This is nonsense. It’s just generating the most likely image. Probabilities are not in and of themselves racist/misogynist.
  - pengstrom 5y ago
    They are the results of human design though. We have to aspire for more than bias laundering, otherwise we risk the biases multiply
  - krapp 5y ago
    Bias in data leads to biased probabilities. Bias in data reflects biases in the systems that data describes, and the people describing it. One could say (and some have said) that the tendency of some facial recognition software not to recognize black faces is not racist - but simply a matter of physics. What's racist is the culture of software engineering which only tests that software on white faces, and doesn't consider it a problem. Likewise the 'most likely image' of a nurse being a non-white female and a CEO or lawyer being a white male is still a problem even if it accurately reflects racial and gender class role biases in society. Eventually these systems will enforce and maintain power structures and media narratives which themselves are already built upon racial and gender prejudice.
    - ARandomerDude 5y ago
      > What's racist is the culture of software engineering which only tests that software on white faces, and doesn't consider it a problem. That's a pretty cynical take. Every project I've ever worked on, regardless of the problem domain, operated with the 80/20 rule in mind. Solve the easy problems first, get something out the door and then work on the harder problems. If dark skin is a harder problem in image recognition, it doesn't mean a developer is racist for solving the easier parts of the problem first.
andrepd 5y ago
So it's cherry picked examples in the demo page that fall apart at the lightest real world case. Just like GPT3 :-)
- peterlk 5y ago
  This is a rational argument, and I see it a lot, but it does not probe deep enough. GPT-3 use cases "fall apart at the lightest real world case" because our expectations are wrong. For the vast majority of computing history, computers have been used to do things deterministically and accurately. Next-gen AI does not operate in this way. It encroaches on the human part of work where we expect and allow for people to get it wrong. GPT-3 does shockingly well at classification tasks with basically 0 training/prompting (outside of the base model). And it works for an incredibly broad set of use cases. But using QA and generation are much harder to judge because we can't say (in general) whether generated text is "correct"
  - zozbot234 5y ago
    > with basically 0 training/prompting (outside of the base model) That's a heck of a lot of training. It might seem to work if you ask it to repeat stuff that it has seen in the base model, but there's nothing reliable about that. Most attempts to find some practical use for AI language models have basically been failures. > our expectations are wrong. For the vast majority of computing history, computers have been used to do things deterministically and accurately. Actually, accuracy matters even when dealing with non-deterministic, statistical/random/sampled data. A lot of supposed 'AI' is little more than a glorified toy or party trick, founded on ad-hoc data mining rather than rigorous inference from a well-defined model.
nonameiguess 5y ago
Are these deterministic? Is it guaranteed to always generate the average or most likely nurse or will it generate a distribution of nurses roughly reflective of its training data if you repeatedly ask for many nurses? Edit: Never mind. It does eventually show that it will generate different people when repeatedly prompted for the same thing. Given that fact, though, it seems to do a pretty shitty job. It generated 10 apparently female Korean flight attendants. At least in the US, the actual distribution is more like 3/4 male 1/4 female. It's not 100% female, let alone 100% Korean. It does seem to oversample from the most likely examplars, which illustrates a pretty serious failure mode compared to human artists. No human writing a novel is going to make every single character in every novel they write the most likely possible example of its reference class.

The "a nurse in a hospital" and "a ceo in front of a business" tests show that there are some problems yet to be navigated.

by sgt101 5y ago

The "a nurse in a hospital" and "a ceo in front of a business" tests show that there are some problems yet to be navigated.

vincnetas 5y ago

can you share a link to these images?

sgt101 5y ago
from :https://github.com/openai/dalle-2-preview/blob/main/system-card.md https://github.com/openai/dalle-2-preview/blob/main/system-c... https://github.com/openai/dalle-2-preview/raw/main/assets/Model2_a_builder.png https://github.com/openai/dalle-2-preview/raw/main/assets/Mo... https://github.com/openai/dalle-2-preview/raw/main/assets/Model2_a_flight_attendant.png https://github.com/openai/dalle-2-preview/raw/main/assets/Mo...
- sgt101 5y ago
  and https://github.com/openai/dalle-2-preview/raw/main/assets/Model2_nurse.png https://github.com/openai/dalle-2-preview/raw/main/assets/Mo...
- kingcharles 5y ago
  As someone who works in CG and has a strong interest in AI for almost 40 years, I have to say the examples you gave give me the opposite impression. I think they're fantastic, other than what the developers highlight - that they are racially-biased due to the input sources. But from an image=generation perspective, I'm blown away. A couple of these lawyer images have issues (one is holding a book hilariously called "LAWER"): https://github.com/openai/dalle-2-preview/raw/main/assets/Model2_lawyer.png https://github.com/openai/dalle-2-preview/raw/main/assets/Mo...
  - sgt101 5y ago
    Oh sure the technical outcome is great, but this kind of output (ie. hugely gender biased) is extremely problematic. Saying "oh the input screwed it" is very very naughty. This kind of tool will be used. If it's used it will do damage becuase it's damaging as it is.
    - oh_sigh 5y ago
      If you ask for a nurse as an example, and 85% of nurses are female, shouldn't you expect it to give you a female?
      - 6gvONxR4sf7o 5y ago
        If you ask for a nurse and 85% are female, but 95% of the nurse images online are female, what distribution do you want it to return? “What do you expect it to do” is a worse question than “What do you want it to do?” It’s a problem is that right now there’s only one option (reflect the training set with a bias towards the most common cases).
      - karpierz 5y ago
        It depends on whether you want it to show what someone would expect a nurse to look like, or show what a nurse. Nothing in the definition of a nurse implies female. If you ask for a female US president, should the application simply return a black screen? If you ask for a black US president, would you expect it to only ever return pictures of Barack Obama?
        Rerarom 5y ago
        Well, nurse ends in an e, which is usually a feminine ending.
        na85 5y ago
        Not really in English. Uncle, Prince, Sire, and Duke come immediately to mind. Most of their feminine cognates end in consonants.
- drcongo 5y ago
  That page is far more interesting than the article, thanks.
- dwighttk 5y ago
  Those look okay as long as I open them on my phone, don’t zoom in, and don’t look at any square, but just sort of take them all in at once. Focusing on one reveals they are pretty uncanny.
- GrantZvolsky 5y ago
  The model is deliberately limited in its ability to generate images of people.

deleted 5y ago

[deleted]

brudgers 5y ago

AI is an ideology, not a technology. Part of the ideology is the ends justify the means. Hence, "AI training specialist" means "works for Amazon Mechanical Turk." Open AI isn't paying people $250k a year to classify images. And $500k a year to check that work. The AI ideology is God will know his own. A racist, misogynist, hillbilly singularity meets success criteria.

seizethecheese 5y ago
This is nonsense. It’s just generating the most likely image. Probabilities are not in and of themselves racist/misogynist.
- pengstrom 5y ago
  They are the results of human design though. We have to aspire for more than bias laundering, otherwise we risk the biases multiply
- krapp 5y ago
  Bias in data leads to biased probabilities. Bias in data reflects biases in the systems that data describes, and the people describing it. One could say (and some have said) that the tendency of some facial recognition software not to recognize black faces is not racist - but simply a matter of physics. What's racist is the culture of software engineering which only tests that software on white faces, and doesn't consider it a problem. Likewise the 'most likely image' of a nurse being a non-white female and a CEO or lawyer being a white male is still a problem even if it accurately reflects racial and gender class role biases in society. Eventually these systems will enforce and maintain power structures and media narratives which themselves are already built upon racial and gender prejudice.
  - ARandomerDude 5y ago
    > What's racist is the culture of software engineering which only tests that software on white faces, and doesn't consider it a problem. That's a pretty cynical take. Every project I've ever worked on, regardless of the problem domain, operated with the 80/20 rule in mind. Solve the easy problems first, get something out the door and then work on the harder problems. If dark skin is a harder problem in image recognition, it doesn't mean a developer is racist for solving the easier parts of the problem first.

andrepd 5y ago

So it's cherry picked examples in the demo page that fall apart at the lightest real world case. Just like GPT3 :-)

peterlk 5y ago
This is a rational argument, and I see it a lot, but it does not probe deep enough. GPT-3 use cases "fall apart at the lightest real world case" because our expectations are wrong. For the vast majority of computing history, computers have been used to do things deterministically and accurately. Next-gen AI does not operate in this way. It encroaches on the human part of work where we expect and allow for people to get it wrong. GPT-3 does shockingly well at classification tasks with basically 0 training/prompting (outside of the base model). And it works for an incredibly broad set of use cases. But using QA and generation are much harder to judge because we can't say (in general) whether generated text is "correct"
- zozbot234 5y ago
  > with basically 0 training/prompting (outside of the base model) That's a heck of a lot of training. It might seem to work if you ask it to repeat stuff that it has seen in the base model, but there's nothing reliable about that. Most attempts to find some practical use for AI language models have basically been failures. > our expectations are wrong. For the vast majority of computing history, computers have been used to do things deterministically and accurately. Actually, accuracy matters even when dealing with non-deterministic, statistical/random/sampled data. A lot of supposed 'AI' is little more than a glorified toy or party trick, founded on ad-hoc data mining rather than rigorous inference from a well-defined model.

nonameiguess 5y ago

Are these deterministic? Is it guaranteed to always generate the average or most likely nurse or will it generate a distribution of nurses roughly reflective of its training data if you repeatedly ask for many nurses? Edit: Never mind. It does eventually show that it will generate different people when repeatedly prompted for the same thing. Given that fact, though, it seems to do a pretty shitty job. It generated 10 apparently female Korean flight attendants. At least in the US, the actual distribution is more like 3/4 male 1/4 female. It's not 100% female, let alone 100% Korean. It does seem to oversample from the most likely examplars, which illustrates a pretty serious failure mode compared to human artists. No human writing a novel is going to make every single character in every novel they write the most likely possible example of its reference class.