6 ms·

I strongly suspect Google tried really, really hard here to overcome the criticism is got with previous image recognition models saying that black people looked

by dekhn 3y ago

I strongly suspect Google tried really, really hard here to overcome the criticism is got with previous image recognition models saying that black people looked like gorillas. I am not really sure what I would want out of an image generation system, but I think Google's system probably went too far in trying to incorporate diversity in image generation.

michaelt 3y ago
As well as that, I suspect the major AI companies are fearful of generating images of real people - presumably not wanting to be involved with people generating fake images of "Donald Trump rescuing wildfire victims" or "Donald Trump fighting cops". Their efforts to add diversity would have been a lot more subtle if, when you asked for images of "British Politician" the images were recognisably Rishi Sunak, Liz Truss, Kwasi Kwarteng, Boris Johnson, Theresa May, and Tony Blair. That would provide diversity while also being firmly grounded in reality. The current attempts at being diverse and simultaneously trying not to resemble any real person seems to produce some wild results.
- deleted 3y ago
  [deleted]
- deleted 3y ago
  [deleted]
- sho_hn 3y ago
  My takeaway from all of this is that alignment tech is currently quite primitive and relies on very heavy-handed band-aids.
  - slowmovintarget 3y ago
    I think that's a bit overly charitable. Would it not be reasonable to also draw the conclusion that notion of alignment itself is flawed?
- _heimdall 3y ago
  We're honestly just seeing generative algorithms failing at diversity initiatives as badly as humans for. Forcing diversity into a system is an extremely tough, if not impossible, challenge. Initiatives have to be driven my goals and metrics, meaning we have to boil diversity down to a specific list of quantitative metrics. Things will always be missed when our best tool to tackle a moral or noble goal is to boil a complex spectrum of qualitative data to a subset of measurable numbers.
photoGrant 3y ago
Remind yourself we're discussing censorship, misinformation, inability to define or source truth and we're concerned on Day 1 about the results of image gen being controlled by a for profit single entity with incentives that focus solely on business and not humanity... Where do we go from here? Things will magically get better on their own? Businesses will align with humanity and morals, not their investors? This is the tip of the iceberg of concerns and it's ignored as a bug in the code not a problem with trusting private companies with defining truth.
- chasd00 3y ago
  > Where do we go from here? opensource models and training sets. So basically the "secret sauce" minus the hardware. I don't see it happening voluntarily.
  - photoGrant 3y ago
    Absolutely it won't. We've armed the issue with a supersonic jet engine and we're assuming if we build a slingshot out of pop sticks we'll somehow catch up and knock it off course.
    - chasd00 3y ago
      i can't predict the future but there is precedent. Models, weights, and dataasets are the keys to the kingdom like operating system kernels, databases, and libraries use to be. At some point, enough people decided to re-invent and release these things or functionality to all that it became a self-sustaining community and, eventually, transformative to daily life. On the other hand, there may be enough people in power who came from and understand that community to make sure it never happens again.
      - photoGrant 3y ago
        No, compute is keys to the kingdom. The rest are assets and ammunition. You out-compute your enemy, you out-compute your competition. That's the race. The data is part of the problem, not the root. These companies are silo'ing the worlds resources. GPU, Finance, Information. Those combined are the weapon. You make your competition starve in the dust. These companies are pure evil pushing an agenda of pure evil. OpenAI is closed. Google is Google. We're like, ok, there you go! Take it all. No accountability, no transparency, we trust you.
redox99 3y ago
I remember checking like a year ago and they still had the word "gorilla" blacklisted (i.e. it never returns anything even if you have gorilla images).
- _heimdall 3y ago
  Gotta love such a high quality fix. When your upper high tech, state of the art algorithm learns racist patterns just blocklist the word and move on. Don't worry about why it learned such patterns in the first place.
  - AlecSchueler 3y ago
    [flagged]
    - alpaca128 3y ago
      It's not unavoidable, but it would cost more to produce high quality training data.
      - AlecSchueler 3y ago
        Yes, somewhat unavoidable.
    - _heimdall 3y ago
      Do we have enough info for to say that decisively? Ideally we would see the training data, though its probably reasonable to assume a random collection of internet content includes racist imagery. My understanding, though, is that the algorithm and the model of data learned is still a black box that people can't parse and understand. How would we know for sure racist output is due to the racist input, rather than a side effect of some part of the training or querying algorithms?
  - nitwit005 3y ago
    Humans do look like gorillas. We're related. It's natural that an imperfect program that deals with images will will mistake the two. Humans, unfortunately, are offended if you imply they look like gorillas. What's a good fix? Human sensitivity is arbitrary, so the fix is going to tend to be arbitrary too.
DebtDeflation 3y ago
Surely there is a middle ground. "Generate a scene of a group of friends enjoying lunch in the park." -> Totally expect racial and gender diversity in the output. "Generate a scene of 17th century kings of Scotland playing golf." -> The result should not be a bunch of black men and Asian women dressed up as Scottish kings, it should be a bunch of white guys.
- ceejayoz 3y ago
  You can see how this gets challenging, though, right? If you train your model to prioritize real photos (as they're often more accurate representations than artistic ones), you might wind up with Denzel Washington as the archetype; https://en.wikipedia.org/wiki/The_Tragedy_of_Macbeth_(2021_film) https://en.wikipedia.org/wiki/The_Tragedy_of_Macbeth_(2021_f.... There's a vast gap between human understanding and what LLMs "understand".
  - transitionnel 3y ago
    If they actually want it to work as intelligently as possible, they'll begin taking these complaints into consideration and building in a wisdom curating feature where people can contribute. This much is obvious, but they seem to be satisfied with theory over practicality. Anyway I'm just ranting b/c they haven't paid me. How about an off the wall algorithm to estimate how much each scraped input turns out to influence the bigger picture, as a way to work towards satisfying the copyright question.
    - sho_hn 3y ago
      An LLM-style system designed to understand Wikipedia relevance and citation criteria and apply them might be a start. Not that Wikipedia is perfect and controversy-free, but it's certainly a more sophisticated approach than the current system prompts.
    - photoGrant 3y ago
      Then who in this black box private company is the Oracle of infinite wisdom and truth!? Who are you putting in charge? Can I get a vote?
int_19h 3y ago
Judging by the way it words some of the responses to those queries, they "fixed" it by forcibly injecting something like "diverse image showcasing a variety of ethnicities and genders" in all prompts that are classified as "people".
raxxorraxor 3y ago
They have now added a strong bias for generating black people now. Some have prompted to generate a picture of a German WW2 soldier, and now there are many pictures of black people floating around in NAZI uniforms. I think their strategy to "enhance" outcomes is very misdirected. The most widely used base models to really fine tune models are those that are not censored and I think you have to construct a problem to find one here. Of course AI won't generate a perfect world, but this is something that will probably only get better with time when users are able to adapt models to their liking.
- slowmovintarget 3y ago
  > ...when users are able to adapt models to their liking. Therein lies the rub, as it were, because the large providers of AI models are working hard to ensure legislation that wouldn't allow people access to uncensored models in the name of "safety." And "safety" in this case includes the notion that models may not push the "correct" world-view enough.

by dekhn 3y ago

michaelt 3y ago

As well as that, I suspect the major AI companies are fearful of generating images of real people - presumably not wanting to be involved with people generating fake images of "Donald Trump rescuing wildfire victims" or "Donald Trump fighting cops". Their efforts to add diversity would have been a lot more subtle if, when you asked for images of "British Politician" the images were recognisably Rishi Sunak, Liz Truss, Kwasi Kwarteng, Boris Johnson, Theresa May, and Tony Blair. That would provide diversity while also being firmly grounded in reality. The current attempts at being diverse and simultaneously trying not to resemble any real person seems to produce some wild results.

deleted 3y ago
[deleted]
deleted 3y ago
[deleted]
sho_hn 3y ago
My takeaway from all of this is that alignment tech is currently quite primitive and relies on very heavy-handed band-aids.
- slowmovintarget 3y ago
  I think that's a bit overly charitable. Would it not be reasonable to also draw the conclusion that notion of alignment itself is flawed?
_heimdall 3y ago
We're honestly just seeing generative algorithms failing at diversity initiatives as badly as humans for. Forcing diversity into a system is an extremely tough, if not impossible, challenge. Initiatives have to be driven my goals and metrics, meaning we have to boil diversity down to a specific list of quantitative metrics. Things will always be missed when our best tool to tackle a moral or noble goal is to boil a complex spectrum of qualitative data to a subset of measurable numbers.

photoGrant 3y ago

Remind yourself we're discussing censorship, misinformation, inability to define or source truth and we're concerned on Day 1 about the results of image gen being controlled by a for profit single entity with incentives that focus solely on business and not humanity... Where do we go from here? Things will magically get better on their own? Businesses will align with humanity and morals, not their investors? This is the tip of the iceberg of concerns and it's ignored as a bug in the code not a problem with trusting private companies with defining truth.

chasd00 3y ago
> Where do we go from here? opensource models and training sets. So basically the "secret sauce" minus the hardware. I don't see it happening voluntarily.
- photoGrant 3y ago
  Absolutely it won't. We've armed the issue with a supersonic jet engine and we're assuming if we build a slingshot out of pop sticks we'll somehow catch up and knock it off course.
  - chasd00 3y ago
    i can't predict the future but there is precedent. Models, weights, and dataasets are the keys to the kingdom like operating system kernels, databases, and libraries use to be. At some point, enough people decided to re-invent and release these things or functionality to all that it became a self-sustaining community and, eventually, transformative to daily life. On the other hand, there may be enough people in power who came from and understand that community to make sure it never happens again.
    - photoGrant 3y ago
      No, compute is keys to the kingdom. The rest are assets and ammunition. You out-compute your enemy, you out-compute your competition. That's the race. The data is part of the problem, not the root. These companies are silo'ing the worlds resources. GPU, Finance, Information. Those combined are the weapon. You make your competition starve in the dust. These companies are pure evil pushing an agenda of pure evil. OpenAI is closed. Google is Google. We're like, ok, there you go! Take it all. No accountability, no transparency, we trust you.

redox99 3y ago

I remember checking like a year ago and they still had the word "gorilla" blacklisted (i.e. it never returns anything even if you have gorilla images).

_heimdall 3y ago
Gotta love such a high quality fix. When your upper high tech, state of the art algorithm learns racist patterns just blocklist the word and move on. Don't worry about why it learned such patterns in the first place.
- AlecSchueler 3y ago
  [flagged]
  - alpaca128 3y ago
    It's not unavoidable, but it would cost more to produce high quality training data.
    - AlecSchueler 3y ago
      Yes, somewhat unavoidable.
  - _heimdall 3y ago
    Do we have enough info for to say that decisively? Ideally we would see the training data, though its probably reasonable to assume a random collection of internet content includes racist imagery. My understanding, though, is that the algorithm and the model of data learned is still a black box that people can't parse and understand. How would we know for sure racist output is due to the racist input, rather than a side effect of some part of the training or querying algorithms?
- nitwit005 3y ago
  Humans do look like gorillas. We're related. It's natural that an imperfect program that deals with images will will mistake the two. Humans, unfortunately, are offended if you imply they look like gorillas. What's a good fix? Human sensitivity is arbitrary, so the fix is going to tend to be arbitrary too.

DebtDeflation 3y ago

Surely there is a middle ground. "Generate a scene of a group of friends enjoying lunch in the park." -> Totally expect racial and gender diversity in the output. "Generate a scene of 17th century kings of Scotland playing golf." -> The result should not be a bunch of black men and Asian women dressed up as Scottish kings, it should be a bunch of white guys.

ceejayoz 3y ago
You can see how this gets challenging, though, right? If you train your model to prioritize real photos (as they're often more accurate representations than artistic ones), you might wind up with Denzel Washington as the archetype; https://en.wikipedia.org/wiki/The_Tragedy_of_Macbeth_(2021_film) https://en.wikipedia.org/wiki/The_Tragedy_of_Macbeth_(2021_f.... There's a vast gap between human understanding and what LLMs "understand".
- transitionnel 3y ago
  If they actually want it to work as intelligently as possible, they'll begin taking these complaints into consideration and building in a wisdom curating feature where people can contribute. This much is obvious, but they seem to be satisfied with theory over practicality. Anyway I'm just ranting b/c they haven't paid me. How about an off the wall algorithm to estimate how much each scraped input turns out to influence the bigger picture, as a way to work towards satisfying the copyright question.
  - sho_hn 3y ago
    An LLM-style system designed to understand Wikipedia relevance and citation criteria and apply them might be a start. Not that Wikipedia is perfect and controversy-free, but it's certainly a more sophisticated approach than the current system prompts.
  - photoGrant 3y ago
    Then who in this black box private company is the Oracle of infinite wisdom and truth!? Who are you putting in charge? Can I get a vote?

int_19h 3y ago

Judging by the way it words some of the responses to those queries, they "fixed" it by forcibly injecting something like "diverse image showcasing a variety of ethnicities and genders" in all prompts that are classified as "people".

raxxorraxor 3y ago

They have now added a strong bias for generating black people now. Some have prompted to generate a picture of a German WW2 soldier, and now there are many pictures of black people floating around in NAZI uniforms. I think their strategy to "enhance" outcomes is very misdirected. The most widely used base models to really fine tune models are those that are not censored and I think you have to construct a problem to find one here. Of course AI won't generate a perfect world, but this is something that will probably only get better with time when users are able to adapt models to their liking.

slowmovintarget 3y ago
> ...when users are able to adapt models to their liking. Therein lies the rub, as it were, because the large providers of AI models are working hard to ensure legislation that wouldn't allow people access to uncensored models in the name of "safety." And "safety" in this case includes the notion that models may not push the "correct" world-view enough.