3 ms·
I find it funny and a bit concerning that if this is true version of the prompt, then in their drive to ensure it produces diverse output (a goal I support), t
by vidarh 3y ago
I find it funny and a bit concerning that if this is true version of the prompt, then in their drive to ensure it produces diverse output (a goal I support), they are giving it a bias that doesn't match reality for anyone (which I definitely don't support).
E.g. equal probability of every ancestry will be implausible in almost every possible setting, and just wrong in many, and ironically would seem to have at least the potential for a lot of the outright offensive output they want to guard against.
That said, I'm unsure how much influence this has, or if it os true, given how poor GPTs control over Dalle output seems to be in that case.
E.g. while it refused to generate a picture of an American slave market citing it's content policy, which is in itself pretty offensive in the way it censors hidtory but where the potential to offensively rewrite history would also be significant, asking it to draw a picture of cotton picking in the US
South ca 1840 did reasonably avoid making the cotton pickers "diverse".
Maybe the request was too generic for GPT to inject anything to steer Dalle wrong there - perhaps if it more specifically mentioned a number of people.
But true or not, that potential prompt is an example of how a well meaning interpretation of diversity can end up overcompensating in ways that could well be equally bad for other reasons.
- itronitron 3y agoCould you be more specific in regards to who 'they' is in your first sentence?
- xg15 3y agoOpenAI? The people who wrote the system prompt?
- 211512a4-82d4 3y ago> While DALL·E 3 aims for accuracy and user customization, inherent challenges arise in achieving desirable default behavior, especially when faced with under-specified prompts. This choice may not precisely align with the demographic makeup of every, or even any, specific culture or geographic region. We anticipate further refining our approach, including through helping users customize how ChatGPT interacts with DALL·E 3, to navigate the nuanced intersection between different authentic representations, user preferences, and inclusiveness This was explicitly called out in the DALLE system card [0] as a choice. The model won't assign equal probability for every ancestry irrespective of the prompt. [0] https://cdn.openai.com/papers/DALL_E_3_System_Card.pdf https://cdn.openai.com/papers/DALL_E_3_System_Card.pdf
- vidarh 3y ago> The model won't assign equal probability for every ancestry irrespective of the prompt. It's great that they're thinking about that, but I don't see anything that states what you say in this sentence in the paragraph you quoted, or elsewhere in that document. Have I missed something? It may very well be true - as I noted, GPT doesn't appear to have particularly good control over what Dalle generates (for this, or, frankly, a whole lot of other things)
- pests 3y agoI think he's saying they said it will follow the prompt? Kind of a double negative there
- 211512a4-82d4 3y agoEmphasis on equal - while a bit academic, you can evaluate this empirically to see that every time it assigns a <Race, Gender, etc.> doesn't have the same probability mass (via the logprobs API setting).
- vidarh 3y agoThis is presuming that ChatGPT's integration with Dalle uses the same API with the same restrictions as the public API. That might well be true, but if so that just makes the prompt above even more curious if genuine.