4 ms·

Some of what they've done is described here – https://openai.com/blog/reducing-bias-and-improving-safety-in-dall-e-2/ https://openai.com/blog/reducing-bias-and-

by gojomo 4y ago

Some of what they've done is described here – https://openai.com/blog/reducing-bias-and-improving-safety-in-dall-e-2/ https://openai.com/blog/reducing-bias-and-improving-safety-i... – & in other statements. I'm not sure they've claimed, or we can be sure, that every one of their training or output limits has been disclosed. It seems one of their techniques is to silently add extra words to some prompts to increase the variety of returned images in certain dimensions, eg: https://twitter.com/rzhang88/status/1549472829304741888 https://twitter.com/rzhang88/status/1549472829304741888 While I can't find the link right now, someone also tweeted examples of asking DALL-E2 for certain historical roles that would have been, by actual history and likely representation in training data, fairly uniform in race/sex – but the new OpenAI 'bias mitigations' generate ahistorical race/gender-varied examples. That's interesting – I'm a big fan of non-traditional castings like 'Hamilton!' etc – but serves to make the AI look ignorant of history, when in fact it has just been design-limited to simulate ignorance.

astrange 4y ago
I don't think it simply adds extra words; I think they run it through a GPT prompt, something like "restate this with generic references to humans replaced with specific ethnicities". Can't tell from the outside of course.
gojomo 4y ago
Here's the link I couldn't find earlier: https://twitter.com/Plinz/status/1554255192316526594 https://twitter.com/Plinz/status/1554255192316526594 Prompts like "a movie still of ancient philosopher Heraclitus", "ancient philosopher Aristotle", and "famous physicists of the 17th century" render largely plausible portraits – except with an added sprinkling of ahistorical, 21st Century "AI safety" race/gender variety. This is less a common-sense failure of the AI than an artifact of its creators' imposed harnesses. And thus I'd suggest that more generally, unless a model's full design/training-data/constraints are fully described, it remains possible that other "failures" could sometimes be side-effects of undisclosed designer choices.
joe_the_user 4y ago
OK, I looked at your link. It talks about safety but all of measures described are about avoiding bias. I'm not personally against this - having more diverse images is great. But I think my point that the Lesswrong version of AI safety does not seem to be being considered stands. The various anti-bias stuff seems unlikely to be reducing the system's capabilities and I think your original point was that we could not judge Dall-E 'cause of safety-reduced capacities and if this is your evidence, it seems unconvincing to say the least.

by gojomo 4y ago

astrange 4y ago

I don't think it simply adds extra words; I think they run it through a GPT prompt, something like "restate this with generic references to humans replaced with specific ethnicities". Can't tell from the outside of course.

gojomo 4y ago

Here's the link I couldn't find earlier: https://twitter.com/Plinz/status/1554255192316526594 https://twitter.com/Plinz/status/1554255192316526594 Prompts like "a movie still of ancient philosopher Heraclitus", "ancient philosopher Aristotle", and "famous physicists of the 17th century" render largely plausible portraits – except with an added sprinkling of ahistorical, 21st Century "AI safety" race/gender variety. This is less a common-sense failure of the AI than an artifact of its creators' imposed harnesses. And thus I'd suggest that more generally, unless a model's full design/training-data/constraints are fully described, it remains possible that other "failures" could sometimes be side-effects of undisclosed designer choices.

joe_the_user 4y ago

OK, I looked at your link. It talks about safety but all of measures described are about avoiding bias. I'm not personally against this - having more diverse images is great. But I think my point that the Lesswrong version of AI safety does not seem to be being considered stands. The various anti-bias stuff seems unlikely to be reducing the system's capabilities and I think your original point was that we could not judge Dall-E 'cause of safety-reduced capacities and if this is your evidence, it seems unconvincing to say the least.