37 ms·

I read a modest amount of popular "AI safety" text (lesswrong basically) and this is the first time I've heard of Dall-E being crippled for safety reasons. You

I read a modest amount of popular "AI safety" text (lesswrong basically) and this is the first time I've heard of Dall-E being crippled for safety reasons. Your argument overall seems to take "you skeptics keep moving the bar, give me a benchmark I can pass and I'll show you", which seems reasonable on it's face but I don't think actually works. The problem is that while algorithm may be defined by theory and tested by benchmark, the only "definition" we have for general intelligence except "what we can see people doing". If I or anyone had a clear, accepted benchmark for general intelligence, we'd be quite a bit further towards creating it but we're not there. That said, I think one thing that current AIs lack is an understanding of it's own processing and an understanding of the limits of that processing. And there are many levels of this. But I won't promise that if this problem is corrected, I won't look at other things. IDK, achieving AGI isn't like just passing some test, no reason it should be like that.

astrange 4y ago
> I read a modest amount of popular "AI safety" text (lesswrong basically) and this is the first time I've heard of Dall-E being crippled for safety reasons. You can read OpenAI's paper or try using it. They've intentionally not taught it many things; it doesn't know most celebrities' names, or copyrighted characters, and there's several filters before and after the big model to prevent NSFW generations of any type. (Oddly, it does know "Homer Simpson" and "Hatsune Miku".)
- joe_the_user 4y ago
  Ah, OK That sound data sanitizing rather than the AI safety/danger that Bostrom and company worry about. For them, it's "OMG, must limit it's capabilities" rather than "Must keep it from doxing or offending people". The parent I was replying to was kind of ambiguous what sort of crippling they meant (We also should be careful using DALL-E as an accurate measure of what's possible, because OpenAI has intentionally crippled their offering in a number of ways to avoid scaring or offending people, under the rubric of "AI safety".)
  - gojomo 4y ago
    Some of what they've done is described here – https://openai.com/blog/reducing-bias-and-improving-safety-in-dall-e-2/ https://openai.com/blog/reducing-bias-and-improving-safety-i... – & in other statements. I'm not sure they've claimed, or we can be sure, that every one of their training or output limits has been disclosed. It seems one of their techniques is to silently add extra words to some prompts to increase the variety of returned images in certain dimensions, eg: https://twitter.com/rzhang88/status/1549472829304741888 https://twitter.com/rzhang88/status/1549472829304741888 While I can't find the link right now, someone also tweeted examples of asking DALL-E2 for certain historical roles that would have been, by actual history and likely representation in training data, fairly uniform in race/sex – but the new OpenAI 'bias mitigations' generate ahistorical race/gender-varied examples. That's interesting – I'm a big fan of non-traditional castings like 'Hamilton!' etc – but serves to make the AI look ignorant of history, when in fact it has just been design-limited to simulate ignorance.
    - astrange 4y ago
      I don't think it simply adds extra words; I think they run it through a GPT prompt, something like "restate this with generic references to humans replaced with specific ethnicities". Can't tell from the outside of course.
    - gojomo 4y ago
      Here's the link I couldn't find earlier: https://twitter.com/Plinz/status/1554255192316526594 https://twitter.com/Plinz/status/1554255192316526594 Prompts like "a movie still of ancient philosopher Heraclitus", "ancient philosopher Aristotle", and "famous physicists of the 17th century" render largely plausible portraits – except with an added sprinkling of ahistorical, 21st Century "AI safety" race/gender variety. This is less a common-sense failure of the AI than an artifact of its creators' imposed harnesses. And thus I'd suggest that more generally, unless a model's full design/training-data/constraints are fully described, it remains possible that other "failures" could sometimes be side-effects of undisclosed designer choices.
    - joe_the_user 4y ago
gojomo 4y ago
The problem is that even the "what we can see people doing" test is slippery. It tends to start as "what we see all people, even young children doing". But then when an agent surpasses, it changes to "what we see modal [or median or average] people doing". Then when an agent surpasses that, it becomes "what we see the most-skilled-ever people doing". And as agents increasingly surpass that, the focus shifts, or retreats, to even murkier standards. "Sure, but it's just a mechanistic mirror, it doesn't really have [X, Y, Z] inside, even if it's externally indistinguishable from human excellence." So, an originally 'black-box' internally-oblivious test then shifts to a 'white-box' internally-dependent test, just to maintain the same comforting conclusion. And by subtly shifting the grounds-of-evaluation, & definitions, it becomes easier to ignore massive improvements, & disturbing potential impacts. It creates an illusion of "zeno's paradox" where agents are never reaching the (shifting) endpoint, when in fact they're speeding past every robust test that can be devised. That's a really important thing to accurately notice! Also, regarding: "current AIs lack… an understanding of it's own processing and an understanding of the limits of that processing" People only have vague, incomplete understanding of their own reasoning, & limits, too. (Experts in restricted domains may be better – and those domains are often easier to train in an artificial agent.) The mere act of "thinking-about-thinking" can sometimes improve the quality of reasoning in humans – forcing something to be explainable – and it turns out to help in these modern large models too. Asking a GPT-like Large Language Model (LLM) to explain its reasoning step-by-step can improve its answers; prompting it with the info that it is an agent that may make errors, and asking it to state its confidence, also often generates more truthful, properly-qualified output. So are the current LLMs that much behind, or inherently incapable of, human-level self-understanding in this sort of test? It seems an open question to me, and even if current SotA models miss a mark, perhaps 5x or 1000x larger models coming soon will outdo humans on any measurable test of "self-understanding", too.

astrange 4y ago

> I read a modest amount of popular "AI safety" text (lesswrong basically) and this is the first time I've heard of Dall-E being crippled for safety reasons. You can read OpenAI's paper or try using it. They've intentionally not taught it many things; it doesn't know most celebrities' names, or copyrighted characters, and there's several filters before and after the big model to prevent NSFW generations of any type. (Oddly, it does know "Homer Simpson" and "Hatsune Miku".)

joe_the_user 4y ago
Ah, OK That sound data sanitizing rather than the AI safety/danger that Bostrom and company worry about. For them, it's "OMG, must limit it's capabilities" rather than "Must keep it from doxing or offending people". The parent I was replying to was kind of ambiguous what sort of crippling they meant (We also should be careful using DALL-E as an accurate measure of what's possible, because OpenAI has intentionally crippled their offering in a number of ways to avoid scaring or offending people, under the rubric of "AI safety".)
- gojomo 4y ago
  Some of what they've done is described here – https://openai.com/blog/reducing-bias-and-improving-safety-in-dall-e-2/ https://openai.com/blog/reducing-bias-and-improving-safety-i... – & in other statements. I'm not sure they've claimed, or we can be sure, that every one of their training or output limits has been disclosed. It seems one of their techniques is to silently add extra words to some prompts to increase the variety of returned images in certain dimensions, eg: https://twitter.com/rzhang88/status/1549472829304741888 https://twitter.com/rzhang88/status/1549472829304741888 While I can't find the link right now, someone also tweeted examples of asking DALL-E2 for certain historical roles that would have been, by actual history and likely representation in training data, fairly uniform in race/sex – but the new OpenAI 'bias mitigations' generate ahistorical race/gender-varied examples. That's interesting – I'm a big fan of non-traditional castings like 'Hamilton!' etc – but serves to make the AI look ignorant of history, when in fact it has just been design-limited to simulate ignorance.
  - astrange 4y ago
    I don't think it simply adds extra words; I think they run it through a GPT prompt, something like "restate this with generic references to humans replaced with specific ethnicities". Can't tell from the outside of course.
  - gojomo 4y ago
    Here's the link I couldn't find earlier: https://twitter.com/Plinz/status/1554255192316526594 https://twitter.com/Plinz/status/1554255192316526594 Prompts like "a movie still of ancient philosopher Heraclitus", "ancient philosopher Aristotle", and "famous physicists of the 17th century" render largely plausible portraits – except with an added sprinkling of ahistorical, 21st Century "AI safety" race/gender variety. This is less a common-sense failure of the AI than an artifact of its creators' imposed harnesses. And thus I'd suggest that more generally, unless a model's full design/training-data/constraints are fully described, it remains possible that other "failures" could sometimes be side-effects of undisclosed designer choices.
  - joe_the_user 4y ago

gojomo 4y ago

The problem is that even the "what we can see people doing" test is slippery. It tends to start as "what we see all people, even young children doing". But then when an agent surpasses, it changes to "what we see modal [or median or average] people doing". Then when an agent surpasses that, it becomes "what we see the most-skilled-ever people doing". And as agents increasingly surpass that, the focus shifts, or retreats, to even murkier standards. "Sure, but it's just a mechanistic mirror, it doesn't really have [X, Y, Z] inside, even if it's externally indistinguishable from human excellence." So, an originally 'black-box' internally-oblivious test then shifts to a 'white-box' internally-dependent test, just to maintain the same comforting conclusion. And by subtly shifting the grounds-of-evaluation, & definitions, it becomes easier to ignore massive improvements, & disturbing potential impacts. It creates an illusion of "zeno's paradox" where agents are never reaching the (shifting) endpoint, when in fact they're speeding past every robust test that can be devised. That's a really important thing to accurately notice! Also, regarding: "current AIs lack… an understanding of it's own processing and an understanding of the limits of that processing" People only have vague, incomplete understanding of their own reasoning, & limits, too. (Experts in restricted domains may be better – and those domains are often easier to train in an artificial agent.) The mere act of "thinking-about-thinking" can sometimes improve the quality of reasoning in humans – forcing something to be explainable – and it turns out to help in these modern large models too. Asking a GPT-like Large Language Model (LLM) to explain its reasoning step-by-step can improve its answers; prompting it with the info that it is an agent that may make errors, and asking it to state its confidence, also often generates more truthful, properly-qualified output. So are the current LLMs that much behind, or inherently incapable of, human-level self-understanding in this sort of test? It seems an open question to me, and even if current SotA models miss a mark, perhaps 5x or 1000x larger models coming soon will outdo humans on any measurable test of "self-understanding", too.