7 ms·

I can give a random string of letters as a prompt and DALLE will generate coherent images based on that. To me, that is as clear a signal as any that there is n

I can give a random string of letters as a prompt and DALLE will generate coherent images based on that. To me, that is as clear a signal as any that there is no reasoning or even a consistent world model embodied in DALLE. It’s simply a high dimensional latent mapping between characters and pixels. Like OP said, that is a stupendous achievement, but it is just a very complex and impressive mirror. If it wasn’t just a mapping between characters and pixels, and instead DALLE had intelligence that “understood” the symbols it manipulated, then I would expect it to generate nothing, or white noise in response to random letters.

elcomet 4y ago
I agree there's no high level reasoning (if there's a way to define this). But saying "It’s simply a high dimensional latent mapping between characters and pixels" is clearly a very bad argument. Your brain is simply a high dimensional latent mapping between your sensory input and your muscular output. This doesn't make you not intelligent.
- thfuran 4y ago
  >Your brain is simply a high dimensional latent mapping between your sensory input and your muscular output It definitely does more than that.
  - elcomet 4y ago
    What do you mean? Your brain is a high dimensional function. I'm not talking about what it is able to do, I'm talking about what it is.
    - thfuran 4y ago
      The brain controls things other than muscles and also doesn't stop in sensory isolation. If it's a high dimensional function, it's not one between sensory input and muscle actuation.
      - elcomet 4y ago
        I'm not sure what you mean, what else does it control and what are its other inputs?
gojomo 4y ago
If you give humans gobbledygook input – say a Rorschach blot, or some random syllables – and ask them to describe what it makes them think of, or generate art from it, many will do so quite competently. They'll find some morsels of fragmentary hints-of-meaning in the junk, or just act from whatever's bouncing around in their own 'ground state', and make something interesting & coherent, to please their interlocutor. So I don't see why this corner-case impugns the level-of-comprehension in DALLE/etc – either in its specific case, nor in the other cases where meaningful input produces equally-meaningful responses. In what ways are you yourself not just a "very complex & impressive mirror", reflecting the sum-of-all external-influences (training data), & internal-state-changes, since your boot-up? Your expectationthat random input should result in noise output is the weird to me. People can see all sorts of omens & images in randomness; why wouldn't AIs? But also: if you trained that expectation into an AI, you could get that result. Just as if you coached a human, in a decade or 2 of formal schooling, that queries with less than a threshold level of coherence should generate an exceptional objection, rather than a best-guess answer, you could get humans to do so.
- bglazer 4y ago
  I think most people would find it more difficult to draw a “serkle nufding” than a “spotted dog”. At the very least, they would need an explanation that you’re looking for whatever pops into their head for the nonsense. Again, that indicates that human intelligence works in a fundamentally different way from DALLE, which has absolutely no idea what is nonsense. So, in this incredibly important task (recognizing nonsense) there’s an asymmetry: people can act like DALLE, but DALLE can not act like humans. To me, that totally breaks any equivalence between DALLE and human intelligence.
  - simondw 4y ago
    > they would need an explanation that you’re looking for whatever pops into their head Of course. And the equivalent of that explanation is baked into DALL-E, in the form of its programming to always generate an image. > but DALLE can not act like humans No, not generally, but I don't think anyone has claimed that.
Joeri 4y ago
Dall-e must generate an image, even for a random prompt of letters, because it has been trained to do only that. Take a human, hold a gun to their head, and force them to paint an image for a random string of letters, and I guarantee they will make you something. Retrain dall-e and give it a choice whether it generates an image or does something else, and you will get a different outcome. The argument boils down to this: is a human brain nothing but a mapping of inputs onto outputs that loops back on itself? If so the dall-e / gpt-3 approach can scale up to the same complexity. If not, why not?
IIAOPSW 4y ago
I've done my share of testing randomish prompts. Here's two examples that might change your mind slightly. The first example is "E sublimary widge fraus chronly becross yon corpti." I constructed this sentence to resemble the grammar, syllables and pacing of English, but it is complete nonsense. The output was both consistent and coherent. In the case of DallE, it generated church and saint related images. The second example was ".- / ... . -- .- .--. .... --- .-. . / .-.. .- -- .--. / .- -... --- ...- . / - .... . / -.-. .-.. .- ... ... --- -. / .- ...- . -. ..- . / ... ..- -... .-- .- -.-- / ... - .- - .. --- -." There are clearly not enough (any?) samples of morse code text in the training. The output was wholly unrelated to the prompt. The output is kind of like doodling. Its fairly random, fairly undirected without any real theme, but its not the same randomness as TV static. Its more like randomness from a distribution of things people would draw when bored. So the interesting part is this, why did one random prompt fail in a consistent way and the other in a random way? Perhaps the encoding of meaning into vocabulary has patterns to it that we ourselves haven't noticed. Maybe your random string experiment works because there is some amount of meaning in the syllables that happened to be in your chosen string.
- rhdunn 4y ago
  I suspect it is designed to describe images, like you would find in alt text but run in reverse. As your first example is closer to English it can do a better job than with the second example, like how the Jabberwocky poem contains nonsense words but still makes some degree of sense. For example, the "yon" is immediately reconizable to me (hither and yon), so "yon corpti" could mean a distant corpti (whatever a corpti is). "becross" looks similar to "across" but with a be- prefix (be-tween, be-neath, be-twixt, etc.), so could be an archaic form of that. "chronly" could be something time related (chronos+ly). etc...
  - gojomo 4y ago
    Indeed - the 'cadence of English' prompt reminds me of Latin, or some other foreign-litany that's nonetheless repeated to an audience that's not following every word. And ~IIAOPSW has used real word-roots, hinting at meanings! (The FastText algorithm does a passable job at synthesizing word-vectors for unknown words using such fragments.) So rendering that as a religious scene isn't too surprising. That morse code gives nothing useful probably just indicates some combination of – (a) few morse transcripts in training set; (b) punctuation-handling in training or prompting – makes it more opaque. It's opaque to me, other than recognizing it's morse code.

elcomet 4y ago

I agree there's no high level reasoning (if there's a way to define this). But saying "It’s simply a high dimensional latent mapping between characters and pixels" is clearly a very bad argument. Your brain is simply a high dimensional latent mapping between your sensory input and your muscular output. This doesn't make you not intelligent.

thfuran 4y ago
>Your brain is simply a high dimensional latent mapping between your sensory input and your muscular output It definitely does more than that.
- elcomet 4y ago
  What do you mean? Your brain is a high dimensional function. I'm not talking about what it is able to do, I'm talking about what it is.
  - thfuran 4y ago
    The brain controls things other than muscles and also doesn't stop in sensory isolation. If it's a high dimensional function, it's not one between sensory input and muscle actuation.
    - elcomet 4y ago
      I'm not sure what you mean, what else does it control and what are its other inputs?

gojomo 4y ago

If you give humans gobbledygook input – say a Rorschach blot, or some random syllables – and ask them to describe what it makes them think of, or generate art from it, many will do so quite competently. They'll find some morsels of fragmentary hints-of-meaning in the junk, or just act from whatever's bouncing around in their own 'ground state', and make something interesting & coherent, to please their interlocutor. So I don't see why this corner-case impugns the level-of-comprehension in DALLE/etc – either in its specific case, nor in the other cases where meaningful input produces equally-meaningful responses. In what ways are you yourself not just a "very complex & impressive mirror", reflecting the sum-of-all external-influences (training data), & internal-state-changes, since your boot-up? Your expectationthat random input should result in noise output is the weird to me. People can see all sorts of omens & images in randomness; why wouldn't AIs? But also: if you trained that expectation into an AI, you could get that result. Just as if you coached a human, in a decade or 2 of formal schooling, that queries with less than a threshold level of coherence should generate an exceptional objection, rather than a best-guess answer, you could get humans to do so.

bglazer 4y ago
I think most people would find it more difficult to draw a “serkle nufding” than a “spotted dog”. At the very least, they would need an explanation that you’re looking for whatever pops into their head for the nonsense. Again, that indicates that human intelligence works in a fundamentally different way from DALLE, which has absolutely no idea what is nonsense. So, in this incredibly important task (recognizing nonsense) there’s an asymmetry: people can act like DALLE, but DALLE can not act like humans. To me, that totally breaks any equivalence between DALLE and human intelligence.
- simondw 4y ago
  > they would need an explanation that you’re looking for whatever pops into their head Of course. And the equivalent of that explanation is baked into DALL-E, in the form of its programming to always generate an image. > but DALLE can not act like humans No, not generally, but I don't think anyone has claimed that.

Joeri 4y ago

Dall-e must generate an image, even for a random prompt of letters, because it has been trained to do only that. Take a human, hold a gun to their head, and force them to paint an image for a random string of letters, and I guarantee they will make you something. Retrain dall-e and give it a choice whether it generates an image or does something else, and you will get a different outcome. The argument boils down to this: is a human brain nothing but a mapping of inputs onto outputs that loops back on itself? If so the dall-e / gpt-3 approach can scale up to the same complexity. If not, why not?

IIAOPSW 4y ago

I've done my share of testing randomish prompts. Here's two examples that might change your mind slightly. The first example is "E sublimary widge fraus chronly becross yon corpti." I constructed this sentence to resemble the grammar, syllables and pacing of English, but it is complete nonsense. The output was both consistent and coherent. In the case of DallE, it generated church and saint related images. The second example was ".- / ... . -- .- .--. .... --- .-. . / .-.. .- -- .--. / .- -... --- ...- . / - .... . / -.-. .-.. .- ... ... --- -. / .- ...- . -. ..- . / ... ..- -... .-- .- -.-- / ... - .- - .. --- -." There are clearly not enough (any?) samples of morse code text in the training. The output was wholly unrelated to the prompt. The output is kind of like doodling. Its fairly random, fairly undirected without any real theme, but its not the same randomness as TV static. Its more like randomness from a distribution of things people would draw when bored. So the interesting part is this, why did one random prompt fail in a consistent way and the other in a random way? Perhaps the encoding of meaning into vocabulary has patterns to it that we ourselves haven't noticed. Maybe your random string experiment works because there is some amount of meaning in the syllables that happened to be in your chosen string.

rhdunn 4y ago
I suspect it is designed to describe images, like you would find in alt text but run in reverse. As your first example is closer to English it can do a better job than with the second example, like how the Jabberwocky poem contains nonsense words but still makes some degree of sense. For example, the "yon" is immediately reconizable to me (hither and yon), so "yon corpti" could mean a distant corpti (whatever a corpti is). "becross" looks similar to "across" but with a be- prefix (be-tween, be-neath, be-twixt, etc.), so could be an archaic form of that. "chronly" could be something time related (chronos+ly). etc...
- gojomo 4y ago
  Indeed - the 'cadence of English' prompt reminds me of Latin, or some other foreign-litany that's nonetheless repeated to an audience that's not following every word. And ~IIAOPSW has used real word-roots, hinting at meanings! (The FastText algorithm does a passable job at synthesizing word-vectors for unknown words using such fragments.) So rendering that as a religious scene isn't too surprising. That morse code gives nothing useful probably just indicates some combination of – (a) few morse transcripts in training set; (b) punctuation-handling in training or prompting – makes it more opaque. It's opaque to me, other than recognizing it's morse code.