4 ms·

I've done my share of testing randomish prompts. Here's two examples that might change your mind slightly. The first example is "E sublimary widge fraus chronly

I've done my share of testing randomish prompts. Here's two examples that might change your mind slightly. The first example is "E sublimary widge fraus chronly becross yon corpti." I constructed this sentence to resemble the grammar, syllables and pacing of English, but it is complete nonsense. The output was both consistent and coherent. In the case of DallE, it generated church and saint related images. The second example was ".- / ... . -- .- .--. .... --- .-. . / .-.. .- -- .--. / .- -... --- ...- . / - .... . / -.-. .-.. .- ... ... --- -. / .- ...- . -. ..- . / ... ..- -... .-- .- -.-- / ... - .- - .. --- -." There are clearly not enough (any?) samples of morse code text in the training. The output was wholly unrelated to the prompt. The output is kind of like doodling. Its fairly random, fairly undirected without any real theme, but its not the same randomness as TV static. Its more like randomness from a distribution of things people would draw when bored. So the interesting part is this, why did one random prompt fail in a consistent way and the other in a random way? Perhaps the encoding of meaning into vocabulary has patterns to it that we ourselves haven't noticed. Maybe your random string experiment works because there is some amount of meaning in the syllables that happened to be in your chosen string.

rhdunn 4y ago
I suspect it is designed to describe images, like you would find in alt text but run in reverse. As your first example is closer to English it can do a better job than with the second example, like how the Jabberwocky poem contains nonsense words but still makes some degree of sense. For example, the "yon" is immediately reconizable to me (hither and yon), so "yon corpti" could mean a distant corpti (whatever a corpti is). "becross" looks similar to "across" but with a be- prefix (be-tween, be-neath, be-twixt, etc.), so could be an archaic form of that. "chronly" could be something time related (chronos+ly). etc...
- gojomo 4y ago
  Indeed - the 'cadence of English' prompt reminds me of Latin, or some other foreign-litany that's nonetheless repeated to an audience that's not following every word. And ~IIAOPSW has used real word-roots, hinting at meanings! (The FastText algorithm does a passable job at synthesizing word-vectors for unknown words using such fragments.) So rendering that as a religious scene isn't too surprising. That morse code gives nothing useful probably just indicates some combination of – (a) few morse transcripts in training set; (b) punctuation-handling in training or prompting – makes it more opaque. It's opaque to me, other than recognizing it's morse code.
  - IIAOPSW 4y ago
    I accidentally omitted a sentence. I tested it on DallE mini as well. It also behaved consistently there, but that version gave me mostly Beetles with a few bike races mixed in. I also played slightly with the nonsense words to see if perhaps something I picked was too close to a real word. The revised prompt gave me doctors on DallE mini and letter tiles / notebook pages in DallE, with the mini-version producing two noteworthy images that exhibit the "name one thing in this photo" effect. https://i.dailymail.co.uk/1s/2019/04/25/14/12707994-6959547-image-m-36_1556198258556.jpg https://i.dailymail.co.uk/1s/2019/04/25/14/12707994-6959547-...
    - gojomo 4y ago
      I'm not particularly surprised that official DALL-E & unofficial DallE-Mini went different directions with the same input, as they're different-sized models with different training sets. Below a certain level of complexity, perhaps only "vaguely like beetle or bicycle/race names" arbitrarily survived all the other competing influences on the model's internal weights. Above, much more subtle patterns – like many more latinate word roots – might start to survive. Compare also what I linked in another thread branch – https://twitter.com/gojomo/status/1540095089615089665 https://twitter.com/gojomo/status/1540095089615089665 – where simply growing the size of a similar (non-public Google PARTI) model leads to a phase-change in its ability to render meaningful human text.
semi-extrinsic 4y ago
Your first random prompt is far from random. It contains the fragments "sublim", "chr", "cross" and "corpt" in addition to the isolated "E", which all project the solution down towards Latin and Christianity. You can easily see that these language models are in some sense working on fragments as much as they are on the actual words isolated by spaces in your sentence. Just take a test sentence and enter as a prompt to get some images. Then take that same sentence, remove all spaces and add new spaces in random locations, making gibberish words. You will see that the results will retain quite a few elements from the original prompt, while other things (predominantly monosyllables) become lost. To me, I have not seen a single example that cannot just be explained by saying this is all just linear algebra, with a mind-bogglingly huge and nasty set of operators that has some randomness in it and that projects from the vector space of sentences written in ASCII letters onto a small subset of the vector space of 1024x1024x24bit images. If you then think about doing this just in the "stupid way", imagine you have an input vector that is 4096 bytes long (in some sense the character limit of DALL-E 2) and an output vector that is 3 million bytes long. A single dense matrix representing one such mapping has 6 billion parameters - but you want something very sparse here, since you know that the output is very sparse in the possible output vector space. So let's say you have a sparsity factor of somewhere around 10^5. Then with the 3.5 billion parameters of DALL-E 2, you can "afford" somewhere around 10^5 such matrices. Of course you can apply these matrices successively. Is it then so far fetched to believe that if you thought of those 10^5 matrices as a basis set for your transformation, with a separate ordering vector to say which matrices to apply in what order, and you then spent a huge amount of computing power running an optimizer to get a very good basis set and a very good dictionary of ordering vectors, based on a large corpus of images with caption, that you would not get something comparably impressive as DALL-E 2? When people are wowed that you can change the style of the image by saying "oil painting" or "impressionist", what more is that than one more of the basis set matrices being tacked on in the ordering vector?
- IIAOPSW 4y ago
  We don't disagree at all about what DallE does or what I would expect to see on certain outputs. What I'm saying is maybe that output is actually correct and we just don't know it yet. Maybe the brain really does encode some amount of information on just the sound fragments. Maybe vocabulary words aren't wholly arbitrary signifiers but instead come from some deeper system our brain uses to encode things. There is already a known human version of consistent output given nonsense words. Its called Bouba and Kiki. https://en.wikipedia.org/wiki/Bouba/kiki_effect https://en.wikipedia.org/wiki/Bouba/kiki_effect There's other evidence in support of this idea, but it would be a hassle for me to dig it up now. >Your first random prompt is far from random. It contains the fragments "sublim", "chr", "cross" and "corpt" in addition to the isolated "E", which all project the solution down towards Latin and Christianity. Oh right, I omitted a sentence. DallE gave me churches. DallE-mini gave me bicycle racing and beetles. Both of them behaved self-consistently, but were not consistent with each other. I did test changing some of the words that seemed like they might be having the steering effects like you pointed out. The new phrase was "E sublumary widge fraus chronly non estoi". DallE mini gave me mostly doctors/medical procedures with a few indiscernible scenes mixed in. Real DallE gave me scrabble, word tiles, pages from a book. Its like this. If you talk to a dog with the "who's a good boy" voice, the dog will understand "who's a good boy", even if you're actually saying "who's a little asshole". Likewise, a pronounceable string will generally feel like it belongs within a certain envelope of meanings, even if its not actually a phrase in any real spoken language. I showed some maps that DallE generated which happened to have some places labelled with its usual gibberish. People said the place names felt vaguely like they might be in Eastern Europe. Interpretation of nonsense words works both in the direction of human to DallE and DallE to human!

rhdunn 4y ago

I suspect it is designed to describe images, like you would find in alt text but run in reverse. As your first example is closer to English it can do a better job than with the second example, like how the Jabberwocky poem contains nonsense words but still makes some degree of sense. For example, the "yon" is immediately reconizable to me (hither and yon), so "yon corpti" could mean a distant corpti (whatever a corpti is). "becross" looks similar to "across" but with a be- prefix (be-tween, be-neath, be-twixt, etc.), so could be an archaic form of that. "chronly" could be something time related (chronos+ly). etc...

gojomo 4y ago
Indeed - the 'cadence of English' prompt reminds me of Latin, or some other foreign-litany that's nonetheless repeated to an audience that's not following every word. And ~IIAOPSW has used real word-roots, hinting at meanings! (The FastText algorithm does a passable job at synthesizing word-vectors for unknown words using such fragments.) So rendering that as a religious scene isn't too surprising. That morse code gives nothing useful probably just indicates some combination of – (a) few morse transcripts in training set; (b) punctuation-handling in training or prompting – makes it more opaque. It's opaque to me, other than recognizing it's morse code.
- IIAOPSW 4y ago
  I accidentally omitted a sentence. I tested it on DallE mini as well. It also behaved consistently there, but that version gave me mostly Beetles with a few bike races mixed in. I also played slightly with the nonsense words to see if perhaps something I picked was too close to a real word. The revised prompt gave me doctors on DallE mini and letter tiles / notebook pages in DallE, with the mini-version producing two noteworthy images that exhibit the "name one thing in this photo" effect. https://i.dailymail.co.uk/1s/2019/04/25/14/12707994-6959547-image-m-36_1556198258556.jpg https://i.dailymail.co.uk/1s/2019/04/25/14/12707994-6959547-...
  - gojomo 4y ago
    I'm not particularly surprised that official DALL-E & unofficial DallE-Mini went different directions with the same input, as they're different-sized models with different training sets. Below a certain level of complexity, perhaps only "vaguely like beetle or bicycle/race names" arbitrarily survived all the other competing influences on the model's internal weights. Above, much more subtle patterns – like many more latinate word roots – might start to survive. Compare also what I linked in another thread branch – https://twitter.com/gojomo/status/1540095089615089665 https://twitter.com/gojomo/status/1540095089615089665 – where simply growing the size of a similar (non-public Google PARTI) model leads to a phase-change in its ability to render meaningful human text.

semi-extrinsic 4y ago

Your first random prompt is far from random. It contains the fragments "sublim", "chr", "cross" and "corpt" in addition to the isolated "E", which all project the solution down towards Latin and Christianity. You can easily see that these language models are in some sense working on fragments as much as they are on the actual words isolated by spaces in your sentence. Just take a test sentence and enter as a prompt to get some images. Then take that same sentence, remove all spaces and add new spaces in random locations, making gibberish words. You will see that the results will retain quite a few elements from the original prompt, while other things (predominantly monosyllables) become lost. To me, I have not seen a single example that cannot just be explained by saying this is all just linear algebra, with a mind-bogglingly huge and nasty set of operators that has some randomness in it and that projects from the vector space of sentences written in ASCII letters onto a small subset of the vector space of 1024x1024x24bit images. If you then think about doing this just in the "stupid way", imagine you have an input vector that is 4096 bytes long (in some sense the character limit of DALL-E 2) and an output vector that is 3 million bytes long. A single dense matrix representing one such mapping has 6 billion parameters - but you want something very sparse here, since you know that the output is very sparse in the possible output vector space. So let's say you have a sparsity factor of somewhere around 10^5. Then with the 3.5 billion parameters of DALL-E 2, you can "afford" somewhere around 10^5 such matrices. Of course you can apply these matrices successively. Is it then so far fetched to believe that if you thought of those 10^5 matrices as a basis set for your transformation, with a separate ordering vector to say which matrices to apply in what order, and you then spent a huge amount of computing power running an optimizer to get a very good basis set and a very good dictionary of ordering vectors, based on a large corpus of images with caption, that you would not get something comparably impressive as DALL-E 2? When people are wowed that you can change the style of the image by saying "oil painting" or "impressionist", what more is that than one more of the basis set matrices being tacked on in the ordering vector?

IIAOPSW 4y ago
We don't disagree at all about what DallE does or what I would expect to see on certain outputs. What I'm saying is maybe that output is actually correct and we just don't know it yet. Maybe the brain really does encode some amount of information on just the sound fragments. Maybe vocabulary words aren't wholly arbitrary signifiers but instead come from some deeper system our brain uses to encode things. There is already a known human version of consistent output given nonsense words. Its called Bouba and Kiki. https://en.wikipedia.org/wiki/Bouba/kiki_effect https://en.wikipedia.org/wiki/Bouba/kiki_effect There's other evidence in support of this idea, but it would be a hassle for me to dig it up now. >Your first random prompt is far from random. It contains the fragments "sublim", "chr", "cross" and "corpt" in addition to the isolated "E", which all project the solution down towards Latin and Christianity. Oh right, I omitted a sentence. DallE gave me churches. DallE-mini gave me bicycle racing and beetles. Both of them behaved self-consistently, but were not consistent with each other. I did test changing some of the words that seemed like they might be having the steering effects like you pointed out. The new phrase was "E sublumary widge fraus chronly non estoi". DallE mini gave me mostly doctors/medical procedures with a few indiscernible scenes mixed in. Real DallE gave me scrabble, word tiles, pages from a book. Its like this. If you talk to a dog with the "who's a good boy" voice, the dog will understand "who's a good boy", even if you're actually saying "who's a little asshole". Likewise, a pronounceable string will generally feel like it belongs within a certain envelope of meanings, even if its not actually a phrase in any real spoken language. I showed some maps that DallE generated which happened to have some places labelled with its usual gibberish. People said the place names felt vaguely like they might be in Eastern Europe. Interpretation of nonsense words works both in the direction of human to DallE and DallE to human!