11 ms·

The most important thing I think DALL-E shows is that it has a model of our world and culture. It's not intelligence, but it is knowledge. Google can give you

by orlp 4y ago

The most important thing I think DALL-E shows is that it has a model of our world and culture. It's not intelligence, but it is knowledge. Google can give you endless pictures of giraffes if you search for it. But it can only connect you to what exists. It doesn't know things, it knows OF things. DALL-E has knowledge of the concept of a giraffe, and can synthesize an endless amount of never-before seen giraffes for you. It actually knows what a giraffe is.

TremendousJudge 4y ago
I find talking about "knowing" is too much anthropomorphizing to my taste. A more accurate description would be that it can synthesize an endless amount of pictures that the viewer will recognize as giraffes (or as anything else the viewer wants). And I agree, it's pretty amazing. But my point is that talking like this skips over the entire useless "what is knowledge" debate.
- soco 4y ago
  Maybe we need to "de-antropomorphize" these terms and rules. It's a long due process, I believe.
  - TremendousJudge 4y ago
    I don't think it's possible. "Knowing" and "understanding" are inherently human experiences. The most reasonable thing to do would be to invent new terms and relegate the older ones, that have too much cultural baggage attached to them. The same way when the creator of humanity turned out to be "evolution by natural selection" we didn't redefine the term "God" to mean that. Eventually we just started using the new term.
  - psyc 4y ago
    If it'll get people to stop accusing other people of anthropomorphizing, I'm 100% on board. Now if you can think of a way to get people to quit asking for definitions of the words intelligence and consciousness, you'll be a legend and maybe these discussions will advance for the first time.
    - TremendousJudge 4y ago
      Well, I tried my best. Another commenter summed it up better without using the term "anthropomorphizing" at all, but it came out a bit more condescending than necessary.
    - IIAOPSW 4y ago
      Hang on while I become a legend. I use GPT3 to generate the usual trite arguments about intelligence and consciousness why computers won't ever get there. Of course I don't actually reveal that a computer is generating my responses until later on. Eventually everyone will become jaded and skeptical that the other participants in that conversation are real people. Soon all arguments about machine intelligence will devolve into accusations of using GPT3 or not. Some day, even mentioning consciousness will just make everyone assume you're probably a GPT3 troll. This kills the conversation in a way that makes a valid point. If the bots can't be reliably identified, the proof is in the pudding and the matter is settled.
- MauranKilom 4y ago
  I too can write a program that will synthesize an endless amount of pictures that the viewer will recognize as a giraffe. They would all be the same picture, just with one bit of noise randomly applied to each pixel - enough to generate more distinct 256x256 pictures of giraffes than there are atoms in the universe. We need to go a bit deeper to tease out what makes DALL-E 2 amazing.
soco 4y ago
A honest question: what is then intelligence? Or are you just trying to avoid being caught into a debate...
- wcarss 4y ago
  you could think of "knowing what a giraffe is" in this instance as a slice of an intelligent thought, but it's not a whole thought, let alone a train of thoughts, or a plan involving many trains of thought over time for internally motivated reasons. If we could get it to tell us a story about a particular giraffe, and then ask it next week about that same giraffe, and then the giraffe could be referenced by it while it tells a joke on a talk show in a decade -- that's maybe too high a bar, but that's real intelligence.
  - shawnz 4y ago
    Why is long term memory a requirement? Are amnesiacs not intelligent?
    - wcarss 4y ago
      > that's maybe too high a bar, but that's real intelligence. It's not necessarily a requirement, but I couldn't witness someone do it and then deny their intelligence.
      - shawnz 4y ago
        But it's not sufficient either, right? Just having memory alone also wouldn't prove intelligence, so it's not necessary or sufficient
        wcarss 4y ago
        I agree that memory alone is not sufficient to "prove intelligence". Above, I wrote a paragraph sketching an example of complex behaviours, entailing creativity, humour, longevity, planning, memory, storytelling, object recognition, social skills, etc., as a "clear display of intelligence", not a minima, to use as a counterpoint to merely "recognizing a giraffe". So I think you've missed my point, or, I've severely missed yours. If I have, sorry! As a general feeling toward internet nitpicks, we're unlikely to work in good faith to help each other out from here, so I'm out.
trention 4y ago
It knows what combinations of pixels correspond to the metadata having "giraffe" in it. Nothing more.
- blooperdeoo 4y ago
tellarin 4y ago
“ It actually knows what a giraffe is.” -> It maybe has a sense of what a giraffe looks like.
- bpodgursky 4y ago
  Let's not play word games. Do you know what a giraffe is? No, you just know what a giraffe looks like, where it lives, and maybe that it's vaguely related to a horse.
  - shafoshaf 4y ago
    That last bit about the horse is pretty loaded. Knowing what a giraffe is means knowing that it is a mammal, has a blood system, eats leaves, reacts certain ways to certain stimuli, etc... That is a lot more than just knowing what it looks like. Image recognition is not the same a relationships or knowledge.
    - influxmoment 4y ago
      > mammal, has a blood system, eats leaves, reacts certain DALL-E likely can map all those concepts to a giraffe also.
    - orlp 4y ago
      Now ask DALL-E "giraffe feeding young", "giraffe organs", "giraffe eating", etc, I think you'd find most knowledge is there.
      - teolandon 4y ago
        What about ,"giraffe laying eggs"?
    - karatinversion 4y ago
      By your criteria, no human before the 18th century (when Carl Linnaeus invented the class “mammalia”) knew what a giraffe was.
romellem 4y ago
If this was around in 2009 it would have made the https://www.onemilliongiraffes.com/ https://www.onemilliongiraffes.com/ project much easier.
blooperdeoo 4y ago
“ It actually knows what a giraffe is.” No. You know what a giraffe is, Dall•E simply creates pixel groups which correlate to the text pattern you submitted. Watching people discuss a logical mirror scares me that most people are not themselves conscious.
- gfodor 4y ago
  Not exactly. If you're using the word "simply" you probably don't quite understand what exactly DALL-E is doing. It's not just mapping correlations between words to pixels. The intermediate representations and textual attention are what make DALL-E interesting.
- jvanderbot 4y ago
  Precisely this! You don't know what a giraffe is, either. But you can talk about a giraffe to yourself, using words with correlating imagery that you can "see" by feeding through your visual cortex, and that's close enough.
  - astrange 4y ago
    You know more of what a giraffe is than DALLE; you can imagine one as a 3D model, for instance. It can't do this, it can only render different 2D images of one it's seen in photos. Of course, this is still impressive that it knows how to remove the backgrounds from those. Examples of prompts that don't really work are "spherical giraffe" and "the underside of a giraffe". We'll need more 3Dness to generate other media like models and video, so it should be coming.
    - jvanderbot 4y ago
      I agree. One thing I've noticed with most image generation, including faces, is that it makes the same mistakes that a novice artist makes. It draws eyes where eyes should be, mouths where mouths should be, but doesn't actually shape them correctly for the 3D head and viewer perspective. It can perfect a specific viewpoint with lots of training (like a novice artist who perfects drawing realistic faces and not much else). But my interpretation of much of the artifacts are that they are just missing structural cues.
amelius 4y ago
Reminds me of a classifier that couldn't tell the difference between a panther and a couch with a panther-print.
- dane-pgp 4y ago
  But what if the couch wasn't panther-print but made from actual panther hide? Is a dead panther still a panther? How much of the panther's body has to be visible for the object depicted to still be a panther? I'm reminded of the classic joke about the mathematician on a train who sees a sheep... https://bestjokehub.com/joke-103626/ https://bestjokehub.com/joke-103626/
  - amelius 4y ago
    True, but the problem with this view is that it is still a couch for all practical purposes.
deleted 4y ago
[deleted]
SketchySeaBeast 4y ago
> It actually knows what a giraffe is. Every time I see a painting it does with blank biege swirls where faces should be I question that it does know what anything is or what the important parts are.
- LordDragonfang 4y ago
  I suspect you're thinking of Craiyon (formerly DALL-E Mini) which is a derivative (but distinct) project, and consistently does the beige swirls. DALL-E 2 proper is much higher fidelity, and seems much more reliable at generating passable (if imperfect) faces from what I've seen.
  - astrange 4y ago
    It's pretty easy to get it to start losing details; if you ask for too much in one picture, somehow everything gets worse. https://labs.openai.com/s/xI4wV6WOKOcgwZATmvPXyfBu https://labs.openai.com/s/xI4wV6WOKOcgwZATmvPXyfBu (this also shows you can get different variants on a concept by mixing languages)
    - LordDragonfang 4y ago
      Fair, but even that is better than the best faces I've seen from Craiyon, and far from a "blank biege swirl". I think faces in particular are an unfair/extreme example anyway, since there's evidence of human brains being ridiculously over-tuned for recognizing faces and facial abnormalities.
ramoz 4y ago
Because you’ve mentioned Google, here’s this: https://imagen.research.google/ https://imagen.research.google/
rob74 4y ago
What's still missing, as the article mentions, is the correct relationship between objects. As in this post from a few months ago: https://astralcodexten.substack.com/i/57243633/the-tenth-virtue-precision https://astralcodexten.substack.com/i/57243633/the-tenth-vir... - it knows that Tycho Brahe was a bearded guy, and it knows what a telescope looks like, but when you ask for "Tycho Brahe looking through a telescope", the person sometimes looks at the telescope, or past it, sometimes the telescope just sits there, but the person never actually looks through it.

by orlp 4y ago

TremendousJudge 4y ago

I find talking about "knowing" is too much anthropomorphizing to my taste. A more accurate description would be that it can synthesize an endless amount of pictures that the viewer will recognize as giraffes (or as anything else the viewer wants). And I agree, it's pretty amazing. But my point is that talking like this skips over the entire useless "what is knowledge" debate.

soco 4y ago
Maybe we need to "de-antropomorphize" these terms and rules. It's a long due process, I believe.
- TremendousJudge 4y ago
  I don't think it's possible. "Knowing" and "understanding" are inherently human experiences. The most reasonable thing to do would be to invent new terms and relegate the older ones, that have too much cultural baggage attached to them. The same way when the creator of humanity turned out to be "evolution by natural selection" we didn't redefine the term "God" to mean that. Eventually we just started using the new term.
- psyc 4y ago
  If it'll get people to stop accusing other people of anthropomorphizing, I'm 100% on board. Now if you can think of a way to get people to quit asking for definitions of the words intelligence and consciousness, you'll be a legend and maybe these discussions will advance for the first time.
  - TremendousJudge 4y ago
    Well, I tried my best. Another commenter summed it up better without using the term "anthropomorphizing" at all, but it came out a bit more condescending than necessary.
  - IIAOPSW 4y ago
    Hang on while I become a legend. I use GPT3 to generate the usual trite arguments about intelligence and consciousness why computers won't ever get there. Of course I don't actually reveal that a computer is generating my responses until later on. Eventually everyone will become jaded and skeptical that the other participants in that conversation are real people. Soon all arguments about machine intelligence will devolve into accusations of using GPT3 or not. Some day, even mentioning consciousness will just make everyone assume you're probably a GPT3 troll. This kills the conversation in a way that makes a valid point. If the bots can't be reliably identified, the proof is in the pudding and the matter is settled.
MauranKilom 4y ago
I too can write a program that will synthesize an endless amount of pictures that the viewer will recognize as a giraffe. They would all be the same picture, just with one bit of noise randomly applied to each pixel - enough to generate more distinct 256x256 pictures of giraffes than there are atoms in the universe. We need to go a bit deeper to tease out what makes DALL-E 2 amazing.

soco 4y ago

A honest question: what is then intelligence? Or are you just trying to avoid being caught into a debate...

wcarss 4y ago
you could think of "knowing what a giraffe is" in this instance as a slice of an intelligent thought, but it's not a whole thought, let alone a train of thoughts, or a plan involving many trains of thought over time for internally motivated reasons. If we could get it to tell us a story about a particular giraffe, and then ask it next week about that same giraffe, and then the giraffe could be referenced by it while it tells a joke on a talk show in a decade -- that's maybe too high a bar, but that's real intelligence.
- shawnz 4y ago
  Why is long term memory a requirement? Are amnesiacs not intelligent?
  - wcarss 4y ago
    > that's maybe too high a bar, but that's real intelligence. It's not necessarily a requirement, but I couldn't witness someone do it and then deny their intelligence.
    - shawnz 4y ago
      But it's not sufficient either, right? Just having memory alone also wouldn't prove intelligence, so it's not necessary or sufficient
      - wcarss 4y ago
        I agree that memory alone is not sufficient to "prove intelligence". Above, I wrote a paragraph sketching an example of complex behaviours, entailing creativity, humour, longevity, planning, memory, storytelling, object recognition, social skills, etc., as a "clear display of intelligence", not a minima, to use as a counterpoint to merely "recognizing a giraffe". So I think you've missed my point, or, I've severely missed yours. If I have, sorry! As a general feeling toward internet nitpicks, we're unlikely to work in good faith to help each other out from here, so I'm out.

trention 4y ago

It knows what combinations of pixels correspond to the metadata having "giraffe" in it. Nothing more.

blooperdeoo 4y ago

tellarin 4y ago

“ It actually knows what a giraffe is.” -> It maybe has a sense of what a giraffe looks like.

bpodgursky 4y ago
Let's not play word games. Do you know what a giraffe is? No, you just know what a giraffe looks like, where it lives, and maybe that it's vaguely related to a horse.
- shafoshaf 4y ago
  That last bit about the horse is pretty loaded. Knowing what a giraffe is means knowing that it is a mammal, has a blood system, eats leaves, reacts certain ways to certain stimuli, etc... That is a lot more than just knowing what it looks like. Image recognition is not the same a relationships or knowledge.
  - influxmoment 4y ago
    > mammal, has a blood system, eats leaves, reacts certain DALL-E likely can map all those concepts to a giraffe also.
  - orlp 4y ago
    Now ask DALL-E "giraffe feeding young", "giraffe organs", "giraffe eating", etc, I think you'd find most knowledge is there.
    - teolandon 4y ago
      What about ,"giraffe laying eggs"?
  - karatinversion 4y ago
    By your criteria, no human before the 18th century (when Carl Linnaeus invented the class “mammalia”) knew what a giraffe was.

romellem 4y ago

If this was around in 2009 it would have made the https://www.onemilliongiraffes.com/ https://www.onemilliongiraffes.com/ project much easier.

blooperdeoo 4y ago

“ It actually knows what a giraffe is.” No. You know what a giraffe is, Dall•E simply creates pixel groups which correlate to the text pattern you submitted. Watching people discuss a logical mirror scares me that most people are not themselves conscious.

gfodor 4y ago
Not exactly. If you're using the word "simply" you probably don't quite understand what exactly DALL-E is doing. It's not just mapping correlations between words to pixels. The intermediate representations and textual attention are what make DALL-E interesting.
jvanderbot 4y ago
Precisely this! You don't know what a giraffe is, either. But you can talk about a giraffe to yourself, using words with correlating imagery that you can "see" by feeding through your visual cortex, and that's close enough.
- astrange 4y ago
  You know more of what a giraffe is than DALLE; you can imagine one as a 3D model, for instance. It can't do this, it can only render different 2D images of one it's seen in photos. Of course, this is still impressive that it knows how to remove the backgrounds from those. Examples of prompts that don't really work are "spherical giraffe" and "the underside of a giraffe". We'll need more 3Dness to generate other media like models and video, so it should be coming.
  - jvanderbot 4y ago
    I agree. One thing I've noticed with most image generation, including faces, is that it makes the same mistakes that a novice artist makes. It draws eyes where eyes should be, mouths where mouths should be, but doesn't actually shape them correctly for the 3D head and viewer perspective. It can perfect a specific viewpoint with lots of training (like a novice artist who perfects drawing realistic faces and not much else). But my interpretation of much of the artifacts are that they are just missing structural cues.

amelius 4y ago

Reminds me of a classifier that couldn't tell the difference between a panther and a couch with a panther-print.

dane-pgp 4y ago
But what if the couch wasn't panther-print but made from actual panther hide? Is a dead panther still a panther? How much of the panther's body has to be visible for the object depicted to still be a panther? I'm reminded of the classic joke about the mathematician on a train who sees a sheep... https://bestjokehub.com/joke-103626/ https://bestjokehub.com/joke-103626/
- amelius 4y ago
  True, but the problem with this view is that it is still a couch for all practical purposes.

deleted 4y ago

[deleted]

SketchySeaBeast 4y ago

> It actually knows what a giraffe is. Every time I see a painting it does with blank biege swirls where faces should be I question that it does know what anything is or what the important parts are.

LordDragonfang 4y ago
I suspect you're thinking of Craiyon (formerly DALL-E Mini) which is a derivative (but distinct) project, and consistently does the beige swirls. DALL-E 2 proper is much higher fidelity, and seems much more reliable at generating passable (if imperfect) faces from what I've seen.
- astrange 4y ago
  It's pretty easy to get it to start losing details; if you ask for too much in one picture, somehow everything gets worse. https://labs.openai.com/s/xI4wV6WOKOcgwZATmvPXyfBu https://labs.openai.com/s/xI4wV6WOKOcgwZATmvPXyfBu (this also shows you can get different variants on a concept by mixing languages)
  - LordDragonfang 4y ago
    Fair, but even that is better than the best faces I've seen from Craiyon, and far from a "blank biege swirl". I think faces in particular are an unfair/extreme example anyway, since there's evidence of human brains being ridiculously over-tuned for recognizing faces and facial abnormalities.

ramoz 4y ago

Because you’ve mentioned Google, here’s this: https://imagen.research.google/ https://imagen.research.google/

rob74 4y ago

What's still missing, as the article mentions, is the correct relationship between objects. As in this post from a few months ago: https://astralcodexten.substack.com/i/57243633/the-tenth-virtue-precision https://astralcodexten.substack.com/i/57243633/the-tenth-vir... - it knows that Tycho Brahe was a bearded guy, and it knows what a telescope looks like, but when you ask for "Tycho Brahe looking through a telescope", the person sometimes looks at the telescope, or past it, sometimes the telescope just sits there, but the person never actually looks through it.