4 ms·

> It is wholly unconcerned with the actual objects rendered that are the ostensible focus of the prompt. I disagree. To the extent that the training data are i

> It is wholly unconcerned with the actual objects rendered that are the ostensible focus of the prompt. I disagree. To the extent that the training data are images of actual objects, recreating images of actual objects is the only thing DALL-E cares about. If we define "caring" about something as changing behavior to cause that to happen, then a neural network doesn't "care" about inference at all, because inference never changes the network's behavior. It also doesn't know or care about your approval. It only cares about minimizing the loss function. (But now that you bring this up, I think it would be really interesting to create a network that, after training initially on training data, began interacting with people and continued training to maximize approval.)

mjburgess 4y ago
Images of objects arent objects and measure only their 2D geometrical properties projected onto the image plane. These properties are correlated across images, inexplicably, since the machine has no access to the physical properties of those objects, nor the laws of physics.
- leereeves 4y ago
  It's hypothesized (and, I think, likely) that the machine learns some of those hidden/latent properties, though, obviously, only to the extent that they are useful to recreate the images. In any case, it does at least care about the appearance of the actual objects. So I think it would be fair to say that there are aspects of the actual objects that the network doesn't care about, but there are also aspects that it cares very much about. Thus it's not "wholly unconcerned with the actual objects".
  - MauranKilom 4y ago
    > It's hypothesized (and, I think, likely) that the machine learns some of those hidden/latent properties, though, obviously, only to the extent that they are useful to recreate the images. Notably, this precisely describes humans too. We don't know the "true" properties of anything we interact with. We just have models - some more sophisticated than others - but only to the extent that we care for reasoning about the objects. From "This stone is too heavy to lift." to "Oh yeah Becky is always late."
  - IIAOPSW 4y ago
    My gripe with this is that it has no understanding of what the objects are beyond "the thing that's supposed to be output". Its a textbook case of the Chinese room argument [1]. Sure it can follow translation rules, even infer some sophisticated rules and patterns about the slips of paper that are passed in, but it still has no idea what any of it means. https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room Dalle is very concerned with the 2d shapes of things we call objects, and has correctly inferred some rules about those shapes, but it neither knows nor cares about the things we call objects and how the shapes it has learned are representations of them. It doesn't do reasoning about the round peg fitting in the round hole. It just glues a few pegs and holes together in a way that's mostly consistent with our expectations and says "is this what you wanted"? Its a student that cares about passing the test, not learning the material.
    - leereeves 4y ago
      Isn't that true of all of us? We all care selectively about certain attributes of objects. I care that my car is quiet and has comfortable seats, I don't care (or know) what material the muffler is made of, but somewhere there is an engineer who cared about that. A road designer cares what the tires are made of and how much it weighs, but doesn't care what color the paint is. An AI recreating an image of my car would care what color the paint is, but not how comfortable the seats are. I think I see what you're describing - the AI has a very limited scope and doesn't know or care about most of the things we do - but I think that's just a limitation of our current small models and limited training data, not an inherent limitation of neural networks.
      - IIAOPSW 4y ago
        You may not know what material the muffler is made of, but you know its made of a material. You "care about the objects themselves" in the sense of having this framework of objects, all of which have certain properties and relations to each other which form a predictive model you have about the world. You expect everything to have a position in x,y,z, to have a size, shape, mass, hardness etc. You might not care about a particular property of a particular object in a particular context, but you fundamentally think in the semantics of objects. You know what to expect when the heavy hard object meets the brittle lite object. DallE doesn't really have a system of the semantics of objects. It doesn't know why it would be useless for a building to have all the doors on the 2nd level. Its not even clear that DallE makes use of discrete "objects" in its limited understanding. Here's an example from budget DallE https://ibb.co/B2B5zky https://ibb.co/B2B5zky It understood the shape of "stick figure" and "boobs", but had no understanding of what a stick figure is meant to represent and thus where it should place the boobs. The results are hilarious. I'm not sure which I like more, the guy with a boob casually walking down his arm, or the lady with a boob head that's shrugging with uncertainty.
- Earw0rm 4y ago
  Where this gets interesting is when the AI attempts to create animations or moving images - this requires a much deeper understanding of objects and their physical properties, even to create a highly stylised or "drawn" animation.
- icehawk 4y ago
  Now explain how that's different than anything you look at. - Your visual system only has access to the 2D geometrical properties projected on your retina. The properties are correlated across images, inexplicably. (I certainly cannot explain how chairs are, in a fashion that includes all chairs I've encountered and excludes anything I've encountered that is not a chair) - Any other interaction is also a correlation. - Humans don't have access to the laws of physics, just reasonable approximations in certain contexts.
  - Barrin92 4y ago
    >Now explain how that's different than anything you look at. For starters you just don't look at things, you're embedded in the world. You have sensory input far beyond visual information, you also have something akin to cybernetic feedback in response to your mechanical actions in the world, and DALL-E has not. In fact DALL-E doesn't even have access to visual information in the same sense you have, which is to a large extent biochemical and analog, not digital.
  - mjburgess 4y ago
    It's exhausting to have people go around repeating this pseudoscience. I hope it's over soon. We have accurate causal models of our own bodies, and are directly causally active in the world so we can create experiments which enable us to discover its properties. We do not live in a schizophrenic nightmare, forever trapped just to correlate pixel patterns. The claim that we do must, imo, now strongly be labelled as pseudoscience. This is the new homeopathy; and AI/ML based on it is a scientific fraud.

mjburgess 4y ago

Images of objects arent objects and measure only their 2D geometrical properties projected onto the image plane. These properties are correlated across images, inexplicably, since the machine has no access to the physical properties of those objects, nor the laws of physics.

leereeves 4y ago
It's hypothesized (and, I think, likely) that the machine learns some of those hidden/latent properties, though, obviously, only to the extent that they are useful to recreate the images. In any case, it does at least care about the appearance of the actual objects. So I think it would be fair to say that there are aspects of the actual objects that the network doesn't care about, but there are also aspects that it cares very much about. Thus it's not "wholly unconcerned with the actual objects".
- MauranKilom 4y ago
  > It's hypothesized (and, I think, likely) that the machine learns some of those hidden/latent properties, though, obviously, only to the extent that they are useful to recreate the images. Notably, this precisely describes humans too. We don't know the "true" properties of anything we interact with. We just have models - some more sophisticated than others - but only to the extent that we care for reasoning about the objects. From "This stone is too heavy to lift." to "Oh yeah Becky is always late."
- IIAOPSW 4y ago
  My gripe with this is that it has no understanding of what the objects are beyond "the thing that's supposed to be output". Its a textbook case of the Chinese room argument [1]. Sure it can follow translation rules, even infer some sophisticated rules and patterns about the slips of paper that are passed in, but it still has no idea what any of it means. https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room Dalle is very concerned with the 2d shapes of things we call objects, and has correctly inferred some rules about those shapes, but it neither knows nor cares about the things we call objects and how the shapes it has learned are representations of them. It doesn't do reasoning about the round peg fitting in the round hole. It just glues a few pegs and holes together in a way that's mostly consistent with our expectations and says "is this what you wanted"? Its a student that cares about passing the test, not learning the material.
  - leereeves 4y ago
    Isn't that true of all of us? We all care selectively about certain attributes of objects. I care that my car is quiet and has comfortable seats, I don't care (or know) what material the muffler is made of, but somewhere there is an engineer who cared about that. A road designer cares what the tires are made of and how much it weighs, but doesn't care what color the paint is. An AI recreating an image of my car would care what color the paint is, but not how comfortable the seats are. I think I see what you're describing - the AI has a very limited scope and doesn't know or care about most of the things we do - but I think that's just a limitation of our current small models and limited training data, not an inherent limitation of neural networks.
    - IIAOPSW 4y ago
      You may not know what material the muffler is made of, but you know its made of a material. You "care about the objects themselves" in the sense of having this framework of objects, all of which have certain properties and relations to each other which form a predictive model you have about the world. You expect everything to have a position in x,y,z, to have a size, shape, mass, hardness etc. You might not care about a particular property of a particular object in a particular context, but you fundamentally think in the semantics of objects. You know what to expect when the heavy hard object meets the brittle lite object. DallE doesn't really have a system of the semantics of objects. It doesn't know why it would be useless for a building to have all the doors on the 2nd level. Its not even clear that DallE makes use of discrete "objects" in its limited understanding. Here's an example from budget DallE https://ibb.co/B2B5zky https://ibb.co/B2B5zky It understood the shape of "stick figure" and "boobs", but had no understanding of what a stick figure is meant to represent and thus where it should place the boobs. The results are hilarious. I'm not sure which I like more, the guy with a boob casually walking down his arm, or the lady with a boob head that's shrugging with uncertainty.
Earw0rm 4y ago
Where this gets interesting is when the AI attempts to create animations or moving images - this requires a much deeper understanding of objects and their physical properties, even to create a highly stylised or "drawn" animation.
icehawk 4y ago
Now explain how that's different than anything you look at. - Your visual system only has access to the 2D geometrical properties projected on your retina. The properties are correlated across images, inexplicably. (I certainly cannot explain how chairs are, in a fashion that includes all chairs I've encountered and excludes anything I've encountered that is not a chair) - Any other interaction is also a correlation. - Humans don't have access to the laws of physics, just reasonable approximations in certain contexts.
- Barrin92 4y ago
  >Now explain how that's different than anything you look at. For starters you just don't look at things, you're embedded in the world. You have sensory input far beyond visual information, you also have something akin to cybernetic feedback in response to your mechanical actions in the world, and DALL-E has not. In fact DALL-E doesn't even have access to visual information in the same sense you have, which is to a large extent biochemical and analog, not digital.
- mjburgess 4y ago
  It's exhausting to have people go around repeating this pseudoscience. I hope it's over soon. We have accurate causal models of our own bodies, and are directly causally active in the world so we can create experiments which enable us to discover its properties. We do not live in a schizophrenic nightmare, forever trapped just to correlate pixel patterns. The claim that we do must, imo, now strongly be labelled as pseudoscience. This is the new homeopathy; and AI/ML based on it is a scientific fraud.