3 ms·

I don't think that they consider 3d too much. Consider the "Girl with a Pearl Earring" image, that's meant to represent something that's state of the art: http

I don't think that they consider 3d too much. Consider the "Girl with a Pearl Earring" image, that's meant to represent something that's state of the art: https://openai.com/blog/dall-e-introducing-outpainting/ https://openai.com/blog/dall-e-introducing-outpainting/ The image might look consistent at first sight, but if you look closer, the dimensions are all over the place. They reflect the method by which they were created: reproducing observed patterns without deeper understanding.

l33tman 4y ago
You can find lots of buggy images with bugs in all possible ways. But the vast majority of the images have consistent shading, perspective, and even global illumination. Some of it can be done in "screen space" (as in some standard game 2D post-processing filters) but lots of them couldn't work this way. At some low level, pixel patterns are rendered and I guess you could say that this is "reproducing observed patterns". Would you say that about a 3D game engine as well that does the same when it textures local regions of pixels? A network is layered for this particular reason. The lowest layer will have less "understanding" than the higher layers.
astrange 4y ago
If you ask for something "reflective" or "in a hall of mirrors" it tends to work, though it often seems like a screen-space effect as it tends to draw the front of objects rather than their back in the reflection. https://labs.openai.com/s/OJEkBZSwagsCakHiE9QNdV92 https://labs.openai.com/s/OJEkBZSwagsCakHiE9QNdV92
- npteljes 4y ago
  The perspective barely changes in the sphere. I'm not convinced that the AI implied 3d, lighting/shadows, and reflections from the dataset which is supposed to be, at most, just 2d projections of 3d environments. Maybe in the future when we feed it 3d (meta) information also, then we can expect some magic, but until then, I don't think so.

l33tman 4y ago

You can find lots of buggy images with bugs in all possible ways. But the vast majority of the images have consistent shading, perspective, and even global illumination. Some of it can be done in "screen space" (as in some standard game 2D post-processing filters) but lots of them couldn't work this way. At some low level, pixel patterns are rendered and I guess you could say that this is "reproducing observed patterns". Would you say that about a 3D game engine as well that does the same when it textures local regions of pixels? A network is layered for this particular reason. The lowest layer will have less "understanding" than the higher layers.

astrange 4y ago

If you ask for something "reflective" or "in a hall of mirrors" it tends to work, though it often seems like a screen-space effect as it tends to draw the front of objects rather than their back in the reflection. https://labs.openai.com/s/OJEkBZSwagsCakHiE9QNdV92 https://labs.openai.com/s/OJEkBZSwagsCakHiE9QNdV92

npteljes 4y ago
The perspective barely changes in the sphere. I'm not convinced that the AI implied 3d, lighting/shadows, and reflections from the dataset which is supposed to be, at most, just 2d projections of 3d environments. Maybe in the future when we feed it 3d (meta) information also, then we can expect some magic, but until then, I don't think so.