3 ms·

Are you familiar with the empirical results that have been observed when you ask a human to 'draw a bicycle'? https://www.booooooom.com/2016/05/09/bicycles-bui

Are you familiar with the empirical results that have been observed when you ask a human to 'draw a bicycle'? https://www.booooooom.com/2016/05/09/bicycles-built-based-on-peoples-attempts-to-draw-them-from-memory/ https://www.booooooom.com/2016/05/09/bicycles-built-based-on... People draw bicycle-like shapes, that don't quite work - they miss important details, or connect things that shouldn't be connected.

woodruffw 5y ago
This is a funny comparison to draw, but I don't think it's one with profound meaning: humans have trouble drawing bicycles, but the semantics of a badly-drawn bicycle are sound. This is in contrast to DALL-E, which might do an excellent job of drawing apples but otherwise misses the semantics of the query (i.e., "without apples"). In other words: humans mis-draw things because we don't always do a good job of visually apprehending the world; DALL-E mis-draws things because it doesn't really understand the problem statement.
- native_samples 5y ago
  The semantics aren't that sound - a lot of the bikes have the seat at the very back, too far from the handlebars.
  - woodruffw 5y ago
    None of them are good, but the semantics are correct: the seat is behind the handlebars, the handlebars curve correctly, &c. The fact that humans tend to draw the pieces too far apart is more a testament to our problems drawing scale and perspective than a semantic error. (I think this would be apparent if participants were asked to draw a bicycle, followed by a bicycle without a wheel. The two drawings would be roughly as "good" in terms of quality, but the humans will correctly obey the prompt and omit one of the two wheels. I suspect DALL-E would struggle with that prompt.)
dtech 5y ago
This is from memory. Humans wouldn't make this (serious level) of mistakes if they had an enormous data-set of bicycles to draw from.
- jameshart 5y ago
  That’s an odd distinction to make. DALL-E is a trained model, everything it produces is ‘from memory’. It doesn’t go out and google for reference material based on the prompt.
  - gwern 5y ago
    > It doesn’t go out and google for reference material based on the prompt. And note that even if it did (which is definitely architecturally possible given progress in retrieval and demos like WebGPT), it still wouldn't necessarily be 'copying', in the goalpost-moving excuse. Facebook has a neat diffusion model which uses retrieval on a dataset of images for generation: https://arxiv.org/abs/2204.02849#facebook https://arxiv.org/abs/2204.02849#facebook You can see the exemplars it uses: they both greatly improve the quality of generated samples, but also don't look that much like the generated sample.
  - dtech 5y ago
    Yes, but it was still was trained on a large dataset for the purpose of drawing. It's not an AGI that just happened to be asked to draw bicycles. Maybe a better comparison would be a trained artist, who likely wouldn't make such egregious mistakes.
- gwern 5y ago
  Humans do, because they have spent a lifetime looking at and riding bikes. They draw from memory just as much as DALL-E 2 does.

woodruffw 5y ago

This is a funny comparison to draw, but I don't think it's one with profound meaning: humans have trouble drawing bicycles, but the semantics of a badly-drawn bicycle are sound. This is in contrast to DALL-E, which might do an excellent job of drawing apples but otherwise misses the semantics of the query (i.e., "without apples"). In other words: humans mis-draw things because we don't always do a good job of visually apprehending the world; DALL-E mis-draws things because it doesn't really understand the problem statement.

native_samples 5y ago
The semantics aren't that sound - a lot of the bikes have the seat at the very back, too far from the handlebars.
- woodruffw 5y ago
  None of them are good, but the semantics are correct: the seat is behind the handlebars, the handlebars curve correctly, &c. The fact that humans tend to draw the pieces too far apart is more a testament to our problems drawing scale and perspective than a semantic error. (I think this would be apparent if participants were asked to draw a bicycle, followed by a bicycle without a wheel. The two drawings would be roughly as "good" in terms of quality, but the humans will correctly obey the prompt and omit one of the two wheels. I suspect DALL-E would struggle with that prompt.)

dtech 5y ago

This is from memory. Humans wouldn't make this (serious level) of mistakes if they had an enormous data-set of bicycles to draw from.

jameshart 5y ago
That’s an odd distinction to make. DALL-E is a trained model, everything it produces is ‘from memory’. It doesn’t go out and google for reference material based on the prompt.
- gwern 5y ago
  > It doesn’t go out and google for reference material based on the prompt. And note that even if it did (which is definitely architecturally possible given progress in retrieval and demos like WebGPT), it still wouldn't necessarily be 'copying', in the goalpost-moving excuse. Facebook has a neat diffusion model which uses retrieval on a dataset of images for generation: https://arxiv.org/abs/2204.02849#facebook https://arxiv.org/abs/2204.02849#facebook You can see the exemplars it uses: they both greatly improve the quality of generated samples, but also don't look that much like the generated sample.
- dtech 5y ago
  Yes, but it was still was trained on a large dataset for the purpose of drawing. It's not an AGI that just happened to be asked to draw bicycles. Maybe a better comparison would be a trained artist, who likely wouldn't make such egregious mistakes.
gwern 5y ago
Humans do, because they have spent a lifetime looking at and riding bikes. They draw from memory just as much as DALL-E 2 does.