4 ms·

As someone who appreciates the craftsmanship and care that goes into human-created art, I found the abilities of DALL-E to be a little distressing. These limita

by yosito 5y ago

As someone who appreciates the craftsmanship and care that goes into human-created art, I found the abilities of DALL-E to be a little distressing. These limitations really point out how much DALL-E is still very different from humans, in a way that I find reassuring. Human reasoning still seems to be out of the reach of these kinds of models. What they really seem to be is very very good imitators which, after being trained on very large amounts of data, can imitate what they've seen. You could make an argument that that's essentially what humans do too, just in a more complex way. But I think it underscores why humans and machines will always be different. We'll always be trained on different inputs, and have our own unique way of combining those inputs, even as the abilities of AI rival and even surpass our own in many ways.

ravi-delia 5y ago
Honestly, I'd assume that pretty much any of these flaws are more fixable than you'd expect. We probably won't hit upon the right way for a while, since large models are so expensive, but with some trickery on the architectural level and bigger models, they'll spit out better and better art until it's indistinguishable from human level. Most of your brain is doing the same kind of dumb learning these models are, just much better and faster. There are some fundamental architectural differences, especially relating to the higher-level functions we associate with "consciousness" (working and short term memory, attention, maybe the executive?), but for almost every task you stick with the big dumb blob of compute and it gets the job done better than your higher faculties ever could. Don't underestimate the extent to which your brain plows through difficulties by throwing neurons at the problem until it goes away. Our biggest models are still a fraction of the size and a smaller fraction of the complexity of the bits of brain which perform the same function. I won't rule out anything until we see models of that scale.
- Jimmy 5y ago
  >Honestly, I'd assume that pretty much any of these flaws are more fixable than you'd expect. The first 90% of the project takes 90% of the time, the last 10% of the project takes the other 90% of the time. I don't think it's wise to speculate at this point on what problems are "actually easy" or "actually hard", or how long it'll take to achieve certain milestones. I don't think that even the people at the forefront of this research have any way of knowing for sure.
  - IIAOPSW 5y ago
    your projects take up 180% of the time?
    - Jimmy 5y ago
      Yours don't?
    - webmaven 5y ago
      > your projects take up 180% of the time? For some projects, 180% would have been an improvement. I've even had a few projects take ~500% of the time. Arguably I should have cut my losses sooner, but I wasn't very good at recognizing the sunk cost fallacy back then.
- Sebb767 5y ago
  To completly replace artists, the model would need to fully understand speech and have a good model of the world. The failure cases of the monk and the negation show pretty clearly that the model does something more akin to a keyword search rather than actual NLP. This alone will be a major problem for the last 10% - the more complex edge cases you introduce, the closer you need to be to AGI to handle them. This might come at some point, but the effort needed will skyrocket for less and less payoff, so I think we can safely assume that this will still take some time.
- webmaven 5y ago
  > Don't underestimate the extent to which your brain plows through difficulties by throwing neurons at the problem until it goes away. We also mustn't underestimate the extent in which the human brain is a fuzzy ensemble of partial experts, and composes the first few salient answers provided rather than trying to find a "best" answer that would be too late. Or how often the human brain creates post hoc rationalizations simply to provide narrative continuity.

by yosito 5y ago

ravi-delia 5y ago

Honestly, I'd assume that pretty much any of these flaws are more fixable than you'd expect. We probably won't hit upon the right way for a while, since large models are so expensive, but with some trickery on the architectural level and bigger models, they'll spit out better and better art until it's indistinguishable from human level. Most of your brain is doing the same kind of dumb learning these models are, just much better and faster. There are some fundamental architectural differences, especially relating to the higher-level functions we associate with "consciousness" (working and short term memory, attention, maybe the executive?), but for almost every task you stick with the big dumb blob of compute and it gets the job done better than your higher faculties ever could. Don't underestimate the extent to which your brain plows through difficulties by throwing neurons at the problem until it goes away. Our biggest models are still a fraction of the size and a smaller fraction of the complexity of the bits of brain which perform the same function. I won't rule out anything until we see models of that scale.

Jimmy 5y ago
>Honestly, I'd assume that pretty much any of these flaws are more fixable than you'd expect. The first 90% of the project takes 90% of the time, the last 10% of the project takes the other 90% of the time. I don't think it's wise to speculate at this point on what problems are "actually easy" or "actually hard", or how long it'll take to achieve certain milestones. I don't think that even the people at the forefront of this research have any way of knowing for sure.
- IIAOPSW 5y ago
  your projects take up 180% of the time?
  - Jimmy 5y ago
    Yours don't?
  - webmaven 5y ago
    > your projects take up 180% of the time? For some projects, 180% would have been an improvement. I've even had a few projects take ~500% of the time. Arguably I should have cut my losses sooner, but I wasn't very good at recognizing the sunk cost fallacy back then.
Sebb767 5y ago
To completly replace artists, the model would need to fully understand speech and have a good model of the world. The failure cases of the monk and the negation show pretty clearly that the model does something more akin to a keyword search rather than actual NLP. This alone will be a major problem for the last 10% - the more complex edge cases you introduce, the closer you need to be to AGI to handle them. This might come at some point, but the effort needed will skyrocket for less and less payoff, so I think we can safely assume that this will still take some time.
webmaven 5y ago
> Don't underestimate the extent to which your brain plows through difficulties by throwing neurons at the problem until it goes away. We also mustn't underestimate the extent in which the human brain is a fuzzy ensemble of partial experts, and composes the first few salient answers provided rather than trying to find a "best" answer that would be too late. Or how often the human brain creates post hoc rationalizations simply to provide narrative continuity.