3 ms·

Honestly, I'd assume that pretty much any of these flaws are more fixable than you'd expect. We probably won't hit upon the right way for a while, since large m

Honestly, I'd assume that pretty much any of these flaws are more fixable than you'd expect. We probably won't hit upon the right way for a while, since large models are so expensive, but with some trickery on the architectural level and bigger models, they'll spit out better and better art until it's indistinguishable from human level. Most of your brain is doing the same kind of dumb learning these models are, just much better and faster. There are some fundamental architectural differences, especially relating to the higher-level functions we associate with "consciousness" (working and short term memory, attention, maybe the executive?), but for almost every task you stick with the big dumb blob of compute and it gets the job done better than your higher faculties ever could. Don't underestimate the extent to which your brain plows through difficulties by throwing neurons at the problem until it goes away. Our biggest models are still a fraction of the size and a smaller fraction of the complexity of the bits of brain which perform the same function. I won't rule out anything until we see models of that scale.

Jimmy 4y ago
>Honestly, I'd assume that pretty much any of these flaws are more fixable than you'd expect. The first 90% of the project takes 90% of the time, the last 10% of the project takes the other 90% of the time. I don't think it's wise to speculate at this point on what problems are "actually easy" or "actually hard", or how long it'll take to achieve certain milestones. I don't think that even the people at the forefront of this research have any way of knowing for sure.
- IIAOPSW 4y ago
  your projects take up 180% of the time?
  - Jimmy 4y ago
    Yours don't?
  - webmaven 4y ago
    > your projects take up 180% of the time? For some projects, 180% would have been an improvement. I've even had a few projects take ~500% of the time. Arguably I should have cut my losses sooner, but I wasn't very good at recognizing the sunk cost fallacy back then.
Sebb767 4y ago
To completly replace artists, the model would need to fully understand speech and have a good model of the world. The failure cases of the monk and the negation show pretty clearly that the model does something more akin to a keyword search rather than actual NLP. This alone will be a major problem for the last 10% - the more complex edge cases you introduce, the closer you need to be to AGI to handle them. This might come at some point, but the effort needed will skyrocket for less and less payoff, so I think we can safely assume that this will still take some time.
webmaven 4y ago
> Don't underestimate the extent to which your brain plows through difficulties by throwing neurons at the problem until it goes away. We also mustn't underestimate the extent in which the human brain is a fuzzy ensemble of partial experts, and composes the first few salient answers provided rather than trying to find a "best" answer that would be too late. Or how often the human brain creates post hoc rationalizations simply to provide narrative continuity.

Jimmy 4y ago

>Honestly, I'd assume that pretty much any of these flaws are more fixable than you'd expect. The first 90% of the project takes 90% of the time, the last 10% of the project takes the other 90% of the time. I don't think it's wise to speculate at this point on what problems are "actually easy" or "actually hard", or how long it'll take to achieve certain milestones. I don't think that even the people at the forefront of this research have any way of knowing for sure.

IIAOPSW 4y ago
your projects take up 180% of the time?
- Jimmy 4y ago
  Yours don't?
- webmaven 4y ago
  > your projects take up 180% of the time? For some projects, 180% would have been an improvement. I've even had a few projects take ~500% of the time. Arguably I should have cut my losses sooner, but I wasn't very good at recognizing the sunk cost fallacy back then.

Sebb767 4y ago

To completly replace artists, the model would need to fully understand speech and have a good model of the world. The failure cases of the monk and the negation show pretty clearly that the model does something more akin to a keyword search rather than actual NLP. This alone will be a major problem for the last 10% - the more complex edge cases you introduce, the closer you need to be to AGI to handle them. This might come at some point, but the effort needed will skyrocket for less and less payoff, so I think we can safely assume that this will still take some time.

webmaven 4y ago

> Don't underestimate the extent to which your brain plows through difficulties by throwing neurons at the problem until it goes away. We also mustn't underestimate the extent in which the human brain is a fuzzy ensemble of partial experts, and composes the first few salient answers provided rather than trying to find a "best" answer that would be too late. Or how often the human brain creates post hoc rationalizations simply to provide narrative continuity.