3 ms·
> But the truly nasty part about AI and RL in particular is that the AI will act as if anything that it didn't see often enough during training simply doesn't e
by gwern 5y ago
> But the truly nasty part about AI and RL in particular is that the AI will act as if anything that it didn't see often enough during training simply doesn't exist. If it never sees a pink truck from the side, no "virtual neurons" will grow to detect this. AIs in general don't generalize. So if your driving dataset lacks enough examples of 0.1% black swan events, you can be sure that your AI is going to go totally haywire when they happen. Like "I've never seen a truck sideways before => it doesn't exist => boom."
Let's not overstate the problem here. There are plenty of AI things which would work well to recognize a sideways truck. Look at CLIP, which can also be plugged into DRL agents (per the cake); find an image of your pink truck and text prompt CLIP with "a photograph of a pink truck" and a bunch of random prompts, and I bet you it'll pick the correct one. Small-scale DRL trained solely on a single task is extremely brittle, yes, but trained over a diversity of tasks and you start seeing transfer to new tasks and composition of behaviors and flexibility (look at, say, Hide-and-Seek or XLAND).
These are all in line with the bitter hypothesis that much of what is wrong with them is not some fundamental problem that will require special hand-designed "generalization modules" bolted onto them by generations of grad students laboring in the math mines, but simply that they are still trained on too undiverse problems for too short a time with too little data using too little models, and that just as we already see strikingly better results in terms of generalization & composition & rare datapoints from past scaling, we'll see more in the future.
What goes wrong with Tesla cars specifically, I don't know, but I will point out that Waymo manages to kill many fewer people and so we shouldn't consider Tesla performance to even be SOTA on the self-driving task, much less tell us anything about fundamental limits to self-driving cars and/or NNs.
- mattnewton 5y ago> What goes wrong with Tesla cars specifically, I don't know, but I will point out that Waymo manages to kill many fewer people and so we shouldn't consider Tesla performance to even be SOTA on the self-driving task, much less tell us anything about fundamental limits to self-driving cars and/or NNs. Side note, but I think Waymo is treating this more like a JPL "moon landing" style problem and Tesla is trying to sell cars today. It's very different to start with making it possible and then scaling it down vs trying to build something working backwards from the sensors and compute economical to ship today.
- fxtentacle 5y agoMy theory is that gradient descent won't work for AGI and we'll need a completely new approach before we get safe general driving. But that's just my theory. "Let's not overstate the problem here. There are plenty of AI things which would work well to recognize a sideways truck" Yes, but nothing reliable enough to trust your life with. Clip also has some pretty weird bugs. https://aiweirdness.tumblr.com/post/660687015733559296/gallery-of-failed-unicorn-cakes/amp https://aiweirdness.tumblr.com/post/660687015733559296/galle...
- gwern 5y agoGradient descent has ushered out many challengers who were sure they could do things gradient descent could never do even on bigger compute. I'm not worried. (My own belief is that gradient descent is so useful that any better optimization approach will simply evolve gradient descent as an intermediate phase for bootstrapping problem-specific learning. It's a tower of optimizers all the way down/up.) You can't call that a 'CLIP bug' because using CLIP for gradient ascent on a diffusion model is not remotely what it was trained or intended to do, and is not much like your use-case of detecting real world objects. It's basically adversarial pixel-wise hacking, which is not what real world pink trucks are like. Also, that post was 7 months ago, and the AI art community advances really fast. (However bad you think those samples are, I assure you the first BigSleep samples, back in February 2021 when CLIP had just come out, would have been far worse.) 'Unicorn cake' may not have worked 7 months ago, but maybe it does now... Check out the Midjourney samples all over AI art Twitter the past month.