3 ms·
This is not good enough to handle edge cases. It's a little bit like ChatGPT: ChatGPT is a next-word prediction engine. It has ingested and analyzed "human dat
by sho_hn 4y ago
This is not good enough to handle edge cases.
It's a little bit like ChatGPT: ChatGPT is a next-word prediction engine. It has ingested and analyzed "human data" which has given it a take on what might come after a prompt. With good training data, a large enough token window and a helping of RLHF this can do very cool things. But there's no actual reasoning going on.
An "edge case" is a prompt that throws this approach for a spin, because statistics over the training data corpus don't yield a good prediction on it. For example because it's novel or its structure is uncommon. In those cases ChatGPT will do its best and confidently return something very dumb.
With the driving problem, Tesla and others are trying to mitigate this with two approaches:
- superstructures of rule-based planning to keep the predictions within safe lanes (no pun intended), i.e. bake in some reasoning after all
- trying to write simulations that auto-generate "edge cases" or just stuff that is underrepresented in the training data set and add those simulated 3d renders to editorialize the predictions
It's currently still unclear if this is enough to solve the problem or if we're still some innovations (e.g. ones that get us closer to AGI) away from being able to make it work.
- yowzadave 4y agoI wonder if self-driving doesn't actually require reasoning, though. Most of us are able to drive instinctively, without fully engaging all of our attention; we certainly aren't solving problems with deep analytical thinking. Perhaps it's a problem that's well-suited to the approach an AI like ChatGPT takes?
- gamegoblin 4y agoI agree. I go weeks at a time without having to do any critical thinking whatsoever. There probably are some situations that do actually require a looping reasoning process that a simple feed-forward neural network can't do, but they are fairly rare. The most common ones probably actually involve complex parking lot navigation tasks, where you have to negotiate the space with other drivers, understand their intentions, yield space, take space, etc.
- archagon 4y agoYou may end up using reasoning more often than you think. I run into situations that require my full attention and are very likely un-AI-able once a week or so. (And I live in California.)
- sho_hn 4y ago> Most of us are able to drive instinctively, without fully engaging all of our attention; we certainly aren't solving problems with deep analytical thinking Depends. Many of the type of situations where FSD runs into trouble or disengages are also the ones where a human would slow down/pause and ponder a moment what to do next. In the FSD case, it then often eventually gives up and disengages, because it can't confidently predict what action to take. Humans show better performance in those cases. If you ask me what separates a human from ChatGPT right now, it's that ability to use reasoning to fill in for bad training data. If, as some people have posited, the training data for ChatGPT-like systems is possibly already exhausted, we need some new ideas. But humans are good at that. :-) Aside from that, note also that your learned instinctive predictions take into account many more parameters than the models a Tesla currently runs anyway. For example you have an understanding of social dynamics (so what those other human drivers think and what might be acting on them, etc.) that is far in excess of what the Tesla models can learn implicitly from videos that observe them driving. With some of these things we have no idea yet how to feed them into the models.
- moralestapia 4y ago>This is not good enough to handle edge cases. Is Tesla doing any better, though? At least comma can handle the "huge unmovable object in front of me, I should brake" edge case.
- sho_hn 4y agoOP claimed that Comma's approach would be able to handle edge cases better, but I don't think it's fundamentally a solution to that problem. Tesla is struggling with the same tech limitations certainly.
- gamegoblin 4y agoI think it will handle edges better because it's explicitly not trying to classify everything into objects (for the purpose of loading into their virtual representation). So it doesn't need to "understand" a situation in terms of what is physically true, it just needs to understand what information is valuable for the task of driving. Consider this edge case: https://www.youtube.com/shorts/kM-xBgz26pE https://www.youtube.com/shorts/kM-xBgz26pE A utility truck carrying a load of new traffic lights. Tesla correctly identifies them as traffic lights and loads virtual representations into its world model. It doesn't seem to cause any weird behavior (thankfully), but this demonstrates a case where doing this fancy classification actually makes your system less robust.
- sho_hn 4y agoI'd be interested in a direct comparison of how Comma's models handle the same situation. Has someone tried?
- moralestapia 4y agoI know, and I'm saying that whatever Tesla is doing, along with those "mitigations" you mentioned, is crap because of the several crashes where it has failed to detect big stationary objects in front of the car. I haven't seen comma doing stuff as dumb as that, so I'm also betting on comma's approach.