3 ms·
When self-driving cars were first becoming a thing, I was one of the skeptics who would say things like, "Call me when the AI can avoid situations like https://
by CamperBob2 3y ago
When self-driving cars were first becoming a thing, I was one of the skeptics who would say things like, "Call me when the AI can avoid situations like https://i.imgur.com/cmR2PDm.jpeg https://i.imgur.com/cmR2PDm.jpeg ." At the time, my thinking was that the software would have to recognize both fixed and moving objects nearby, capture their position and velocity, infer or look up their inertia tensors, and run at least a basic dynamics simulation to anticipate and prevent accidents like the one that's about to happen in the photo.
However, I didn't consider the possibility that such a simulation might emerge from a model driven by nothing but visual input, given a large-enough data set and the right training. At this point my old argument seems like a losing one, even if present-day systems can't answer the "What happens next?" question reliably enough to trust in all driving situations. It won't exactly have to be perfect in order to outperform most humans.
As a matter of fact, I just checked, and the situation in the meme is already partially recognized by ChatGPT 4: https://i.imgur.com/wLSBSkJ.png https://i.imgur.com/wLSBSkJ.png , even if it misses the comedic implications of the truck heading for the overpass. Whether it was somehow trained to give a useful, actionable answer for this particular question, I don't know, but the writing's on the proverbial wall.
-------
Edit due to rate-limiting: note that I intentionally cropped out the WAIT FOR IT frame when I submitted the pic, so that it wouldn't see it as a meme loaded with comedy value. When I explicitly asked it what was funny about the image, ChatGPT4 eventually answered:
"When the truck hits the overpass, given that the convertible is following closely behind, the occupants could be in for quite a shock. The impact might cause the portable toilets to become dislodged or damaged, potentially resulting in the contents being spilled. For the people in the convertible, this could mean an unexpected and unpleasant shower, especially because the convertible's top is down. The humor perceived here is in the anticipation of the unexpected and the sudden reversal of fortune for the individuals in what appears to be a fancy car – it's a form of slapstick humor."
But I did have to provide a couple of strong hints, including telling it that the truck was not towing the car ( https://i.imgur.com/AlPRgEQ.png https://i.imgur.com/AlPRgEQ.png ). I was going to feed Claude the same hints, but it repeatedly failed to recognize the SMS code for registration.
- minimaxir 3y agoVideo data doesn't work well with these multimodal LLMs because images are still somewhat intensive for them process. Both the GPT-4-Vision demos and Gemini take a few frames from each video (~1 frame per second). Self-driving cars use much more efficient approaches for the real-time outputs needed.
- CamperBob2 3y agoSure, but my point is that self-driving may be easier to achieve if we don't insist on doing it the hard way with hardwired logic running task-specific models, the way I originally assumed it would have to be done.
- HellDunkel 3y agoI thought about this in the exact same way when there was a lot of talk about end to end learning in the early days. LLMs made me feel unsure too but i am still not conviced.
- habitue 3y agoNot to detract from your point, because I think you're right, but gpt4 doesn't seem to mention the biggest thing a human wpuld notice, which is that the portable toilets are about to be knocked off the truck by driving under the bridge. still, impressive
- sfn42 3y agoHow do we know the cars are moving? It doesn't seem clear to me, they could both be standing still.
- deleted 3y ago[deleted]
- CamperBob2 3y agoIt's a valid question, but I tried to account for it by asking "What is about to happen?" rather than just "What does this photo portray?" If I had asked the latter question, it would have been acceptable for the model to assume the cars are parked, I think.