5 ms·

As someone who is a decent-ish programmer that's recently worked on recognition from videos, I would be totally lost trying to figure out how to handle a situat

As someone who is a decent-ish programmer that's recently worked on recognition from videos, I would be totally lost trying to figure out how to handle a situation like this. As a human person (I swear!), I could see in the video that the headlights were being obstructed by something but, as a human programmer, I'm completely at a loss for how I would even determine if the headlights are blocked by something up close vs. far away using just the camera's view. Headlights get blocked by things all the time. How in the hell would you figure out that that's what's going on just by the visual? In other words, it's a good thing I don't make this stuff. My car would be phantom braking every 10 seconds, especially at night.

a_square_peg 5y ago
Yeah, I've wondered about this too. If cars in the adjacent lanes slightly ahead of me braked all of sudden, I would brake too even if the road ahead of me is completely clear because I would think there is something going on. I wonder know how this type of thing gets programmed in to self-driving vehicles.
- dkonofalski 5y ago
  Spoiler alert: It doesn't. It's almost completely based on the what the vehicle in front of you is doing.
jazzyjackson 5y ago
I would definitely want night vision infrared, but my main strategy would be making sure the texture of the road surface is flowing toward me at the expected speed. I imagine I could use something like the motion vectors in mpeg compression. I would exploit the fact that the road 1000 feet away is coming at me slower than the road 100 feet away: if there is a visual texture in the middle of my field of view (where the horizon should be: road approaching relatively slowly), and it’s coming at me faster than 1000-feet-away asphalt ought to be, I assume the road is obstructed and slow down until I get a clear view again. Granted, I am a lowly web programmer with a hobbyist interest in compression, but I have thought about getting the comma.ai kit and learning how it works, trying to implement my own world modeling, but, sounds like a full time job :) EDIT: as for the headlights, I don’t want to act like I could whip this up in a weekend or anything, but having objects flicker in and out of existence would at least subtract a few points from my confidence score.
- dkonofalski 5y ago
  That's an interesting way to go about it but would the texture of the road surface be consistent enough and reliable enough to make those calculations? This is the kind of stuff I'm talking about. I definitely do not have the confidence to do that. It's why machine learning is so scary to me. We're basically saying "I can't figure out all the possible permutations of this so let's throw them at a computer and hope it can do a better job".
  - jazzyjackson 5y ago
    Can you tell me more about what kind of road (in)consistency you’re describing? It’s fun for me to think of how to improve this strategy. I am similarly opposed/nervous around machine learning, seems impossible to debug weird decisions the black box makes, I would prefer a traditional, deterministic computer vision approach. Basically I want to segment incoming images into a grid and do a cosine similarity score between each cell and its neighbors one frame ago, to detect which direction and how quickly all the textures are moving. So I think it should be robust to whatever the road texture is, provided its not pitch black like wet asphalt.
  - avereveard 5y ago
    > consistent enough you want the opposite, if it's consistent, then it's more difficult to gauge it's motion just by finding similar zones. think on mouse on glass or metal, it's too uniform for the sensor to detect motion. anyway: paper https://dspace.mit.edu/bitstream/1721.1/6298/2/AIM-476.pdf https://dspace.mit.edu/bitstream/1721.1/6298/2/AIM-476.pdf lecture: https://www.youtube.com/watch?v=0BVZDyRrYtQ https://www.youtube.com/watch?v=0BVZDyRrYtQ usually circular motion is used, but linear works just as fine
michael1999 5y ago
The biggest change I’ve made over 35 years of driving is slowing more and sooner when things are unclear. At 75, just taking your foot off the accelerator will shed 10-20% of stopping distance in a few seconds. Waiting until you are sure it is bad news is the difference between defensive driving and avoidable crashes.
- dkonofalski 5y ago
  >At 75, just taking your foot off the accelerator will shed 10-20% of stopping distance in a few seconds. Even more in a Tesla as the regen braking will slow you down a lot. I drive almost exclusively with 1 pedal now because of it and, while I don't know that that actually is safer, it feels safer because I'm slowing down waaaay earlier than I normally would in an ICE-mobile.
everforward 5y ago
It's not going to be reliable with just cameras. You can use where the headlights disappear and reappear to draw an arc of occluded space. You should be able to measure how quickly that arc changes to estimate the distance If you're 1000 feet from an object and move a foot closer, it barely blocks anything else from view. If you're 5 feet away, moving a foot closer blocks a lot more of your view. The problem is that relies on nearly constant backlighting on the object. The real solution here is LIDAR, which makes this trivial. Tesla won't use it though, so they have stupid problems like trying to hack a way to determine how far away dark objects are.
- dkonofalski 5y ago
  Exactly. I was just going off of Musk's statements that they don't want to use things like LIDAR since people don't have LIDAR but can make these types of assessments. Even if you could make an arc of occlusion (sounds like a D&D spell), there's way too much variability in headlight sizes/designs to accurately determine distance in the dark in the situation this video presents. Again, I am admitting a ton of ignorance when making that statement but that seems nearly impossible to me with just cameras.

a_square_peg 5y ago

Yeah, I've wondered about this too. If cars in the adjacent lanes slightly ahead of me braked all of sudden, I would brake too even if the road ahead of me is completely clear because I would think there is something going on. I wonder know how this type of thing gets programmed in to self-driving vehicles.

dkonofalski 5y ago
Spoiler alert: It doesn't. It's almost completely based on the what the vehicle in front of you is doing.

jazzyjackson 5y ago

I would definitely want night vision infrared, but my main strategy would be making sure the texture of the road surface is flowing toward me at the expected speed. I imagine I could use something like the motion vectors in mpeg compression. I would exploit the fact that the road 1000 feet away is coming at me slower than the road 100 feet away: if there is a visual texture in the middle of my field of view (where the horizon should be: road approaching relatively slowly), and it’s coming at me faster than 1000-feet-away asphalt ought to be, I assume the road is obstructed and slow down until I get a clear view again. Granted, I am a lowly web programmer with a hobbyist interest in compression, but I have thought about getting the comma.ai kit and learning how it works, trying to implement my own world modeling, but, sounds like a full time job :) EDIT: as for the headlights, I don’t want to act like I could whip this up in a weekend or anything, but having objects flicker in and out of existence would at least subtract a few points from my confidence score.

dkonofalski 5y ago
That's an interesting way to go about it but would the texture of the road surface be consistent enough and reliable enough to make those calculations? This is the kind of stuff I'm talking about. I definitely do not have the confidence to do that. It's why machine learning is so scary to me. We're basically saying "I can't figure out all the possible permutations of this so let's throw them at a computer and hope it can do a better job".
- jazzyjackson 5y ago
  Can you tell me more about what kind of road (in)consistency you’re describing? It’s fun for me to think of how to improve this strategy. I am similarly opposed/nervous around machine learning, seems impossible to debug weird decisions the black box makes, I would prefer a traditional, deterministic computer vision approach. Basically I want to segment incoming images into a grid and do a cosine similarity score between each cell and its neighbors one frame ago, to detect which direction and how quickly all the textures are moving. So I think it should be robust to whatever the road texture is, provided its not pitch black like wet asphalt.
- avereveard 5y ago
  > consistent enough you want the opposite, if it's consistent, then it's more difficult to gauge it's motion just by finding similar zones. think on mouse on glass or metal, it's too uniform for the sensor to detect motion. anyway: paper https://dspace.mit.edu/bitstream/1721.1/6298/2/AIM-476.pdf https://dspace.mit.edu/bitstream/1721.1/6298/2/AIM-476.pdf lecture: https://www.youtube.com/watch?v=0BVZDyRrYtQ https://www.youtube.com/watch?v=0BVZDyRrYtQ usually circular motion is used, but linear works just as fine

michael1999 5y ago

The biggest change I’ve made over 35 years of driving is slowing more and sooner when things are unclear. At 75, just taking your foot off the accelerator will shed 10-20% of stopping distance in a few seconds. Waiting until you are sure it is bad news is the difference between defensive driving and avoidable crashes.

dkonofalski 5y ago
>At 75, just taking your foot off the accelerator will shed 10-20% of stopping distance in a few seconds. Even more in a Tesla as the regen braking will slow you down a lot. I drive almost exclusively with 1 pedal now because of it and, while I don't know that that actually is safer, it feels safer because I'm slowing down waaaay earlier than I normally would in an ICE-mobile.

everforward 5y ago

It's not going to be reliable with just cameras. You can use where the headlights disappear and reappear to draw an arc of occluded space. You should be able to measure how quickly that arc changes to estimate the distance If you're 1000 feet from an object and move a foot closer, it barely blocks anything else from view. If you're 5 feet away, moving a foot closer blocks a lot more of your view. The problem is that relies on nearly constant backlighting on the object. The real solution here is LIDAR, which makes this trivial. Tesla won't use it though, so they have stupid problems like trying to hack a way to determine how far away dark objects are.

dkonofalski 5y ago
Exactly. I was just going off of Musk's statements that they don't want to use things like LIDAR since people don't have LIDAR but can make these types of assessments. Even if you could make an arc of occlusion (sounds like a D&D spell), there's way too much variability in headlight sizes/designs to accurately determine distance in the dark in the situation this video presents. Again, I am admitting a ton of ignorance when making that statement but that seems nearly impossible to me with just cameras.