3 ms·

A fairly straight forward algorithm I thought of and have been tempted to test. Take the last few frames of video. Scale and skew them according to perspective

by NoToP 3y ago

A fairly straight forward algorithm I thought of and have been tempted to test. Take the last few frames of video. Scale and skew them according to perspective transform based on steering wheel angle and road speed. Average them together. What should be left is exactly the objects in frame that are truly on a collision path with the camera, all else should average away to grey blur.

t0mas88 3y ago
Collision avoidance in nearly all cars is radar based. And it includes steering angle input. The problem is when the car you're going to narrowly miss is still in the path of your car but going to move out of it. That's something a human could predict based on judgement, but it's very hard for a basic collision avoidance algorithm to predict. Humans also get this wrong. You see a car turning, you assume they keep turning and you'll pass just behind their rear bumper in that case. Now for a reason invisible to you the turning car hits their brakes and stops partially blocking your lane. You can't stop in the remaining distance. But would usually be able to steer around it by going to the edge of your lane or overlapping the next lane. But this isn't something a simple collision avoidance algorithm can do.
- NoToP 3y ago
  You haven't understood my simple algorithm which does exactly what you say a human driver is doing, no advanced ml required just basic understanding of projective geometry. If you have frames of video of the previous few fractions of a second and information on how the car moved in 3d space over those frames, I'm saying you can construct a dead simple filter that preserves precisely the in scene objects which are on a collision course with the camera while everything irrelevant in the scene gets washed out. The simplest case is with no steering. The projective transform is just blowing up the image by a multiplicative factor dependent on the distance moved per frame. So the x, y position of each pixel gets stretched the further back in time it is to compensate for the cars cha ge in parallax. After stretching, all these frames are averaged together. Anything on a collision course with the camera is going to be at the same angular position each time just getting larger. So averaging over will highlight its presence. The rear bumper which you are actually due to miss will be drifting in angular position in each compensated frame, so it gets averaged into the background blur of irrelevant objects. Tldr simple video frame filter highlights exactly which parts of image frame relevant to possible collision, tells computer what can safely ignore, implements exactly your near miss driving algorithm, no fragile machine learning bs.
robotresearcher 3y ago
This doesn’t account for the motion of other objects.
- NoToP 3y ago
  It does because the motion of the other objects is embedded in their position over the previous few frames. On the contrary, it is precisely filtering on moving objects such that the velocity vector is directly on a collision path with the camera. Objects which are moving but ultimately not on a collision course will be in different locations once the perspective transform is applied and thus average out to background. Objects which are moving on a collision course relative to the observer get amplified by this simple filter (irrespective of if the motion is absolute or not relative to ground).
  - robotresearcher 3y ago
    The perspective transform needs the depth of each pixel from the camera (or equivalent 3D Cartesian coordinates of objects). An affine image transform that ignores depth won’t project objects correctly.
  - deleted 3y ago
    [deleted]
  - jeffreygoesto 3y ago
    Let's assume ego and other vehicle are on a straight lane and move with constant speed along the same line. All trajectories where the other vehicle is not faster than ego are eventually colliding. But for each velocity that other vehicle projects to different positions, doesn't it? You can accumulate for one realtive motion only, but there are many dangerous ones. Taking ego speed and steering will accumulate standstill objects on the road plane (if the homography between image and road is accounter for properly). Clearly standstill objects in your path are on a collision course but the original problem was about moving and especially decelerating ones. Also it takes a large amount of frames to really average out. Pre-ML We used to implement multi-hypothesis Kalman filters and took the one with the least invention as best prediction.

by NoToP 3y ago

t0mas88 3y ago

Collision avoidance in nearly all cars is radar based. And it includes steering angle input. The problem is when the car you're going to narrowly miss is still in the path of your car but going to move out of it. That's something a human could predict based on judgement, but it's very hard for a basic collision avoidance algorithm to predict. Humans also get this wrong. You see a car turning, you assume they keep turning and you'll pass just behind their rear bumper in that case. Now for a reason invisible to you the turning car hits their brakes and stops partially blocking your lane. You can't stop in the remaining distance. But would usually be able to steer around it by going to the edge of your lane or overlapping the next lane. But this isn't something a simple collision avoidance algorithm can do.

NoToP 3y ago
You haven't understood my simple algorithm which does exactly what you say a human driver is doing, no advanced ml required just basic understanding of projective geometry. If you have frames of video of the previous few fractions of a second and information on how the car moved in 3d space over those frames, I'm saying you can construct a dead simple filter that preserves precisely the in scene objects which are on a collision course with the camera while everything irrelevant in the scene gets washed out. The simplest case is with no steering. The projective transform is just blowing up the image by a multiplicative factor dependent on the distance moved per frame. So the x, y position of each pixel gets stretched the further back in time it is to compensate for the cars cha ge in parallax. After stretching, all these frames are averaged together. Anything on a collision course with the camera is going to be at the same angular position each time just getting larger. So averaging over will highlight its presence. The rear bumper which you are actually due to miss will be drifting in angular position in each compensated frame, so it gets averaged into the background blur of irrelevant objects. Tldr simple video frame filter highlights exactly which parts of image frame relevant to possible collision, tells computer what can safely ignore, implements exactly your near miss driving algorithm, no fragile machine learning bs.

robotresearcher 3y ago

This doesn’t account for the motion of other objects.

NoToP 3y ago
It does because the motion of the other objects is embedded in their position over the previous few frames. On the contrary, it is precisely filtering on moving objects such that the velocity vector is directly on a collision path with the camera. Objects which are moving but ultimately not on a collision course will be in different locations once the perspective transform is applied and thus average out to background. Objects which are moving on a collision course relative to the observer get amplified by this simple filter (irrespective of if the motion is absolute or not relative to ground).
- robotresearcher 3y ago
  The perspective transform needs the depth of each pixel from the camera (or equivalent 3D Cartesian coordinates of objects). An affine image transform that ignores depth won’t project objects correctly.
- deleted 3y ago
  [deleted]
- jeffreygoesto 3y ago
  Let's assume ego and other vehicle are on a straight lane and move with constant speed along the same line. All trajectories where the other vehicle is not faster than ego are eventually colliding. But for each velocity that other vehicle projects to different positions, doesn't it? You can accumulate for one realtive motion only, but there are many dangerous ones. Taking ego speed and steering will accumulate standstill objects on the road plane (if the homography between image and road is accounter for properly). Clearly standstill objects in your path are on a collision course but the original problem was about moving and especially decelerating ones. Also it takes a large amount of frames to really average out. Pre-ML We used to implement multi-hypothesis Kalman filters and took the one with the least invention as best prediction.