6 ms·

Using neural networks to detect car crashes in dashcam footage

119 points by gk1 9y ago

amelius 9y ago
The author could also have used 3d simulations to obtain more data, and determine feasibility and proper hyperparameters for the network before training with actual data.
- minimaxir 9y ago
  3D simulations can’t create photorealistic images. (Not currently, anyways)
  - bkberry352 9y ago
    You don't need photorealistic images for these situations. The images are always downsampled to a smaller resolution. So getting "good enough" but not photorealistic is fine.
    - minimaxir 9y ago
      If you have software to simulate real-world physics for hundreds of independent agents that can be run on a personal computer with close to real world graphic fidelity (including realistic noise), you’re running a multibilion dollar company, not writing thought pieces on Medium.
      - nl 9y ago
        There are a few companies you can get simulation software from specifically designed for self-driving cars. See https://blogs.nvidia.com/blog/2017/10/26/self-driving-simulation/ https://blogs.nvidia.com/blog/2017/10/26/self-driving-simula... for some discussion. It's true that this isn't photo-realistic (see https://www.youtube.com/watch?v=D7OccC0W_CU https://www.youtube.com/watch?v=D7OccC0W_CU for a video), but at the same time it is useful enough for training.
  - dharma1 9y ago
    There were a couple of projects using GTA 5 for training data successfully (one from OpenAI) but they got taken down due to licensing issues
  - shoyer 9y ago
    One option is to train a GAN to do domain adaptation. Google (disclaimer: I work there) recently did that very successfully for training robots to grasp: https://research.googleblog.com/2017/10/closing-simulation-to-reality-gap-for.html?m=1 https://research.googleblog.com/2017/10/closing-simulation-t...
  - amelius 9y ago
    Reread my comment. I didn't suggest to only use the simulations.
  - tomdre 9y ago
    Check out "domain adaptation" https://en.wikipedia.org/wiki/Domain_adaptation https://en.wikipedia.org/wiki/Domain_adaptation
djaychela 9y ago
As someone who's interested (but not remotely capable yet) in this sort of thing, I think it would have been nice to see some more detail as to how the system worked - this seemed more of an executive summary to me rather than giving any significant technical details; is this true or is it my lack of knowledge showing?
- deleted 9y ago
  [deleted]
- sillysaurus3 9y ago
  Nah, you're right. It'd be cool to have a codecademy-type course on getting a project like this up and running. I'd sign up for $10 or so.
  - joshvm 9y ago
    http://www.fast.ai/ http://www.fast.ai/ is what you want, and it's free. Or you can look at Stanford's CS231N for the heavy details.
- Drdrdrq 9y ago
  To me this level of detail is perfect - but I do have some prior knowledge on the subject. I think it is difficult to put such a project in writing in more detail because many choices (framework, NN architecture, data cleanup procedures...) make sense only with data which probably can't be shared. So I think author did a great job selecting those pieces of information which are relevant to general public. Btw, there are courses, tutorials and challenges online if you ever wish to learn more... It is fairly easy to get good info, but it takes time to grasp concepts. It is fun though and enables you to solve a whole new class of programming challenges. :)
  - sillysaurus3 9y ago
    Btw, there are courses, tutorials and challenges online if you ever wish to learn more... It is fairly easy to get good info, but it takes time to grasp concepts. It is fun though and enables you to solve a whole new class of programming challenges. :) Any tips? Which classes would you recommend?
kakali 9y ago
Can't they just use an accelerometer?
- gumby 9y ago
  The key factors are 1> reducing BOM and 2> working in cases where you have whatever data are available (perhaps a phone or CCTV footage or whatever). As it happens most if not all dash cams have accelerometers.
- keenrodent 9y ago
  They are not crash-testing automobiles, so an accelerometer won't help. They are looking at dashcam footage from a library of that footage.
- mikeash 9y ago
  It looks like they are detecting crashes of other cars caught on camera, not crashes of the car with the camera.
deleted 9y ago
[deleted]
kalal 9y ago
I would like to see how the system generalizes on different videos. The motion pattern seems to be quite general ... this can be used to analyze crashes during racing events for instance (http://www.tldvision.com/tld3.html http://www.tldvision.com/tld3.html).
matt4077 9y ago
The source, as well as more in-depth description, is available at https://github.com/rwk506/CrashCatcher https://github.com/rwk506/CrashCatcher This is actually quite neat! Having little data and not enough resources to manually label thousands of examples seems to be a somewhat neglected problem. Maybe we should boycott Google's captchas until they feel the same pain :) The comments in the code do read like a parody, though: count = 0 ### start a counter at zero
- vlasev 9y ago
  No kidding... while success: ### while success == True
- chestervonwinch 9y ago
  Aside from the necessity of the comments, I notice they have many comment blocks that are to the right of the code block, but left-aligned to some column number. Is this easy in some editor? I can't imagine doing this in vim without some extra fanciness in the vimrc, for example.
- nl 9y ago
  Having little data and not enough resources to manually label thousands of examples seems to be a somewhat neglected problem. It's not a neglected problem - it is an extremely active area of research, and something people deal with every single day when using ML in industry (do you feel my frustration there?!) The problem is that there isn't a single generalization method that works well. For specific problems there are numerous approaches which can help. Distant supervision, adversarial techniques, data augmentation, reinforcement learning etc are all areas to look at - but the specific technique depends on your exact problem.
rasz 9y ago
80% for 4 second clips, so running it on random video feed guarantees false positives in under a minute. Not to mention this is trying to predict something human expert in the field (lets say a taxi driver) watching same clips couldnt do, its trying to predict future. Most likely triggers on something semi related, like amount of cars in front/roundabouts/red light, more cars = bigger chance of accident = strong correlation. Garbage in garbage out.

119 points by gk1 9y ago

The author could also have used 3d simulations to obtain more data, and determine feasibility and proper hyperparameters for the network before training with actual data.

minimaxir 9y ago
3D simulations can’t create photorealistic images. (Not currently, anyways)
- bkberry352 9y ago
  You don't need photorealistic images for these situations. The images are always downsampled to a smaller resolution. So getting "good enough" but not photorealistic is fine.
  - minimaxir 9y ago
    If you have software to simulate real-world physics for hundreds of independent agents that can be run on a personal computer with close to real world graphic fidelity (including realistic noise), you’re running a multibilion dollar company, not writing thought pieces on Medium.
    - nl 9y ago
      There are a few companies you can get simulation software from specifically designed for self-driving cars. See https://blogs.nvidia.com/blog/2017/10/26/self-driving-simulation/ https://blogs.nvidia.com/blog/2017/10/26/self-driving-simula... for some discussion. It's true that this isn't photo-realistic (see https://www.youtube.com/watch?v=D7OccC0W_CU https://www.youtube.com/watch?v=D7OccC0W_CU for a video), but at the same time it is useful enough for training.
- dharma1 9y ago
  There were a couple of projects using GTA 5 for training data successfully (one from OpenAI) but they got taken down due to licensing issues
- shoyer 9y ago
  One option is to train a GAN to do domain adaptation. Google (disclaimer: I work there) recently did that very successfully for training robots to grasp: https://research.googleblog.com/2017/10/closing-simulation-to-reality-gap-for.html?m=1 https://research.googleblog.com/2017/10/closing-simulation-t...
- amelius 9y ago
  Reread my comment. I didn't suggest to only use the simulations.
- tomdre 9y ago
  Check out "domain adaptation" https://en.wikipedia.org/wiki/Domain_adaptation https://en.wikipedia.org/wiki/Domain_adaptation

djaychela 9y ago

As someone who's interested (but not remotely capable yet) in this sort of thing, I think it would have been nice to see some more detail as to how the system worked - this seemed more of an executive summary to me rather than giving any significant technical details; is this true or is it my lack of knowledge showing?

deleted 9y ago
[deleted]
sillysaurus3 9y ago
Nah, you're right. It'd be cool to have a codecademy-type course on getting a project like this up and running. I'd sign up for $10 or so.
- joshvm 9y ago
  http://www.fast.ai/ http://www.fast.ai/ is what you want, and it's free. Or you can look at Stanford's CS231N for the heavy details.
Drdrdrq 9y ago
To me this level of detail is perfect - but I do have some prior knowledge on the subject. I think it is difficult to put such a project in writing in more detail because many choices (framework, NN architecture, data cleanup procedures...) make sense only with data which probably can't be shared. So I think author did a great job selecting those pieces of information which are relevant to general public. Btw, there are courses, tutorials and challenges online if you ever wish to learn more... It is fairly easy to get good info, but it takes time to grasp concepts. It is fun though and enables you to solve a whole new class of programming challenges. :)
- sillysaurus3 9y ago
  Btw, there are courses, tutorials and challenges online if you ever wish to learn more... It is fairly easy to get good info, but it takes time to grasp concepts. It is fun though and enables you to solve a whole new class of programming challenges. :) Any tips? Which classes would you recommend?

kakali 9y ago

Can't they just use an accelerometer?

gumby 9y ago
The key factors are 1> reducing BOM and 2> working in cases where you have whatever data are available (perhaps a phone or CCTV footage or whatever). As it happens most if not all dash cams have accelerometers.
keenrodent 9y ago
They are not crash-testing automobiles, so an accelerometer won't help. They are looking at dashcam footage from a library of that footage.
mikeash 9y ago
It looks like they are detecting crashes of other cars caught on camera, not crashes of the car with the camera.

deleted 9y ago

[deleted]

kalal 9y ago

I would like to see how the system generalizes on different videos. The motion pattern seems to be quite general ... this can be used to analyze crashes during racing events for instance (http://www.tldvision.com/tld3.html http://www.tldvision.com/tld3.html).

matt4077 9y ago

The source, as well as more in-depth description, is available at https://github.com/rwk506/CrashCatcher https://github.com/rwk506/CrashCatcher This is actually quite neat! Having little data and not enough resources to manually label thousands of examples seems to be a somewhat neglected problem. Maybe we should boycott Google's captchas until they feel the same pain :) The comments in the code do read like a parody, though: count = 0 ### start a counter at zero

vlasev 9y ago
No kidding... while success: ### while success == True
chestervonwinch 9y ago
Aside from the necessity of the comments, I notice they have many comment blocks that are to the right of the code block, but left-aligned to some column number. Is this easy in some editor? I can't imagine doing this in vim without some extra fanciness in the vimrc, for example.
nl 9y ago
Having little data and not enough resources to manually label thousands of examples seems to be a somewhat neglected problem. It's not a neglected problem - it is an extremely active area of research, and something people deal with every single day when using ML in industry (do you feel my frustration there?!) The problem is that there isn't a single generalization method that works well. For specific problems there are numerous approaches which can help. Distant supervision, adversarial techniques, data augmentation, reinforcement learning etc are all areas to look at - but the specific technique depends on your exact problem.

rasz 9y ago

80% for 4 second clips, so running it on random video feed guarantees false positives in under a minute. Not to mention this is trying to predict something human expert in the field (lets say a taxi driver) watching same clips couldnt do, its trying to predict future. Most likely triggers on something semi related, like amount of cars in front/roundabouts/red light, more cars = bigger chance of accident = strong correlation. Garbage in garbage out.