4 ms·
1000fps+ allows you to have a larger sequence of decisions "to vote" for the next action taken with higher certainty and lower decision latency. Imagine you hav
by bitL 7y ago
1000fps+ allows you to have a larger sequence of decisions "to vote" for the next action taken with higher certainty and lower decision latency. Imagine you have a 5fps self-driving car system; if you drive fast, one frame in 0.2s must give you a correct decision for a quite large travel distance; with 1000fps you can chain e.g. 10 decisions in a row via heatmap or Bayesian rule and get a pretty good certainty about what is going on around you and what action needs to be taken right now. In perception you hit false positives/negatives all the time but appearance of the same issue on two-three consecutive frames goes to 0 pretty quickly.
- nmca 7y agoI... Don't think that's how this works. It helps, for sure, but not nearly as much as you think, because consecutive frames in a video are not iid rvs
- bitL 7y agoDoes it really matter? UKF & PF work quite well even without any IID RV requirements. Where can I learn more about this relationship in the aforementioned context?
- nickodell 7y ago>In perception you hit false positives/negatives all the time but appearance of the same issue on two-three consecutive frames goes to 0 pretty quickly. If the frames are all very similar, wouldn't the probability of failure be correlated? In other words, if I feed a picture of a panda to a NN, and it incorrectly classifies it, why would it be correct the second time around?
- bitL 7y agoSure, but I bet Andrej Karpathy's team is using SOTA and mitigates most of the issues like the ones you mentioned. OK it's an "argument from authority" one might believe or not and I doubt they'd ever disclose things that make it work in their specific case. But realistically, if you e.g. observe results of real-time semantic segmentation, you see that surfaces flip a bit in each frame but mostly stay correct; you can e.g. use average IoU/coverage from past n frames to estimate what is going on. They have a fleet of cars driving around receiving more training data in all kinds of environments to handle various perceptual conditions as well.
- nickodell 7y ago>Sure, but I bet Andrej Karpathy's team is using SOTA and mitigates most of the issues like the ones you mentioned. Can you give an example of an image classification technique where classification failures are independent, or independent enough?
- bitL 7y agoTime-distributed 3D convolutions/attention-enhanced 2D CNNs, i.e. you look at a sequence of images at once, not at a single image.
- pjc50 7y ago> In perception you hit false positives/negatives all the time but appearance of the same issue on two-three consecutive frames goes to 0 pretty quickly. This does not sound like a technology that can be used to produce predictably safe results.