12 ms·

The Next Leap in Self-Driving: Prediction

107 points by edward 7y ago

nihonium 7y ago
I completely agree. If prediction algorithms properly implemented, the Uber crash could be avoided. I think the next step after proper visual prediction is interpreting the noise around. For example, a human can hear the ambulance siren approaching and slow down in a junction without seeing the vehicle. I think sound interpretation is very important especially in city driving.
- Retric 7y ago
  I don’t think accurate prediction is the right long term strategy. It’s edge cases that cause accidents, so a more adversarial approach is safer. Taken to the extreme this causes problems, but gently slowing down in the case of extreme tailgating is a response to slow reaction times not direct path prediction.
  - mellosouls 7y ago
    It's edge cases that will leave 100% self-driving unlikely in the near future.
  - AstralStorm 7y ago
    Adversarial approach to driving results in staying at home, because you're likely to be rammed by a bad driver from behind, and you should almost never brake and drive always as fast as possible. Alternatively always stop when you see any pedestrian on the walkway, because they will drive in front of your car. Driving is cooperative not adversarial, even certain classes of illegal behavior are predictable.
- lm28469 7y ago
  > If prediction algorithms properly implemented, the Uber crash could be avoided. If they didn't willingly disable safety features the crash wouldn't have happened. It wouldn't have happened with the shittiest auto emergency braking system that relies on 0 Ai and 0 predictions.
- sonthonax 7y ago
  If Uber had bothered to follow a basic safety protocol that person would still be alive today. Staying alert on watch is something that nearly every solider, security guard, sailor, will have to do. Where you basically stare into the dark in case someone comes. There are a number of ways to mitigate the inevitable failure of attention: * Having a parter; * Doing short shifts (change with partner); * Very simple non-distracting tasks - like reading out to your parter a figure at a glance. You certainly should not: * Be on your own; * Have a mobile phone on you (all screens should be locked away while moving). It's so fucking arrogant that Uber didn't care to implement the most obvious risk mitigation. The cost of an additional person on watch is negligible compared to the cost of their self driving care program. My gut feeling is Uber simply misled regulators with how developed their self driving car program is, so it could continue to mislead investors.
zie1ony 7y ago
"It's hard to make predictions - especially about the future." — Robert Storm Petersen
irjustin 7y ago
Predicting is absolutely key. If you've been around the Waymo cars they horribly manage simple situations. They're VERY safe because they take no risk, but that won't work at scale. "What's that? Stop wait..., what's that? stop... wait wait wait" They're very jerky at intersections where there's even a mild amount of uncertainty and really only go when it's very clear. It reminds me of lots of first time drivers reacting to new situations where safe is the best way forward. But if we had pure first time drivers, we'd be a gridlock at even a mild amount of traffic that enters and leaves the roadway. Looking forward to this next set of 5 years. Hopefully us drivers will get replaced in 10? maybe?
- kemotep 7y ago
  I truly believe my children (not born yet) are going to buy their first car and it will be fully autonomous. Too many people my age and older will be apprehensive about giving up control, despite it likely being safer. So I think it is fair to say that in 20 years there will be many people who learned to "drive" with a self driving car and their parents and grandparents will refuse to give up driving themselves.
  - smileysteve 7y ago
    Insurance and police enforcement are two reasons why the decision will be removed from law abiding citizens and won't happen linearly. Once proven safe (or if a manufacturer directly insures the product) the cost to adequately insure a vehicle (as prices of what you hit continue to increase), especially an older vehicle, will increase. With income inequality where it is (and state minimums lagging) it has already become common to have underinsured drivers (and the upper middle class are already paying more through uninsured motorist protection). When it comes to police enforcement, but especially jurisdictions that use traffic stops to find or create crime to fundraise; the expense of having traffic police in a world where > 50% of cars can do no wrong flips the model. Finally, there is a large group of people who will not drive a car that's 7 years old or has over 100k miles; combining with less trained ability to do maintenance on more computerized cars. These people claim to be in control, but at one check engine light away from doing whatever a dealer tells them.
mightybyte 7y ago
I could be missing something, but this sounds like a horrible idea. I've had two accidents in my life. One of them was caused by inattention--hooray that self driving cars have the potential to wipe this cause out. The other was caused because I was a new driver and I predicted what the car in front of me would do and it didn't do it. It was in my first year of driving and fortunately it ended up being a very minor collision, but I distinctly remember that my conscious takeaway was that I shouldn't try to predict what other cars will do. If my car is driving, instead of trying to predict the behavior of other objects I want it to do something like a minimax search and do something that will be safe in the presence of the unexpected, not the expected. Prediction sounds like the complete opposite of good driving practices like the rule of following at least 2 seconds behind the car in front of you, which is specifically there to prevent accidents in the rare case where the car in front of you on the freeway does NOT behave in the predicted way of continuing to go 70 mph in the middle of the lane.
- zelos 7y ago
  I think it's another stream of data feeding the model. For example: you see a child walking along close to the edge of the road, not looking in your direction. You don't slam your brakes on, but as a driver your prediction is that he may step in front of you, so you ease back on the accelerator, move out further into the middle of the road, cover the brake.
- kdmccormick 7y ago
  I think "prediction" in this context means something different. It's not specifically that self-driving cars will optimistically assume they know what other drivers are doing, like a naive driver might (although that is a form of prediction). It's more generally that in order to make any smart decisions at all, the AI has to model the future. For example, if you are driving behind a car on the highway, and the car _in front_ of that car puts their brake lights on, you may want to pre-emptively start braking now now. That's a type of pro-safety prediction made based on your understanding of the likely future that driving AIs should also have.
  - quietbritishjim 7y ago
prattatx 7y ago
Context matters. For example, at an intersection, both the context of what is happening with others at that intersection as well as what has happened to the vehicle coming up to that intersection matter. An intersection near a soccer field at 4PM has different dynamics than at 4AM. An intersection following a highway exit has different dynamics than an intersection in a neighborhood.
- zubspace 7y ago
  You're right. And that's why I can't imagine a solution without AGI. There are simply too many context-sensitive parameters and we humans are quite good at applying past knowledge to circumstances which are new to us. And we also have access to information, which a computer may not have. But software in their current state? Is it possible to provide enough logic to handle all cases like humans? Can you do that with pattern matching, feature detection, decision trees, bayesian models and anomaly detection alone? And if we do, didn't we just replicate a human driver with all his flaws?
  - Piskvorrr 7y ago
    Aha, that's an interesting point: is the current SDV paradigm "build a faster horse", to quote Henry Ford?
chadmeister 7y ago
What the hell have we been working on if not that? What good is an autonomous driving system that is not able to consider what happens next?!?!?!?!????
- asdfasgasdgasdg 7y ago
  Presumably that? I think the author is just saying that it's going to get better. He is not privy to what his competitors have been working on, in any case, except whatever has been announced publicly. (Unless he is doing industrial espionage.) Maybe it's just that Voyage's cars weren't predicting up until now?
  - loopz 7y ago
    I would be careful in assuming competence in such a new dev area. Think: rules vs predicting vs rise in false positives Heard of control systems waiting for seconds before acting on new information? So don't assume flawless, perfect technology.
    - asdfasgasdgasdg 7y ago
      I don't think there's any implication that the technology is perfect in my comment. But the idea that e.g. Waymo's software is not making some attempt at predicting future states, even implicitly, is absurd. This video[1] was released two years ago and references prediction at the provided timestamp. [1]: https://youtu.be/B8R148hFxPw?t=77 https://youtu.be/B8R148hFxPw?t=77
  - olivercameron 7y ago
    We've been working on the prediction problem for some time. We should have a blog post coming out about our approach (it's pretty neat and in-line with this post) soon.
    - asdfasgasdgasdg 7y ago
      That's great to hear! I think upon fully reading the blog post it's clear that you meant the next leap forward it solving prediction, to the degree that perception has supposedly been solved. It will be interesting to see how y'all do. I for one want my self driving car yesterday.
deleted 7y ago
[deleted]
tlofreso 7y ago
I've thought in the past, once the compute is available in a low-enough-power form, you can continually increase the resolution (frame rate) that autonomous vehicles view the world. Essentially giving them 'The Flash' like vision. You could then have all kinds of predictive models based on various patterns.
alexcnwy 7y ago
I actually think humans playing a game on their phones is the solution to self-driving. Check out www.sebenz.ai
snowwrestler 7y ago
I would argue that driving in traffic is essentially a physically social activity, like a pack of wolves or a flock of birds or a school of fish moving together. There is a tremendous amount of communication happening between the individuals, but not overt language. It's physical communication which requires empathy (i.e. seeing things from someone else's perspective) to accurately interpret and make predictions from. A simple example is to be in a lane next to a lane that ends. Everyone sees the same signs that indicate that lane is ending; everyone knows the cars in that lane are going to get over. But there are a variety of strategies for doing so: get over early; wait until the last minute; accelerate to merge; slow down to merge; use the opportunity to pass; etc. Some drivers are nervous and cautious; some are aggressive. The way the cars move on the road, the signals they provide like brake lights and turn signals, are a form of communication that you, the potentially yielding driver, can interpret and make decisions about how to deal with the merge. Or prevent it! If that is your inclination. This seems like a hard problem IMO. Perceiving objects is essentially about physics--detecting and recognizing. Next step harder, making physical predictions e.g. how far a moving object will travel in 200 milliseconds. Maybe that's enough to keep a self-driving car safe, but it won't make it very efficient in heavy traffic (which is the norm in urban areas). To interpret intent as part of a prediction seems way harder than even that. Reminder that when humans start driving they typically have 16-18 years of continuous social learning... and even then it usually takes years to become a comfortably safe driver in traffic. Of course this would be way easier if all the cars were self-driving, all at once, but that seems like a really hard way to implement this technology. I think you'd have to have a city with very strong leadership that defines a small area that is self-driving only (working with one or more private companies to provide the vehicles), and then slowly grow the boundaries of that zone.
- DennisP 7y ago
  You may be right, but one of your analogies undermines your point. Realistic bird flocking has been famously replicated in simulations by having birds follow three simple rules, none of them requiring empathy. https://en.wikipedia.org/wiki/Flocking_(behavior)#Flocking_rules https://en.wikipedia.org/wiki/Flocking_(behavior)#Flocking_r... So it might be a relatively easy problem for cars, if all the cars could be made to follow a common set of rules. It gets more difficult with humans who may follow different rules, or even actively exploit the rules of the automated vehicles.
sammyo 7y ago
Looking forward prediction should be superseded by announcement. Each vehicle should be broadcasting it's travel vector and next plan to alter that vector. At some point the SDC traffic will look like a high speed motorcycle team weaving between each other, a pedestrian wanders into the middle is just avoided. High efficiency, high safety.
- Piskvorrr 7y ago
  ... assuming everybody plays nice for the common good, ignoring optimizations with externalities.
Piskvorrr 7y ago
"Now that we can safely detect the critical objects around us"...except if they're a pedestrian with a bike. Or something equally unlikely; meh, just run it over, probably uninteresting anyway. /s That's some massive handwaving.
- xiphias2 7y ago
  I agree with you. ImageNet is so far from really understanding the world, that based on this article I wouldn't put any money in this company. Tesla had a lot of presentations about the complexities of detecting the right light to use in a crossing with about 20-30 traffic lights for example. Depth detection is also crucial, and as you wrote there are lots of strange objects/animals/people to handle as edge cases.
bobosha 7y ago
I would modify the article to say "the next leap in AI is prediction". There is a lot of activity in this space called Predictive Coding. Look up the work of David Cox[1], Karl Friston et al. A fascinating variant is Contrastive Predictive Coding[2]. [1] https://www.youtube.com/watch?v=P0yVuoATjzs https://www.youtube.com/watch?v=P0yVuoATjzs [2] https://ankeshanand.com/blog/2020/01/26/contrative-self-supervised-learning.html https://ankeshanand.com/blog/2020/01/26/contrative-self-supe...
ChrisClark 7y ago
Recently I saw a good video showing Tesla's pedestrian prediction. You can see it on reddit here: https://old.reddit.com/r/teslamotors/comments/f4fbfu/nav_on_autopilot_construction_worker_and/ https://old.reddit.com/r/teslamotors/comments/f4fbfu/nav_on_... In the video (https://reddit.com/link/f4fbfu/video/5yg4oclye5h41/player https://reddit.com/link/f4fbfu/video/5yg4oclye5h41/player), skip to 0:38. There is a construction worker walking at a constant speed towards the car's lane. The car could only assume he would keep walking and swerved and stopped. For a human, we'd probably assume the worker is paying attention and will not walk in front of us. Though at times that also ends up being wrong.
- globular-toast 7y ago
  I have no interest in owning a self-driving car myself because I enjoy driving and would never put myself in a situation where I rely on a car, but I'll be very glad if it takes drivers like the one in front off the road. Drivers who can't control their speed without using brakes infuriate me. Not just because it spoils my driving experience but also because I don't like seeing inefficient use of machines. Back on topic, one thing that is invaluable for human drivers is eye contact. I assume that should be taken into account here, ie. if the pedestrian makes eye contact with the vehicle we can assume he has seen it. Otherwise it will result in a lot of unnecessary swerves and stops especially in towns where pedestrians walk right up the side of the carriageway.
  - ChrisClark 7y ago
    That's true. I didn't think of eye contact at all. That really is a big part of the interaction between driver and another driver or pedestrian.
    - tigen 7y ago
      Eye contact isn't reliable. I suspect it can't really be something software can safely use as input, even if it wasn't a vision issue. It's pretty common for motorcyclists or bicyclists to make eye contact with a car driver who proceeds to pull out in front of them anyway. This is probably in part from car driver brains not interpreting bikes as physical threats. It doesn't have to be deliberate. It can also be the driver scanning for "car" and not seeing "car". A self driving car is on the other side of that size equation, but eye contact might also be assumed by a pedestrian to mean that you will stop. Especially if a "driver" makes eye contact who isn't driving. In the end it's best to slow down and not assume. Humans often make dangerous assumptions, for example going around blind turns or over crests too fast to avoid something that might be there. Most of the time it's ok but eventually somebody gets killed. The rational thing to do is make the software safer than humans, not emulate them.
TaylorAlexander 7y ago
It’s funny, I see people online in random comments saying “AI” hasn’t materialized in to anything useful. But if you realize that most talk of “AI” is talk of ML, and if you pay attention to robotics, you’ll find it’s making a huge difference. The fact that anyone can credibly say perception is not the major hurdle anymore is a testament to what the current generation of machine learning algorithms have done to advance the state of the art. I’m excited for the time when even prediction feels like a solved problem. I would think that perception in poor weather remains a real challenge, but certainly we’re leaps and bounds ahead of where we were even 5 years ago.
- ctdonath 7y ago
  We have cars that can drive themselves at speed under normal conditions. That's pretty huge re: "useful". Extending satisfactory performance under fringe conditions is just a matter of improvement.
  - raugustinus 7y ago
    While yes, they also completely lose track if the white lines on the road aren't available.
- KKKKkkkk1 7y ago
  It's categorically untrue that perception is solved, given that things like this happen: https://twitter.com/greentheonly/status/1228067903666348045 https://twitter.com/greentheonly/status/1228067903666348045. And that's considering that Tesla's perception team is run by one of the world's most respected experts in the field, and that their access to data is essentially unlimited. Now what bothers me is that leaders in the industry say without batting an eyelid that perception is solved, given that they witness this type of events on a regular basis.
  - anonymousCar 7y ago
    There's a lot of assumptions in OPs post, but maybe we can assume they're talking in context of their sensor suite. Tesla has made a name for over promising, under delivering on fsd, and refuses to consider lidar as a member of their sensor suite. I don't think you can adequately compare two companies with vastly different approaches.
vladislav 7y ago
"the state-of-the-art in computer vision has moved so significantly that it’s arguably now not the primary blocker to commercial deployment of self-driving cars." The state of the art in machine perception is still far from human level perception, regardless of how quickly progress has been made. Prediction is a function of perception, so it's also bottlenecked on perception getting better. Let's not confuse recruitment-oriented blogposts with credible scientific writing.
nmaley 7y ago
Has anyone observed that the problem of object classification is really just a component of the prediction problem? That the old lady doing donuts in the wheelchair is identifiable as such because her appearance changes in certain (predictable) ways from moment to moment? In other words prediction error minimisation is the basis of the classification, not something made possible by object classification.
- highfrequency 7y ago
  No, prediction is not the basis of classification. You can build a classifier that looks at a single out of context frame and learns to label the objects in it. It won't be able to predict where those objects move in the next frame--it was never even trained using subsequent frames.
Blake_Emigro 7y ago
I think another leap can be achieved with communication between vehicles. Once there are many more self-driving cars on the road, they can feed info to each other as needed. A car experiencing black ice can pass that info to the cars around it. Cars approaching an intersection can adjust their speed so that they all pass through without stopping, etc. But, because there are several manufacturers, this will be hampered without an open standard, and it doesn't solve the randomness of the world that will still be experienced.

107 points by edward 7y ago

I completely agree. If prediction algorithms properly implemented, the Uber crash could be avoided. I think the next step after proper visual prediction is interpreting the noise around. For example, a human can hear the ambulance siren approaching and slow down in a junction without seeing the vehicle. I think sound interpretation is very important especially in city driving.

Retric 7y ago
I don’t think accurate prediction is the right long term strategy. It’s edge cases that cause accidents, so a more adversarial approach is safer. Taken to the extreme this causes problems, but gently slowing down in the case of extreme tailgating is a response to slow reaction times not direct path prediction.
- mellosouls 7y ago
  It's edge cases that will leave 100% self-driving unlikely in the near future.
- AstralStorm 7y ago
  Adversarial approach to driving results in staying at home, because you're likely to be rammed by a bad driver from behind, and you should almost never brake and drive always as fast as possible. Alternatively always stop when you see any pedestrian on the walkway, because they will drive in front of your car. Driving is cooperative not adversarial, even certain classes of illegal behavior are predictable.
lm28469 7y ago
> If prediction algorithms properly implemented, the Uber crash could be avoided. If they didn't willingly disable safety features the crash wouldn't have happened. It wouldn't have happened with the shittiest auto emergency braking system that relies on 0 Ai and 0 predictions.
sonthonax 7y ago
If Uber had bothered to follow a basic safety protocol that person would still be alive today. Staying alert on watch is something that nearly every solider, security guard, sailor, will have to do. Where you basically stare into the dark in case someone comes. There are a number of ways to mitigate the inevitable failure of attention: * Having a parter; * Doing short shifts (change with partner); * Very simple non-distracting tasks - like reading out to your parter a figure at a glance. You certainly should not: * Be on your own; * Have a mobile phone on you (all screens should be locked away while moving). It's so fucking arrogant that Uber didn't care to implement the most obvious risk mitigation. The cost of an additional person on watch is negligible compared to the cost of their self driving care program. My gut feeling is Uber simply misled regulators with how developed their self driving car program is, so it could continue to mislead investors.

zie1ony 7y ago

"It's hard to make predictions - especially about the future." — Robert Storm Petersen

irjustin 7y ago

Predicting is absolutely key. If you've been around the Waymo cars they horribly manage simple situations. They're VERY safe because they take no risk, but that won't work at scale. "What's that? Stop wait..., what's that? stop... wait wait wait" They're very jerky at intersections where there's even a mild amount of uncertainty and really only go when it's very clear. It reminds me of lots of first time drivers reacting to new situations where safe is the best way forward. But if we had pure first time drivers, we'd be a gridlock at even a mild amount of traffic that enters and leaves the roadway. Looking forward to this next set of 5 years. Hopefully us drivers will get replaced in 10? maybe?

kemotep 7y ago
I truly believe my children (not born yet) are going to buy their first car and it will be fully autonomous. Too many people my age and older will be apprehensive about giving up control, despite it likely being safer. So I think it is fair to say that in 20 years there will be many people who learned to "drive" with a self driving car and their parents and grandparents will refuse to give up driving themselves.
- smileysteve 7y ago
  Insurance and police enforcement are two reasons why the decision will be removed from law abiding citizens and won't happen linearly. Once proven safe (or if a manufacturer directly insures the product) the cost to adequately insure a vehicle (as prices of what you hit continue to increase), especially an older vehicle, will increase. With income inequality where it is (and state minimums lagging) it has already become common to have underinsured drivers (and the upper middle class are already paying more through uninsured motorist protection). When it comes to police enforcement, but especially jurisdictions that use traffic stops to find or create crime to fundraise; the expense of having traffic police in a world where > 50% of cars can do no wrong flips the model. Finally, there is a large group of people who will not drive a car that's 7 years old or has over 100k miles; combining with less trained ability to do maintenance on more computerized cars. These people claim to be in control, but at one check engine light away from doing whatever a dealer tells them.

mightybyte 7y ago

I could be missing something, but this sounds like a horrible idea. I've had two accidents in my life. One of them was caused by inattention--hooray that self driving cars have the potential to wipe this cause out. The other was caused because I was a new driver and I predicted what the car in front of me would do and it didn't do it. It was in my first year of driving and fortunately it ended up being a very minor collision, but I distinctly remember that my conscious takeaway was that I shouldn't try to predict what other cars will do. If my car is driving, instead of trying to predict the behavior of other objects I want it to do something like a minimax search and do something that will be safe in the presence of the unexpected, not the expected. Prediction sounds like the complete opposite of good driving practices like the rule of following at least 2 seconds behind the car in front of you, which is specifically there to prevent accidents in the rare case where the car in front of you on the freeway does NOT behave in the predicted way of continuing to go 70 mph in the middle of the lane.

zelos 7y ago
I think it's another stream of data feeding the model. For example: you see a child walking along close to the edge of the road, not looking in your direction. You don't slam your brakes on, but as a driver your prediction is that he may step in front of you, so you ease back on the accelerator, move out further into the middle of the road, cover the brake.
kdmccormick 7y ago
I think "prediction" in this context means something different. It's not specifically that self-driving cars will optimistically assume they know what other drivers are doing, like a naive driver might (although that is a form of prediction). It's more generally that in order to make any smart decisions at all, the AI has to model the future. For example, if you are driving behind a car on the highway, and the car _in front_ of that car puts their brake lights on, you may want to pre-emptively start braking now now. That's a type of pro-safety prediction made based on your understanding of the likely future that driving AIs should also have.
- quietbritishjim 7y ago

prattatx 7y ago

Context matters. For example, at an intersection, both the context of what is happening with others at that intersection as well as what has happened to the vehicle coming up to that intersection matter. An intersection near a soccer field at 4PM has different dynamics than at 4AM. An intersection following a highway exit has different dynamics than an intersection in a neighborhood.

zubspace 7y ago
You're right. And that's why I can't imagine a solution without AGI. There are simply too many context-sensitive parameters and we humans are quite good at applying past knowledge to circumstances which are new to us. And we also have access to information, which a computer may not have. But software in their current state? Is it possible to provide enough logic to handle all cases like humans? Can you do that with pattern matching, feature detection, decision trees, bayesian models and anomaly detection alone? And if we do, didn't we just replicate a human driver with all his flaws?
- Piskvorrr 7y ago
  Aha, that's an interesting point: is the current SDV paradigm "build a faster horse", to quote Henry Ford?

chadmeister 7y ago

What the hell have we been working on if not that? What good is an autonomous driving system that is not able to consider what happens next?!?!?!?!????

asdfasgasdgasdg 7y ago
Presumably that? I think the author is just saying that it's going to get better. He is not privy to what his competitors have been working on, in any case, except whatever has been announced publicly. (Unless he is doing industrial espionage.) Maybe it's just that Voyage's cars weren't predicting up until now?
- loopz 7y ago
  I would be careful in assuming competence in such a new dev area. Think: rules vs predicting vs rise in false positives Heard of control systems waiting for seconds before acting on new information? So don't assume flawless, perfect technology.
  - asdfasgasdgasdg 7y ago
    I don't think there's any implication that the technology is perfect in my comment. But the idea that e.g. Waymo's software is not making some attempt at predicting future states, even implicitly, is absurd. This video[1] was released two years ago and references prediction at the provided timestamp. [1]: https://youtu.be/B8R148hFxPw?t=77 https://youtu.be/B8R148hFxPw?t=77
- olivercameron 7y ago
  We've been working on the prediction problem for some time. We should have a blog post coming out about our approach (it's pretty neat and in-line with this post) soon.
  - asdfasgasdgasdg 7y ago
    That's great to hear! I think upon fully reading the blog post it's clear that you meant the next leap forward it solving prediction, to the degree that perception has supposedly been solved. It will be interesting to see how y'all do. I for one want my self driving car yesterday.

deleted 7y ago

[deleted]

tlofreso 7y ago

I've thought in the past, once the compute is available in a low-enough-power form, you can continually increase the resolution (frame rate) that autonomous vehicles view the world. Essentially giving them 'The Flash' like vision. You could then have all kinds of predictive models based on various patterns.

alexcnwy 7y ago

I actually think humans playing a game on their phones is the solution to self-driving. Check out www.sebenz.ai

snowwrestler 7y ago

I would argue that driving in traffic is essentially a physically social activity, like a pack of wolves or a flock of birds or a school of fish moving together. There is a tremendous amount of communication happening between the individuals, but not overt language. It's physical communication which requires empathy (i.e. seeing things from someone else's perspective) to accurately interpret and make predictions from. A simple example is to be in a lane next to a lane that ends. Everyone sees the same signs that indicate that lane is ending; everyone knows the cars in that lane are going to get over. But there are a variety of strategies for doing so: get over early; wait until the last minute; accelerate to merge; slow down to merge; use the opportunity to pass; etc. Some drivers are nervous and cautious; some are aggressive. The way the cars move on the road, the signals they provide like brake lights and turn signals, are a form of communication that you, the potentially yielding driver, can interpret and make decisions about how to deal with the merge. Or prevent it! If that is your inclination. This seems like a hard problem IMO. Perceiving objects is essentially about physics--detecting and recognizing. Next step harder, making physical predictions e.g. how far a moving object will travel in 200 milliseconds. Maybe that's enough to keep a self-driving car safe, but it won't make it very efficient in heavy traffic (which is the norm in urban areas). To interpret intent as part of a prediction seems way harder than even that. Reminder that when humans start driving they typically have 16-18 years of continuous social learning... and even then it usually takes years to become a comfortably safe driver in traffic. Of course this would be way easier if all the cars were self-driving, all at once, but that seems like a really hard way to implement this technology. I think you'd have to have a city with very strong leadership that defines a small area that is self-driving only (working with one or more private companies to provide the vehicles), and then slowly grow the boundaries of that zone.

DennisP 7y ago
You may be right, but one of your analogies undermines your point. Realistic bird flocking has been famously replicated in simulations by having birds follow three simple rules, none of them requiring empathy. https://en.wikipedia.org/wiki/Flocking_(behavior)#Flocking_rules https://en.wikipedia.org/wiki/Flocking_(behavior)#Flocking_r... So it might be a relatively easy problem for cars, if all the cars could be made to follow a common set of rules. It gets more difficult with humans who may follow different rules, or even actively exploit the rules of the automated vehicles.

sammyo 7y ago

Looking forward prediction should be superseded by announcement. Each vehicle should be broadcasting it's travel vector and next plan to alter that vector. At some point the SDC traffic will look like a high speed motorcycle team weaving between each other, a pedestrian wanders into the middle is just avoided. High efficiency, high safety.

Piskvorrr 7y ago
... assuming everybody plays nice for the common good, ignoring optimizations with externalities.

Piskvorrr 7y ago

"Now that we can safely detect the critical objects around us"...except if they're a pedestrian with a bike. Or something equally unlikely; meh, just run it over, probably uninteresting anyway. /s That's some massive handwaving.

xiphias2 7y ago
I agree with you. ImageNet is so far from really understanding the world, that based on this article I wouldn't put any money in this company. Tesla had a lot of presentations about the complexities of detecting the right light to use in a crossing with about 20-30 traffic lights for example. Depth detection is also crucial, and as you wrote there are lots of strange objects/animals/people to handle as edge cases.

bobosha 7y ago

I would modify the article to say "the next leap in AI is prediction". There is a lot of activity in this space called Predictive Coding. Look up the work of David Cox[1], Karl Friston et al. A fascinating variant is Contrastive Predictive Coding[2]. [1] https://www.youtube.com/watch?v=P0yVuoATjzs https://www.youtube.com/watch?v=P0yVuoATjzs [2] https://ankeshanand.com/blog/2020/01/26/contrative-self-supervised-learning.html https://ankeshanand.com/blog/2020/01/26/contrative-self-supe...

ChrisClark 7y ago

Recently I saw a good video showing Tesla's pedestrian prediction. You can see it on reddit here: https://old.reddit.com/r/teslamotors/comments/f4fbfu/nav_on_autopilot_construction_worker_and/ https://old.reddit.com/r/teslamotors/comments/f4fbfu/nav_on_... In the video (https://reddit.com/link/f4fbfu/video/5yg4oclye5h41/player https://reddit.com/link/f4fbfu/video/5yg4oclye5h41/player), skip to 0:38. There is a construction worker walking at a constant speed towards the car's lane. The car could only assume he would keep walking and swerved and stopped. For a human, we'd probably assume the worker is paying attention and will not walk in front of us. Though at times that also ends up being wrong.

globular-toast 7y ago
I have no interest in owning a self-driving car myself because I enjoy driving and would never put myself in a situation where I rely on a car, but I'll be very glad if it takes drivers like the one in front off the road. Drivers who can't control their speed without using brakes infuriate me. Not just because it spoils my driving experience but also because I don't like seeing inefficient use of machines. Back on topic, one thing that is invaluable for human drivers is eye contact. I assume that should be taken into account here, ie. if the pedestrian makes eye contact with the vehicle we can assume he has seen it. Otherwise it will result in a lot of unnecessary swerves and stops especially in towns where pedestrians walk right up the side of the carriageway.
- ChrisClark 7y ago
  That's true. I didn't think of eye contact at all. That really is a big part of the interaction between driver and another driver or pedestrian.
  - tigen 7y ago
    Eye contact isn't reliable. I suspect it can't really be something software can safely use as input, even if it wasn't a vision issue. It's pretty common for motorcyclists or bicyclists to make eye contact with a car driver who proceeds to pull out in front of them anyway. This is probably in part from car driver brains not interpreting bikes as physical threats. It doesn't have to be deliberate. It can also be the driver scanning for "car" and not seeing "car". A self driving car is on the other side of that size equation, but eye contact might also be assumed by a pedestrian to mean that you will stop. Especially if a "driver" makes eye contact who isn't driving. In the end it's best to slow down and not assume. Humans often make dangerous assumptions, for example going around blind turns or over crests too fast to avoid something that might be there. Most of the time it's ok but eventually somebody gets killed. The rational thing to do is make the software safer than humans, not emulate them.

TaylorAlexander 7y ago

It’s funny, I see people online in random comments saying “AI” hasn’t materialized in to anything useful. But if you realize that most talk of “AI” is talk of ML, and if you pay attention to robotics, you’ll find it’s making a huge difference. The fact that anyone can credibly say perception is not the major hurdle anymore is a testament to what the current generation of machine learning algorithms have done to advance the state of the art. I’m excited for the time when even prediction feels like a solved problem. I would think that perception in poor weather remains a real challenge, but certainly we’re leaps and bounds ahead of where we were even 5 years ago.

ctdonath 7y ago
We have cars that can drive themselves at speed under normal conditions. That's pretty huge re: "useful". Extending satisfactory performance under fringe conditions is just a matter of improvement.
- raugustinus 7y ago
  While yes, they also completely lose track if the white lines on the road aren't available.
KKKKkkkk1 7y ago
It's categorically untrue that perception is solved, given that things like this happen: https://twitter.com/greentheonly/status/1228067903666348045 https://twitter.com/greentheonly/status/1228067903666348045. And that's considering that Tesla's perception team is run by one of the world's most respected experts in the field, and that their access to data is essentially unlimited. Now what bothers me is that leaders in the industry say without batting an eyelid that perception is solved, given that they witness this type of events on a regular basis.
- anonymousCar 7y ago
  There's a lot of assumptions in OPs post, but maybe we can assume they're talking in context of their sensor suite. Tesla has made a name for over promising, under delivering on fsd, and refuses to consider lidar as a member of their sensor suite. I don't think you can adequately compare two companies with vastly different approaches.

vladislav 7y ago

"the state-of-the-art in computer vision has moved so significantly that it’s arguably now not the primary blocker to commercial deployment of self-driving cars." The state of the art in machine perception is still far from human level perception, regardless of how quickly progress has been made. Prediction is a function of perception, so it's also bottlenecked on perception getting better. Let's not confuse recruitment-oriented blogposts with credible scientific writing.

nmaley 7y ago

Has anyone observed that the problem of object classification is really just a component of the prediction problem? That the old lady doing donuts in the wheelchair is identifiable as such because her appearance changes in certain (predictable) ways from moment to moment? In other words prediction error minimisation is the basis of the classification, not something made possible by object classification.

highfrequency 7y ago
No, prediction is not the basis of classification. You can build a classifier that looks at a single out of context frame and learns to label the objects in it. It won't be able to predict where those objects move in the next frame--it was never even trained using subsequent frames.

Blake_Emigro 7y ago

I think another leap can be achieved with communication between vehicles. Once there are many more self-driving cars on the road, they can feed info to each other as needed. A car experiencing black ice can pass that info to the cars around it. Cars approaching an intersection can adjust their speed so that they all pass through without stopping, etc. But, because there are several manufacturers, this will be hampered without an open standard, and it doesn't solve the randomness of the world that will still be experienced.