4 ms·

I think vision-only approaches can work, but our eyes and brain are amazing and it would take some serious hardware. Our eyes have a 200-degree FOV, providing a

I think vision-only approaches can work, but our eyes and brain are amazing and it would take some serious hardware. Our eyes have a 200-degree FOV, providing a 576 megapixel landscape, with 13ms of latency. Plus there are 6 billion neurons in the visual cortex alone to process the images, which are then fed to another 80 billion neurons that can interpret and react to the data. Peppering a few webcam-quality cameras around a car and plugging it into an Intel Atom processor probably won't be better than our eyes and brain, even if the cameras don't blink or get tired. It's only going to get better though.

gruez 2y ago
>Our eyes have a 200-degree FOV, providing a 576 megapixel landscape, with 13ms of latency. ...only if you count the field of view you get from moving your eyeballs. You wouldn't say a PTZ camera has "360 FOV" just because it can rotate around. The "576 megapixel" figure is also questionable. Peak resolution only exists in the fovea. Everywhere else is blurry and much lower resolution. You don't notice this because your eyes does it automatically, but the actual information you can receive at any given time is far less than "576 megapixel".
- standeven 2y ago
  The quoted latency and neuron counts can also be questioned, but my point stands: it's hard to compete with the human eye and brain with current (affordable) camera and processing hardware.
nomel 2y ago
There are corner cases of vision that cannot work because it's not mathematically possible, like a featureless wall (or ground, in case of recent mars crash). And, vision can't work in/penetrate heavy snow or fog, which is transparent to radar. Vision is an indirect measurement. Lidar/radar is a direct measurement. I'm curious if there are any other safety critical systems that uses such massively indirect measurements?
- standeven 2y ago
  Good point regarding snow and fog, but I’m assuming operation would slow or stop in those conditions and that would be acceptable. Does a truly featureless wall/road with no visible edges actually exist in the wild? I’d expect cameras with high enough resolution, spacing, and FOV would handle any real world examples but maybe I’m wrong.
  - bcrl 2y ago
    For those of us who live in areas with lots of snowy winter conditions, you can't just hand wave that requirement away (as we in industry do so often while developing software)!!!! Winter driving conditions are some of the most important times for driver assistance systems to help drivers out, especially young drivers. Tunnel vision while driving in snow is a very real thing that drivers encounter often enough where I live, and there have been plenty of times when I had to drive home on roads with barely visible boundaries; you figure out where the road is based on the ditch and the sound your tires make when they hit the edge of the road combined with a copious reduction in speed to give you time to recover. At this point it does not feel safe to trust self driving cars or assistive systems designed, built and tested primarily in California or the southern US for one simple reason: they do not get the range of adverse weather conditions that drivers in the rest of the world have to deal with and adapt safely to on a regular basis. It's easy to make a self driving car that "works" on California style freeways which are almost never under construction because they don't wear out as fast. In other places like eastern Ontario we sometimes have to deal with temperature shifts from -30C to +10C in 24 hours, salt our roads like crazy in the winter, and have a much wider range of typical weather conditions. These all take a significant toll on road infrastructure, and mean that what are rare corner cases in California become regular events elsewhere. We have 2 seasons where I live: construction season and winter. Based on several published reports of self driving cars hitting parked emergency vehicles or lane confusion in construction zones, I simply do not trust that the current widely available "self driving" vehicles are provably safe outside of the near ideal conditions present in California. At least Waymo seems to be quite hesitant about rolling out to cities that have less favourable weather. What I would like to see is for regulatory bodies with a safety first approach to accidents (similar to how the FAA investigates and regulates commercial aircraft) be involved in setting the criteria for the design, testing and regulation of self driving cars and driver assistance systems. Reading reports and watching shows about the root cause analysis of airplane crashes is fascinating, and it shows just how hard it is to learn how to make large and complex real world systems safe. It has taken plenty of deaths to get us to the point where commercial flights are safer than the trip to the airport, and it will take many more deaths before self driving cars are appreciably better than humans. Some of the most important lessons from aviation are about the interaction between pilot(s), crew and automation, and how those systems fail. Test cases / data for self driving cars should be shared and made public. If we're trusting our lives to a piece of software, we should be able to see how well it does across standard test cases that the industry has encountered and developed, and be able to help add more. Capitalism does many things well, but making things safe for humans is not one of them.

gruez 2y ago

>Our eyes have a 200-degree FOV, providing a 576 megapixel landscape, with 13ms of latency. ...only if you count the field of view you get from moving your eyeballs. You wouldn't say a PTZ camera has "360 FOV" just because it can rotate around. The "576 megapixel" figure is also questionable. Peak resolution only exists in the fovea. Everywhere else is blurry and much lower resolution. You don't notice this because your eyes does it automatically, but the actual information you can receive at any given time is far less than "576 megapixel".

standeven 2y ago
The quoted latency and neuron counts can also be questioned, but my point stands: it's hard to compete with the human eye and brain with current (affordable) camera and processing hardware.

nomel 2y ago

There are corner cases of vision that cannot work because it's not mathematically possible, like a featureless wall (or ground, in case of recent mars crash). And, vision can't work in/penetrate heavy snow or fog, which is transparent to radar. Vision is an indirect measurement. Lidar/radar is a direct measurement. I'm curious if there are any other safety critical systems that uses such massively indirect measurements?

standeven 2y ago
Good point regarding snow and fog, but I’m assuming operation would slow or stop in those conditions and that would be acceptable. Does a truly featureless wall/road with no visible edges actually exist in the wild? I’d expect cameras with high enough resolution, spacing, and FOV would handle any real world examples but maybe I’m wrong.
- bcrl 2y ago
  For those of us who live in areas with lots of snowy winter conditions, you can't just hand wave that requirement away (as we in industry do so often while developing software)!!!! Winter driving conditions are some of the most important times for driver assistance systems to help drivers out, especially young drivers. Tunnel vision while driving in snow is a very real thing that drivers encounter often enough where I live, and there have been plenty of times when I had to drive home on roads with barely visible boundaries; you figure out where the road is based on the ditch and the sound your tires make when they hit the edge of the road combined with a copious reduction in speed to give you time to recover. At this point it does not feel safe to trust self driving cars or assistive systems designed, built and tested primarily in California or the southern US for one simple reason: they do not get the range of adverse weather conditions that drivers in the rest of the world have to deal with and adapt safely to on a regular basis. It's easy to make a self driving car that "works" on California style freeways which are almost never under construction because they don't wear out as fast. In other places like eastern Ontario we sometimes have to deal with temperature shifts from -30C to +10C in 24 hours, salt our roads like crazy in the winter, and have a much wider range of typical weather conditions. These all take a significant toll on road infrastructure, and mean that what are rare corner cases in California become regular events elsewhere. We have 2 seasons where I live: construction season and winter. Based on several published reports of self driving cars hitting parked emergency vehicles or lane confusion in construction zones, I simply do not trust that the current widely available "self driving" vehicles are provably safe outside of the near ideal conditions present in California. At least Waymo seems to be quite hesitant about rolling out to cities that have less favourable weather. What I would like to see is for regulatory bodies with a safety first approach to accidents (similar to how the FAA investigates and regulates commercial aircraft) be involved in setting the criteria for the design, testing and regulation of self driving cars and driver assistance systems. Reading reports and watching shows about the root cause analysis of airplane crashes is fascinating, and it shows just how hard it is to learn how to make large and complex real world systems safe. It has taken plenty of deaths to get us to the point where commercial flights are safer than the trip to the airport, and it will take many more deaths before self driving cars are appreciably better than humans. Some of the most important lessons from aviation are about the interaction between pilot(s), crew and automation, and how those systems fail. Test cases / data for self driving cars should be shared and made public. If we're trusting our lives to a piece of software, we should be able to see how well it does across standard test cases that the industry has encountered and developed, and be able to help add more. Capitalism does many things well, but making things safe for humans is not one of them.