6 ms·

If we want to be safer, then we need to blend sensors. We do this as humans. We use our ears for sound and inertial detection. Lidar, Radar, colour and near i

If we want to be safer, then we need to blend sensors. We do this as humans. We use our ears for sound and inertial detection. Lidar, Radar, colour and near infrared optical all have a place in autonomous driving. I dont think we are really anywhere near optical working reliably as a single source. I am immediately suspicious of anyone who suggests otherwise, because they either believe to much in the state of AI, or haven't thought enough about life critical systems to be let near anything autonomous. Worse still, they may be out to make a quick buck. The thing that lidar has is speed and accuracy. 100hz update rates, and 1/100th second latency is not unthinkable. With optical systems you're lucky to get 25hz with sub 100ms latency. Thats just for depth estimation using semantics, you still need to feed it into your driving model. Lidar works way better at night, it works a tonne better with unknown objects. However, like optical its shit in rain/snow. Hence why decent resolution radar will be needed as well.

ganeshkrishnan 7y ago
>If we want to be safer, then we need to blend sensors. isnt that what a kalman filter does? I thought most autonomous cars have sensor fusion to blend all sensors
- CardenB 7y ago
  “Sensor fusion” isn’t just a thing, it’s an entire area of study. Kalman filtering is one technique in the umbrella of sensor fusion. There is a lot of room for improvement here.
- teraflop 7y ago
  A Kalman filter solves the specific problem of estimating a finite-dimensional state vector from uncertain measurements, where the measurement errors and their correlations follow (or can be approximated as) a multivariate Gaussian. That works quite well for, say, combining GPS (which has short-term noise) with inertial/odometry measurements (which suffer from long-term drift) to determine your vehicle's position, orientation, and velocity in 3D space (expressible as a 9-dimensional state vector). But it's not directly applicable to problems like combining map data with LIDAR and vision to generate a representation of your surroundings.
  - ganeshkrishnan 7y ago
    That's informative, thanks. I was under the impression that autonomous cars already combine all their sensor readings
- charmides 7y ago
  "Lichess is for professionals. Plenty of GMs ims and masters play on it. Even Magnus plays the tournaments. This is like saying Linux is not enough for computer professionals." This is a misunderstanding. Lichess is used by chess professionals for playing online chess against each other (which is mostly recreation and not serious chess preparation). Software like ChessBase is used for game preparation.
lmm 7y ago
Humans mostly don't use their ears when driving cars (to the frustration of the rest of us using the road).
- rainyMammoth 7y ago
  you should try using some noise cancelling headphones and listening to music while driving. The result is really surprising and it is way more difficult to drive. Humans are actually using their ears a LOT while driving. You don't realize it because the brain is so powerful at blending everything into a single "feel".
  - lmm 7y ago
    You're listening to your own engine yes, but a car control system presumably has more direct access to that information. You simply can't hear sound from outside - and car makers advertise this as a feature! - to say nothing of the fact that many or most drivers will be playing music by one or another means.
    - marshallant 7y ago
      You can't hear sound from outside? So you can't hear sirens? Or horns? Or cars right next to you? Or safety warnings that warn of cars next to you when turning your blinker on? Or needing to break quickly? Etc etc
    - rainyMammoth 7y ago
      You are also blending sounds from other cars, as well as sounds from your tires on the road which gives you some feedback on the driving conditions. All of those match patterns that you have seen before and give you a better feeling about your car status. The brain is using hundreds of different captors and is order of magnitude more powerful than any computer today. Attempting to drive with a couple cameras as only sensors is a recipe for disaster as has already been proved (see also: Tesla crashes)
    - zaarn 7y ago
      You can hear plenty of sound from the outside, most of it in the lower frequency bands and sometimes only via bone transmission, but it's there. Get a pair of active noise-cancelling headphones and try it for yourself, especially in more complicated driving situations. While the brain can still operate a car with the loss of some input, visual input isn't really enough and is heavily supported by all other senses (acceleration and gravity, sound, touch and vibration, temperature and kinesthetic). Unlike modern computers, the brain is massively parallel and asynchronous, so it can simply use all of the inputs to find the correct course of action, even going as far as considering multiple lines of thoughts (and then your memory is manipulated so you only experience the chosen line of thought, similar to your vision being manipulated when you blink).
lightedman 7y ago
Radar is pretty garbage in rain and snow as well. https://www.weather.gov/media/publications/front/14dec-front.pdf https://www.weather.gov/media/publications/front/14dec-front... My father was a missile radar guidance technician for a living. He says radar on a car makes almost no sense, especially for navigating in inclement weather, due to how radar tends to reflect off of anything to some degree.
- cameldrv 7y ago
  Radar works fine in rain and snow. You can use it to detect rain and snow, but the reflection is very weak compared to the reflection from a solid object, especially at the 0-500m range that these operate at. Big weather radars go out to a couple of hundred miles.
  - lightedman 7y ago
    For a vehicle, it will be far less than ideal. It will have problems distinguishing between heavy rain and a solid object, especially at night, without a lot of other high-precision high-refresh sensor data and a good computer to put it all together in realtime. Every sensor, man-made or natural, sucks at detection when it comes to operating in heavy inclement weather. That's why we need tons of processing power and filtering algorithms. If they were any good, we'd not need those higher-order algos and filters. The sensors could simply generate reliable data.
    - cameldrv 7y ago
      Is this based on data you've seen or worked on? In general radar performs a lot better than lidar because the wavelength of automotive radar is reasonably large relative to the size of a raindrop.
    - isopede 7y ago
      This is not true. Automotive radar are in the ~80ghz range, giving it a wavelength on the order of about 4mm. Car radar can definitely see through rain and snow. All radars receive a range of reflections from many objects, rain included, but the return off of rain is nothing compared to the return off of a solid object. The filtering then becomes choosing the peak amplitudes of the radar return signal. Rain will manifest itself as a smearing of the signal and lowering the peak a little bit, but usually much less than a solid object. Optical and lidar work miserably through rain/fog, with wavelengths in the ~1000nm range, but have much higher resolution. Any real self-driving solution in the near to medium future is going to involve sensor fusion of as many sensors the car manufacturers can afford and cram in. This means cameras, ultrasound, lidar, radar, etc. Anybody telling you that there is only one true sensor to rule them all is either selling you something or ignorant of the state of the industry
sllabres 7y ago
When I'll drive down a motorway I doubt that I use sound or inertial for planing my further track. I presume that one could drive 'remote' with only visual, if it would be the same quality as when one was in the vehicle.

ganeshkrishnan 7y ago

>If we want to be safer, then we need to blend sensors. isnt that what a kalman filter does? I thought most autonomous cars have sensor fusion to blend all sensors

CardenB 7y ago
“Sensor fusion” isn’t just a thing, it’s an entire area of study. Kalman filtering is one technique in the umbrella of sensor fusion. There is a lot of room for improvement here.
teraflop 7y ago
A Kalman filter solves the specific problem of estimating a finite-dimensional state vector from uncertain measurements, where the measurement errors and their correlations follow (or can be approximated as) a multivariate Gaussian. That works quite well for, say, combining GPS (which has short-term noise) with inertial/odometry measurements (which suffer from long-term drift) to determine your vehicle's position, orientation, and velocity in 3D space (expressible as a 9-dimensional state vector). But it's not directly applicable to problems like combining map data with LIDAR and vision to generate a representation of your surroundings.
- ganeshkrishnan 7y ago
  That's informative, thanks. I was under the impression that autonomous cars already combine all their sensor readings
charmides 7y ago
"Lichess is for professionals. Plenty of GMs ims and masters play on it. Even Magnus plays the tournaments. This is like saying Linux is not enough for computer professionals." This is a misunderstanding. Lichess is used by chess professionals for playing online chess against each other (which is mostly recreation and not serious chess preparation). Software like ChessBase is used for game preparation.

lmm 7y ago

Humans mostly don't use their ears when driving cars (to the frustration of the rest of us using the road).

rainyMammoth 7y ago
you should try using some noise cancelling headphones and listening to music while driving. The result is really surprising and it is way more difficult to drive. Humans are actually using their ears a LOT while driving. You don't realize it because the brain is so powerful at blending everything into a single "feel".
- lmm 7y ago
  You're listening to your own engine yes, but a car control system presumably has more direct access to that information. You simply can't hear sound from outside - and car makers advertise this as a feature! - to say nothing of the fact that many or most drivers will be playing music by one or another means.
  - marshallant 7y ago
    You can't hear sound from outside? So you can't hear sirens? Or horns? Or cars right next to you? Or safety warnings that warn of cars next to you when turning your blinker on? Or needing to break quickly? Etc etc
  - rainyMammoth 7y ago
    You are also blending sounds from other cars, as well as sounds from your tires on the road which gives you some feedback on the driving conditions. All of those match patterns that you have seen before and give you a better feeling about your car status. The brain is using hundreds of different captors and is order of magnitude more powerful than any computer today. Attempting to drive with a couple cameras as only sensors is a recipe for disaster as has already been proved (see also: Tesla crashes)
  - zaarn 7y ago
    You can hear plenty of sound from the outside, most of it in the lower frequency bands and sometimes only via bone transmission, but it's there. Get a pair of active noise-cancelling headphones and try it for yourself, especially in more complicated driving situations. While the brain can still operate a car with the loss of some input, visual input isn't really enough and is heavily supported by all other senses (acceleration and gravity, sound, touch and vibration, temperature and kinesthetic). Unlike modern computers, the brain is massively parallel and asynchronous, so it can simply use all of the inputs to find the correct course of action, even going as far as considering multiple lines of thoughts (and then your memory is manipulated so you only experience the chosen line of thought, similar to your vision being manipulated when you blink).

lightedman 7y ago

Radar is pretty garbage in rain and snow as well. https://www.weather.gov/media/publications/front/14dec-front.pdf https://www.weather.gov/media/publications/front/14dec-front... My father was a missile radar guidance technician for a living. He says radar on a car makes almost no sense, especially for navigating in inclement weather, due to how radar tends to reflect off of anything to some degree.

cameldrv 7y ago
Radar works fine in rain and snow. You can use it to detect rain and snow, but the reflection is very weak compared to the reflection from a solid object, especially at the 0-500m range that these operate at. Big weather radars go out to a couple of hundred miles.
- lightedman 7y ago
  For a vehicle, it will be far less than ideal. It will have problems distinguishing between heavy rain and a solid object, especially at night, without a lot of other high-precision high-refresh sensor data and a good computer to put it all together in realtime. Every sensor, man-made or natural, sucks at detection when it comes to operating in heavy inclement weather. That's why we need tons of processing power and filtering algorithms. If they were any good, we'd not need those higher-order algos and filters. The sensors could simply generate reliable data.
  - cameldrv 7y ago
    Is this based on data you've seen or worked on? In general radar performs a lot better than lidar because the wavelength of automotive radar is reasonably large relative to the size of a raindrop.
  - isopede 7y ago
    This is not true. Automotive radar are in the ~80ghz range, giving it a wavelength on the order of about 4mm. Car radar can definitely see through rain and snow. All radars receive a range of reflections from many objects, rain included, but the return off of rain is nothing compared to the return off of a solid object. The filtering then becomes choosing the peak amplitudes of the radar return signal. Rain will manifest itself as a smearing of the signal and lowering the peak a little bit, but usually much less than a solid object. Optical and lidar work miserably through rain/fog, with wavelengths in the ~1000nm range, but have much higher resolution. Any real self-driving solution in the near to medium future is going to involve sensor fusion of as many sensors the car manufacturers can afford and cram in. This means cameras, ultrasound, lidar, radar, etc. Anybody telling you that there is only one true sensor to rule them all is either selling you something or ignorant of the state of the industry

sllabres 7y ago

When I'll drive down a motorway I doubt that I use sound or inertial for planing my further track. I presume that one could drive 'remote' with only visual, if it would be the same quality as when one was in the vehicle.