5 ms·

> I wouldn't want any system to drive right over a possible collision based on an assumption of its material. Imagine a scenario where it is worse to change la

> I wouldn't want any system to drive right over a possible collision based on an assumption of its material. Imagine a scenario where it is worse to change lanes or to quickly break than to just drive over the box... Like the time I was on I-95 in heavy, fast moving traffic and I saw a box flutter off the back of a truck and I decided to just let it bounce off my windshield. There are endless amounts of these kinds of problems. It would be useless to attempt to enumerate them, either in writing, or in code, which speaks to the power of neural networks!

solardev 3y ago
It's true, but in that case it's more a matter of prior experience (oh, I bet that cardboard box/squirrel/etc. probably won't hurt me) than resolving conflicting sensors. That probably lives a few levels above the raw sensor inputs anyway. It's an argument for more powerful training, not fewer sensors... those aren't mutually exclusive goals. Isolating it to just the sensor layer, though, there are often legitimate times when a visual spectrum camera alone isn't ideal. Like not being able to maintain distance to the truck ahead because the glare of the sun is bouncing off its reflective back label at just the right (or wrong) angle, or an odd long object is sticking out the back of a pickup and the object detection can't figure out what it is (but a lidar or radar could sense the presence of something). Or for nighttime driving when deer glow in the infrared far enough ahead but not so much in visual. Or driving by another car in "dealer camouflage". Having another type of sensor could be life saving. For what it's worth, that's true for non autonomous systems too. The human sensor suite in drivers is already often augmented by additional sensors, like parking radars or infrared HUDs or or at least an additional pair of stereoscopic visual cameras with their own judgment. I think these are fundamentally different parts of the overall system. Sure, you can train algorithms on one input type alone. Or multiple. But unless there's evidence showing that training on multiple inputs produces worse final outcomes than training on just one alone, it seems that having multiple sensors should be a net positive.
- williamcotton 3y ago
  But unless there's evidence showing that training on multiple inputs produces worse final outcomes than training on just one alone, it seems that having multiple sensors should be a net positive. Go train a CNN on 4k images and then go train a CNN on those same images at VGA resolution and you'll see just how problematic a larger set of inputs can be for minimal gains! I have never argued that LIDAR couldn't be used by an already trained autonomous system in the same manner that a human driver is presented with LIDAR sensory inputs! An android seems to be the trajectory we're on if we want a machine to have the same embodied decision making that humans are capable of. Like I said in my original comment, this infinity of embodied experiences seems to be software problem, not a sensor problem.
  - solardev 3y ago
    Hmm, I think we're just agreeing around each other, here...? lol Yes, being able to distinguish a cardboard box from a plywood one is primarily a software and training problem. But the bigger problem above that -- which is what the overall thread is about, if not your specific scenario -- is whether multiple sensors can make overall driving (autonomous or not) safer. > Go train a CNN on 4k images and then go train a CNN on those same images at VGA resolution and you'll see just how problematic a larger set of inputs can be for minimal gains! Surely there's a difference between incremental resolution gains of the same sensor type vs a completely separate channel of data? For example, the camera-based driver assist systems gradually evolved from black-and-white to color inputs. Maybe "more pixels" isn't as valuable as "sensors able to differentiate color" in terms of being able to easily identify traffic lights, road signs, lane markers etc. Even if the training is harder to do, the outcome for the human passengers is usually better. And it might not even make the training harder, if the additional color channels can help disambiguate the black-and-white data. Same with infrared. Maybe each sensor type has an optimal resolution/sampling rate/etc., but that's not the same question as whether mixing altogether different sensor types can help. One is trying to classify 3D objects from 2D photogrammetry, the other from a 3D point cloud. To use another contrived example, let's say there's a black round thing on the ground, a few meters ahead, in damp twilight conditions. A single camera might be able to see that it could be a manhole cover or a pothole. Two cameras might be able to gauge depth too, ascertaining that it's indeed covered (and not just a hole you're going to fall into). But if it's really dark out or wet, or the thing is surrounded by a puddle of water that visual cameras have trouble making out, being able to map its topography with a lidar seems like a nicer way to make sure it's safe to drive over. I think the overall opposition you're seeing here is not really directed at the "cardboard vs plywood" question, but the bleedover from Tesla's decision to remove lidar and radar (for cost reasons) while cynically using safety/software as excuses. People take issue with that because it's a disingenuous argument that's led to preventable fatalities -- and we're hoping other automakers don't make the same mistake. > I have never argued that LIDAR couldn't be used by an already trained autonomous system in the same manner that a human driver is presented with LIDAR sensory inputs! That's not the ONLY way lidar can be helpful. Even in existing driver-assist systems using basic object recognition technology from yesteryear, having different sensor types can be helpful. Lidar & radar don't require sentience to be useful. If having additional sensors makes training ML-based systems harder... okay? Wait a few more years, rather than cutting them out altogether to rush a camera-only system out the door? I used to be really excited about Teslas, back when they actually were trying to do the hard engineering of integrating all these different sensors. When they switched to gaslighting marketing trying to convince the world that cameras were enough (and they weren't)... well, I lost a lot of trust in them -- especially having seen firsthand the success rate of the camera-only system in my own car, and knowing that it typically fails when my own eyes fail (high glare, low light, fog, etc.). But yes, to your point, being able to distinguish plywood from cardboard is not the strength of lidar.

solardev 3y ago

It's true, but in that case it's more a matter of prior experience (oh, I bet that cardboard box/squirrel/etc. probably won't hurt me) than resolving conflicting sensors. That probably lives a few levels above the raw sensor inputs anyway. It's an argument for more powerful training, not fewer sensors... those aren't mutually exclusive goals. Isolating it to just the sensor layer, though, there are often legitimate times when a visual spectrum camera alone isn't ideal. Like not being able to maintain distance to the truck ahead because the glare of the sun is bouncing off its reflective back label at just the right (or wrong) angle, or an odd long object is sticking out the back of a pickup and the object detection can't figure out what it is (but a lidar or radar could sense the presence of something). Or for nighttime driving when deer glow in the infrared far enough ahead but not so much in visual. Or driving by another car in "dealer camouflage". Having another type of sensor could be life saving. For what it's worth, that's true for non autonomous systems too. The human sensor suite in drivers is already often augmented by additional sensors, like parking radars or infrared HUDs or or at least an additional pair of stereoscopic visual cameras with their own judgment. I think these are fundamentally different parts of the overall system. Sure, you can train algorithms on one input type alone. Or multiple. But unless there's evidence showing that training on multiple inputs produces worse final outcomes than training on just one alone, it seems that having multiple sensors should be a net positive.

williamcotton 3y ago
But unless there's evidence showing that training on multiple inputs produces worse final outcomes than training on just one alone, it seems that having multiple sensors should be a net positive. Go train a CNN on 4k images and then go train a CNN on those same images at VGA resolution and you'll see just how problematic a larger set of inputs can be for minimal gains! I have never argued that LIDAR couldn't be used by an already trained autonomous system in the same manner that a human driver is presented with LIDAR sensory inputs! An android seems to be the trajectory we're on if we want a machine to have the same embodied decision making that humans are capable of. Like I said in my original comment, this infinity of embodied experiences seems to be software problem, not a sensor problem.
- solardev 3y ago
  Hmm, I think we're just agreeing around each other, here...? lol Yes, being able to distinguish a cardboard box from a plywood one is primarily a software and training problem. But the bigger problem above that -- which is what the overall thread is about, if not your specific scenario -- is whether multiple sensors can make overall driving (autonomous or not) safer. > Go train a CNN on 4k images and then go train a CNN on those same images at VGA resolution and you'll see just how problematic a larger set of inputs can be for minimal gains! Surely there's a difference between incremental resolution gains of the same sensor type vs a completely separate channel of data? For example, the camera-based driver assist systems gradually evolved from black-and-white to color inputs. Maybe "more pixels" isn't as valuable as "sensors able to differentiate color" in terms of being able to easily identify traffic lights, road signs, lane markers etc. Even if the training is harder to do, the outcome for the human passengers is usually better. And it might not even make the training harder, if the additional color channels can help disambiguate the black-and-white data. Same with infrared. Maybe each sensor type has an optimal resolution/sampling rate/etc., but that's not the same question as whether mixing altogether different sensor types can help. One is trying to classify 3D objects from 2D photogrammetry, the other from a 3D point cloud. To use another contrived example, let's say there's a black round thing on the ground, a few meters ahead, in damp twilight conditions. A single camera might be able to see that it could be a manhole cover or a pothole. Two cameras might be able to gauge depth too, ascertaining that it's indeed covered (and not just a hole you're going to fall into). But if it's really dark out or wet, or the thing is surrounded by a puddle of water that visual cameras have trouble making out, being able to map its topography with a lidar seems like a nicer way to make sure it's safe to drive over. I think the overall opposition you're seeing here is not really directed at the "cardboard vs plywood" question, but the bleedover from Tesla's decision to remove lidar and radar (for cost reasons) while cynically using safety/software as excuses. People take issue with that because it's a disingenuous argument that's led to preventable fatalities -- and we're hoping other automakers don't make the same mistake. > I have never argued that LIDAR couldn't be used by an already trained autonomous system in the same manner that a human driver is presented with LIDAR sensory inputs! That's not the ONLY way lidar can be helpful. Even in existing driver-assist systems using basic object recognition technology from yesteryear, having different sensor types can be helpful. Lidar & radar don't require sentience to be useful. If having additional sensors makes training ML-based systems harder... okay? Wait a few more years, rather than cutting them out altogether to rush a camera-only system out the door? I used to be really excited about Teslas, back when they actually were trying to do the hard engineering of integrating all these different sensors. When they switched to gaslighting marketing trying to convince the world that cameras were enough (and they weren't)... well, I lost a lot of trust in them -- especially having seen firsthand the success rate of the camera-only system in my own car, and knowing that it typically fails when my own eyes fail (high glare, low light, fog, etc.). But yes, to your point, being able to distinguish plywood from cardboard is not the strength of lidar.