6 ms·

> ”There is no one sensor that will do it. Full stop.” Yet somehow, humans can drive cars with just a pair of optical sensors (mounted on a swivelling gimbal,

> ”There is no one sensor that will do it. Full stop.” Yet somehow, humans can drive cars with just a pair of optical sensors (mounted on a swivelling gimbal, of sorts). In theory, a sufficiently capable AI should be able to drive a car at least as well as a human can using the same input: vision.

fennecfoxy 4y ago
I believe in the sentiment, but it's true that we humans also crash our cars a LOT. From minor bumps and scrapes to multiple fatalities.
CyanBird 4y ago
> sufficiently capable AI And Tesla lacks that, so therefore they ought not simply rely on cameras and ought use extra auxiliary systems to avoid danger to their consumers, they are not doing this because it reduces their profit margins, alas, this hn thread
numpad0 4y ago
> humans can drive cars Needs emphasis on can
freejazz 4y ago
Yes, in theory, something sufficient will suffice. I think we are trying to bring some clarity to the topic, however.
sangnoir 4y ago
Isn't this a bit like saying we can do better than fixed-wing aircraft, because birds can flap their wings? With sufficiently advanced material science, flapping-wing human flight too, is possible. But that doesn't mean Boeing and Cessna are misguided.
TulliusCicero 4y ago
> Yet somehow, humans can drive cars with just a pair of optical sensors A pair of optical sensors and a compute engine vastly superior to anything that we will have in the near future for self-driving cars. Humans can do fine with driving on just a couple of cameras because we have an excellent mental model (at least when not distracted, tired, drunk, etc.). Cars won't have that solid of a mental model for a long, long time, so sensor superiority is a way to compensate for that.
tsimionescu 4y ago
The optical sensors are just a small part of the human (and animal in general) vision system. A much bigger component is our innate (evolutionarily acquired) understanding of basic mechanics, simple agent theory, and object recognition. When we look at the road, we recognize stuff in the images we get as objects, and then most of the work is done by us applying basic logic in terms of those objects - that car is off the side of the road so it's stationary; that color change is due to a police light, not a change in the composition of objects; that small blob is a normal-size far-away car, not a small and near car; that thing on the road is a shadow, not a car, since I can tell that the overpass is casting it and it aligns with other shadows. All of these things are not relying on optics for interpreting the received image (though effects such as parallax do play a role as well, it is actually quite minimal), they are interpreting the image at a slightly higher level of abstraction by applying some assumptions and heuristics that evolution has "found". Without these assumptions, there simply isn't enough information in an image, even with the best possible camera, to interpret the needed details.
- Reason077 4y ago
  > "A much bigger component is our innate (evolutionarily acquired) understanding of basic mechanics, simple agent theory, and object recognition. ... they are interpreting the image at a slightly higher level of abstraction by applying some assumptions and heuristics that evolution has "found"." Of course, and all this is exactly what self-driving AIs are attempting to implement. Things like object recognition and understanding basic physics are already well-solved problems. Higher-level problem-solving and reasoning about / predicting behaviour of the objects you can see is harder, but (presumably) AI will get there some day.
  - tsimionescu 4y ago
    Putting all of these together amounts to building AGI. While I do believe that we will have that one day, I have a very hard time imagining as the quickest path to self-driving. Basically my contention is that vision-only is being touted as the more focused path to self-driving, when in fact vision-only clearly requires a big portion at least of an AGI. I think it's pretty clear this currently means this is not a realistic path to self-driving, while other paths to self-driving using more specialized sensors seem more likely to bear fruit in the near term.
bonestamp2 4y ago
Yes, a "pair" of optical sensors. Tesla is at a disadvantage compared to humans -- they do not do stereoscopic imaging, which makes distance of objects less reliable -- they try to infer distance from a single flat image. Humans having two sensors pointed in the same direction gives us a very reliable way of determining distance (up to a relevant distance for driving at least).
- lolc 4y ago
  Interestingly, even people with missing stereoscopic vision are allowed to drive. We don't require depth perception to drive. The assumption is that they can compensate.
  - mrguyorama 4y ago
    Binocular vision isn't even the only source of depth information available to humans. That's why someone missing an eye can still make reasonable depth estimations.
cbsmith 4y ago
Yes, and a sufficiently smart compiler should make optimization unnecessary. ;-)
cbsmith 4y ago
> Yet somehow, humans can drive cars with just a pair of optical sensors (mounted on a swivelling gimbal, of sorts). In fairness, humans have a lot more than just optical sensors at their disposal, and are pretty terrible drivers. We've added all kinds of safety features to cars and roads to try to compensate for their weaknesses, and it certainly helps, but they still make mistakes with alarming regularity, and they crash all the time. When you have a human driver, conversations about safety and sensor information seem so straightforward. The idea of a car maker saving a buck by foregoing some tool or technology at the expense of safety is largely a non-starter. What's weird is, with a computer driver, (which has unique advantages and disadvantages as compared to a human driver) the conversation is somehow entirely different.
- tibbydudeza 4y ago
  My kid took 12 driving lessons of an hour long before she could drive stick shift on a road albeit very slowly.
- camgunz 4y ago
  > We've added all kinds of safety features to cars and roads to try to compensate for their weaknesses This is a super important point. Whenever self-driving cars comes up in conversation it's like, "we're spending billions of dollars on self-driving cars tech, but what if we just, idk, had rails instead of roads". We're putting all the complexity on the self-driving tech, but it seems pretty clear that if we helped a little on the other end (made driving easier for computers), everything would get better a lot faster.
2muchcoffeeman 4y ago
>Yet somehow, humans can drive cars with just a pair of optical sensors (mounted on a swivelling gimbal, of sorts). This is wrong and I was surprised to hear them say it was enough in the video. We don't have car horns and sirens for your eyes. You will often hear something long before you see it. This is important for emergency vehicles. Once you hear it, a good driver will immediately slow down and pull to the side, or delay movement to give space for the vehicle. Does this mean self driving vehicles can't detect emergency vehicles until they appear on camera? That's not encouraging.
- gizmo 4y ago
  Deaf people (or those who blast music) can drive. People who are blind in one eye can drive.
  - mrguyorama 4y ago
    That might have something to do with the general intelligence prediction supercomputer sitting between the ears. If Tesla is saying they won't have real (not just an 80 percent solution that they then lie and say is complete) self driving until they develop an AGI, I agree
- dsfyu404ed 4y ago
  >Once you hear it, a good driver will immediately slow down and pull to the side, or delay movement to give space for the vehicle. Robotically performing an action in response to single/few stimuli with little consideration for the rest of the setting and whether other responses could yield more optimal results precludes one from ever being a "good" driver IMO. "See lights, pull over" is not going to cut it. See any low effort "idiot drivers and emergency vehicles" type youtube compilation for examples of why these sorts of approaches fall short.
codeflo 4y ago
> In theory, a sufficiently capable AI should be able to drive a car at least as well as a human can using the same input: vision. In theory, cars should be use mechanical legs instead of wheels for transportation, that's how animals do it. In theory, plane wings should flap around, that's the way birds do it. My point being: the way biology solved something may not always be the best way to do it with technology.
- dotancohen 4y ago
  GP was stating that "two cameras mounted 15cm apart on a swivel slightly left of the vehicle center of geometry" has proven to be a _sufficient_ solution, not necessarily the best solution.
- Reason077 4y ago
  > ”In theory, cars should be use mechanical legs instead of wheels for transportation, that's how animals do it.” Wheels and legs solve different problems. Wheels aren’t very useful without perfectly smooth surfaces to run them on. If roads were a natural phenomenon that had existed millions of years ago, then isn’t it plausible that some animals might have evolved wheels to move around faster and more efficiently?
tibbydudeza 4y ago
Running neural networks tweaked by million years of evolution.
naijaboiler 4y ago
Optical sensors, an innate understanding of they works around them that they are previewing. And most importantly, a social understanding of what other humans around them are likely to do.
RC_ITR 4y ago
You seem to be fine with the 30k Americans per year who die in car crashes. That number is probably too high for robots to do though. Humans are weird like that.
michael1999 4y ago
But that's not how people drive. They use their ears, they move their head around to generate parallax, they read the body-language of other drivers, they make eye-contact at intersections, they shift position to look around pillars, or stoop to see an inconveniently placed stop light. Fixed forward cameras do none of that.
mrguyorama 4y ago
Our two eyeballs plus brain is SO MUCH MORE than just two mediocre CCDs. Our eyes provide distance sensing through focusing, the difference in angle of your two eyes looking at a distant object, and other inputs, as well as having incredible range of sensitivity, including a special high contrast mode just for night driving. This incredibly, literally unmatched camera subsystem is then fed into the single best future prediction machine that has ever existed. This machine has a powerful understanding of what things are (classification) and how the world works (simulation) and even physics. This system works to predict and respond to future, currently unseen dangers, and also pick out fast moving objects. Two off the shelf digital image sensors WILL NEVER REPLACE ALL OF THAT. There's literally not enough input. Binocular "vision" with shitty digital image sensors is not enough. Humans are stupidly good at driving. Pretty much the only serious accidents nowadays are ones where people turn off some of their sensors (look away from the road at something else, or drugs and alcohol) or turn off their brain (distractions, drugs and alcohol, and sleeping at the wheel).

fennecfoxy 4y ago

I believe in the sentiment, but it's true that we humans also crash our cars a LOT. From minor bumps and scrapes to multiple fatalities.

CyanBird 4y ago

> sufficiently capable AI And Tesla lacks that, so therefore they ought not simply rely on cameras and ought use extra auxiliary systems to avoid danger to their consumers, they are not doing this because it reduces their profit margins, alas, this hn thread

numpad0 4y ago

> humans can drive cars Needs emphasis on can

freejazz 4y ago

Yes, in theory, something sufficient will suffice. I think we are trying to bring some clarity to the topic, however.

sangnoir 4y ago

Isn't this a bit like saying we can do better than fixed-wing aircraft, because birds can flap their wings? With sufficiently advanced material science, flapping-wing human flight too, is possible. But that doesn't mean Boeing and Cessna are misguided.

TulliusCicero 4y ago

> Yet somehow, humans can drive cars with just a pair of optical sensors A pair of optical sensors and a compute engine vastly superior to anything that we will have in the near future for self-driving cars. Humans can do fine with driving on just a couple of cameras because we have an excellent mental model (at least when not distracted, tired, drunk, etc.). Cars won't have that solid of a mental model for a long, long time, so sensor superiority is a way to compensate for that.

tsimionescu 4y ago

The optical sensors are just a small part of the human (and animal in general) vision system. A much bigger component is our innate (evolutionarily acquired) understanding of basic mechanics, simple agent theory, and object recognition. When we look at the road, we recognize stuff in the images we get as objects, and then most of the work is done by us applying basic logic in terms of those objects - that car is off the side of the road so it's stationary; that color change is due to a police light, not a change in the composition of objects; that small blob is a normal-size far-away car, not a small and near car; that thing on the road is a shadow, not a car, since I can tell that the overpass is casting it and it aligns with other shadows. All of these things are not relying on optics for interpreting the received image (though effects such as parallax do play a role as well, it is actually quite minimal), they are interpreting the image at a slightly higher level of abstraction by applying some assumptions and heuristics that evolution has "found". Without these assumptions, there simply isn't enough information in an image, even with the best possible camera, to interpret the needed details.

Reason077 4y ago
> "A much bigger component is our innate (evolutionarily acquired) understanding of basic mechanics, simple agent theory, and object recognition. ... they are interpreting the image at a slightly higher level of abstraction by applying some assumptions and heuristics that evolution has "found"." Of course, and all this is exactly what self-driving AIs are attempting to implement. Things like object recognition and understanding basic physics are already well-solved problems. Higher-level problem-solving and reasoning about / predicting behaviour of the objects you can see is harder, but (presumably) AI will get there some day.
- tsimionescu 4y ago
  Putting all of these together amounts to building AGI. While I do believe that we will have that one day, I have a very hard time imagining as the quickest path to self-driving. Basically my contention is that vision-only is being touted as the more focused path to self-driving, when in fact vision-only clearly requires a big portion at least of an AGI. I think it's pretty clear this currently means this is not a realistic path to self-driving, while other paths to self-driving using more specialized sensors seem more likely to bear fruit in the near term.

bonestamp2 4y ago

Yes, a "pair" of optical sensors. Tesla is at a disadvantage compared to humans -- they do not do stereoscopic imaging, which makes distance of objects less reliable -- they try to infer distance from a single flat image. Humans having two sensors pointed in the same direction gives us a very reliable way of determining distance (up to a relevant distance for driving at least).

lolc 4y ago
Interestingly, even people with missing stereoscopic vision are allowed to drive. We don't require depth perception to drive. The assumption is that they can compensate.
- mrguyorama 4y ago
  Binocular vision isn't even the only source of depth information available to humans. That's why someone missing an eye can still make reasonable depth estimations.

cbsmith 4y ago

Yes, and a sufficiently smart compiler should make optimization unnecessary. ;-)

cbsmith 4y ago

> Yet somehow, humans can drive cars with just a pair of optical sensors (mounted on a swivelling gimbal, of sorts). In fairness, humans have a lot more than just optical sensors at their disposal, and are pretty terrible drivers. We've added all kinds of safety features to cars and roads to try to compensate for their weaknesses, and it certainly helps, but they still make mistakes with alarming regularity, and they crash all the time. When you have a human driver, conversations about safety and sensor information seem so straightforward. The idea of a car maker saving a buck by foregoing some tool or technology at the expense of safety is largely a non-starter. What's weird is, with a computer driver, (which has unique advantages and disadvantages as compared to a human driver) the conversation is somehow entirely different.

tibbydudeza 4y ago
My kid took 12 driving lessons of an hour long before she could drive stick shift on a road albeit very slowly.
camgunz 4y ago
> We've added all kinds of safety features to cars and roads to try to compensate for their weaknesses This is a super important point. Whenever self-driving cars comes up in conversation it's like, "we're spending billions of dollars on self-driving cars tech, but what if we just, idk, had rails instead of roads". We're putting all the complexity on the self-driving tech, but it seems pretty clear that if we helped a little on the other end (made driving easier for computers), everything would get better a lot faster.

2muchcoffeeman 4y ago

>Yet somehow, humans can drive cars with just a pair of optical sensors (mounted on a swivelling gimbal, of sorts). This is wrong and I was surprised to hear them say it was enough in the video. We don't have car horns and sirens for your eyes. You will often hear something long before you see it. This is important for emergency vehicles. Once you hear it, a good driver will immediately slow down and pull to the side, or delay movement to give space for the vehicle. Does this mean self driving vehicles can't detect emergency vehicles until they appear on camera? That's not encouraging.

gizmo 4y ago
Deaf people (or those who blast music) can drive. People who are blind in one eye can drive.
- mrguyorama 4y ago
  That might have something to do with the general intelligence prediction supercomputer sitting between the ears. If Tesla is saying they won't have real (not just an 80 percent solution that they then lie and say is complete) self driving until they develop an AGI, I agree
dsfyu404ed 4y ago
>Once you hear it, a good driver will immediately slow down and pull to the side, or delay movement to give space for the vehicle. Robotically performing an action in response to single/few stimuli with little consideration for the rest of the setting and whether other responses could yield more optimal results precludes one from ever being a "good" driver IMO. "See lights, pull over" is not going to cut it. See any low effort "idiot drivers and emergency vehicles" type youtube compilation for examples of why these sorts of approaches fall short.

codeflo 4y ago

> In theory, a sufficiently capable AI should be able to drive a car at least as well as a human can using the same input: vision. In theory, cars should be use mechanical legs instead of wheels for transportation, that's how animals do it. In theory, plane wings should flap around, that's the way birds do it. My point being: the way biology solved something may not always be the best way to do it with technology.

dotancohen 4y ago
GP was stating that "two cameras mounted 15cm apart on a swivel slightly left of the vehicle center of geometry" has proven to be a _sufficient_ solution, not necessarily the best solution.
Reason077 4y ago
> ”In theory, cars should be use mechanical legs instead of wheels for transportation, that's how animals do it.” Wheels and legs solve different problems. Wheels aren’t very useful without perfectly smooth surfaces to run them on. If roads were a natural phenomenon that had existed millions of years ago, then isn’t it plausible that some animals might have evolved wheels to move around faster and more efficiently?

tibbydudeza 4y ago

Running neural networks tweaked by million years of evolution.

naijaboiler 4y ago

Optical sensors, an innate understanding of they works around them that they are previewing. And most importantly, a social understanding of what other humans around them are likely to do.

RC_ITR 4y ago

You seem to be fine with the 30k Americans per year who die in car crashes. That number is probably too high for robots to do though. Humans are weird like that.

michael1999 4y ago

But that's not how people drive. They use their ears, they move their head around to generate parallax, they read the body-language of other drivers, they make eye-contact at intersections, they shift position to look around pillars, or stoop to see an inconveniently placed stop light. Fixed forward cameras do none of that.

mrguyorama 4y ago

Our two eyeballs plus brain is SO MUCH MORE than just two mediocre CCDs. Our eyes provide distance sensing through focusing, the difference in angle of your two eyes looking at a distant object, and other inputs, as well as having incredible range of sensitivity, including a special high contrast mode just for night driving. This incredibly, literally unmatched camera subsystem is then fed into the single best future prediction machine that has ever existed. This machine has a powerful understanding of what things are (classification) and how the world works (simulation) and even physics. This system works to predict and respond to future, currently unseen dangers, and also pick out fast moving objects. Two off the shelf digital image sensors WILL NEVER REPLACE ALL OF THAT. There's literally not enough input. Binocular "vision" with shitty digital image sensors is not enough. Humans are stupidly good at driving. Pretty much the only serious accidents nowadays are ones where people turn off some of their sensors (look away from the road at something else, or drugs and alcohol) or turn off their brain (distractions, drugs and alcohol, and sleeping at the wheel).