4 ms·

I always question whether humans are really any better at this calculation. In the gnome example, many people will probably stop to avoid damaging their propert

I always question whether humans are really any better at this calculation. In the gnome example, many people will probably stop to avoid damaging their property, not see the car behind them careening forward, and end up getting rear ended anyway. In an ideal world, ethical issues like the one you presented would be major hurdles but humans aren't going to be perfectly rational in the moment of many accidents, especially the most dangerous high-speed ones when there is physically not enough time to do ethical math. > we all still get in cars believing we'll be better than other drivers. Who's going to get in a car when any given car has an equivalent chance of making a bad decision all on its own? Given the fact that plenty of people take taxis, rely on Uber and Lyft drivers they've never met, and take public transportation like aircraft and trains with little knowledge of the operator, this seems like a non-issue to me.

oaktowner 9y ago
The other thing to bear in mind: I would guess that the vast majority of accidents are caused by people not paying attention -- distracted by a phone, person, great view, or even just because they're looking the wrong direction. Cars don't do any of those things. They will always be paying attention, looking in all directions at once. So if they eliminate all of the accidents that are caused that way, but increase the number of garden-gnome related accidents...wouldn't that be worthwhile? Another, less flippant way to say it: yes, we can imagine scenarios in which the number of accidents will increase. But what is the relative frequency of those types of accidents compared to run-of-the-mill rear-end, red-light run, lane change due to inattentiveness?
- frenchie4111 9y ago
  And even if the number of accidents increases: Does that increase cause greater monetary loss than the considerable cost savings of not paying drivers.
- otakucode 9y ago
  If given a choice between changing from what you are used to which has drawbacks, to something new that has drawbacks, even if those drawbacks are markedly milder, almost everyone will choose to stick with where they already are. Consciously choosing a position which has drawbacks that you know about makes you feel personally responsible for those when they occur. That is infinitely worse than suffering the drawbacks of the "default" you stay with which you can give yourself a moral pass on because it wasn't a situation you chose to begin with. There's also the issue of what a self-driving car could do... and what they WILL do. Much technology COULD be amazing... if companies did not function in the ways that they actually do. Because of that, technology often fails to meet any of the promises made. Development is supposed to be cheap and fast, and safety and security get left at the door. No company is held liable for such things, and it would be expensive and bothersome to fix, so they're not about to bother.
irq11 9y ago
There's no ethical question here. You either see that it's a gnome, or you see that it's a child. Humans may or may not be less situationally aware overall, but we have the gnome question down cold. Cars don't. They see "obstacle in front of bumper radar." The OPs point is well-taken because it illustrates a technical risk that few are discussing: the "last 1%" of self-driving software may be equivalent to solving AGI.
- bobsil1 9y ago
  >the "last 1%" of self-driving software may be equivalent to solving AGI. Exactly right. You need a semantic world model, reasoning and prediction. AGI bottlenecks self-driving, assistants, text understanding…
  - bitL 9y ago
    I think we are overestimating amount of intelligence we humans need while driving...
    - bobsil1 9y ago
      The edge cases are unclear: construction zone, heavy equipment, school kids, another car poised to enter the road.
      - bitL 9y ago
        They are probabilistically modeled already. You keep tracking other objects and compute various possibilities for their movement and depending on the past and present behavior pick the most probable one. If car encounters lower-than-threshold probability, it can always slow down or stop, like what non-reckless humans would do. Even end-to-end deep learning can recognize construction zones and learn to drive around traffic cones.
        bobsil1 9y ago
        The edge cases are endless (wheelchair woman chasing a chicken), and without a semantic world model the car will have no idea what to do. Low-level object avoidance architecture isn't enough.
otakucode 9y ago
Are humans any better at flying? Doesn't seem to impact how they feel about getting into an airplane.

oaktowner 9y ago

The other thing to bear in mind: I would guess that the vast majority of accidents are caused by people not paying attention -- distracted by a phone, person, great view, or even just because they're looking the wrong direction. Cars don't do any of those things. They will always be paying attention, looking in all directions at once. So if they eliminate all of the accidents that are caused that way, but increase the number of garden-gnome related accidents...wouldn't that be worthwhile? Another, less flippant way to say it: yes, we can imagine scenarios in which the number of accidents will increase. But what is the relative frequency of those types of accidents compared to run-of-the-mill rear-end, red-light run, lane change due to inattentiveness?

frenchie4111 9y ago
And even if the number of accidents increases: Does that increase cause greater monetary loss than the considerable cost savings of not paying drivers.
otakucode 9y ago
If given a choice between changing from what you are used to which has drawbacks, to something new that has drawbacks, even if those drawbacks are markedly milder, almost everyone will choose to stick with where they already are. Consciously choosing a position which has drawbacks that you know about makes you feel personally responsible for those when they occur. That is infinitely worse than suffering the drawbacks of the "default" you stay with which you can give yourself a moral pass on because it wasn't a situation you chose to begin with. There's also the issue of what a self-driving car could do... and what they WILL do. Much technology COULD be amazing... if companies did not function in the ways that they actually do. Because of that, technology often fails to meet any of the promises made. Development is supposed to be cheap and fast, and safety and security get left at the door. No company is held liable for such things, and it would be expensive and bothersome to fix, so they're not about to bother.

irq11 9y ago

There's no ethical question here. You either see that it's a gnome, or you see that it's a child. Humans may or may not be less situationally aware overall, but we have the gnome question down cold. Cars don't. They see "obstacle in front of bumper radar." The OPs point is well-taken because it illustrates a technical risk that few are discussing: the "last 1%" of self-driving software may be equivalent to solving AGI.

bobsil1 9y ago
>the "last 1%" of self-driving software may be equivalent to solving AGI. Exactly right. You need a semantic world model, reasoning and prediction. AGI bottlenecks self-driving, assistants, text understanding…
- bitL 9y ago
  I think we are overestimating amount of intelligence we humans need while driving...
  - bobsil1 9y ago
    The edge cases are unclear: construction zone, heavy equipment, school kids, another car poised to enter the road.
    - bitL 9y ago
      They are probabilistically modeled already. You keep tracking other objects and compute various possibilities for their movement and depending on the past and present behavior pick the most probable one. If car encounters lower-than-threshold probability, it can always slow down or stop, like what non-reckless humans would do. Even end-to-end deep learning can recognize construction zones and learn to drive around traffic cones.
      - bobsil1 9y ago
        The edge cases are endless (wheelchair woman chasing a chicken), and without a semantic world model the car will have no idea what to do. Low-level object avoidance architecture isn't enough.

otakucode 9y ago

Are humans any better at flying? Doesn't seem to impact how they feel about getting into an airplane.