4 ms·

It really really bothers me that these folks are using a live city with real, non-volunteer test subjects of all ages (little kids and old folks use public stre

It really really bothers me that these folks are using a live city with real, non-volunteer test subjects of all ages (little kids and old folks use public streets) as a test bed for their massive car-shaped robots. It's bad enough that people are driving cars all over the place, car collisions have killed more Americans than all the wars we've fought put together. I'm one of those people who say, "Self-driving cars can't happen soon enough." But I don't think that justifies e.g. killing Elaine Herzberg. Ask yourself this, why start with cars? Why not make a self-driving golf cart? Make it out of nerf (soft foam) and program it to never go so fast that it can't brake in time to prevent collision. Testing these heavy, fast, buggy robots in crowds of people is extremely irresponsible.

torpfactory 7y ago
There is a different perspective that you could use (and I’m not necessarily advocating for it; hear me out): Human driven cars are dangerous to the tune of ~36,000 deaths per year. Every year without the implementation of full self driving we pay some large percentage of that number in lives. Self driving cars won’t make it out of the lab without real driving on real roads in real scenarios. Taking appropriate precautions (a human safety driver, maybe two) and testing in the real world might save more lives overall than keeping the vehicles in a more lab-like setting for longer, and missing some of the complexity of the real thing.
- carapace 7y ago
  What about the nerf-golf-cart idea?
  - pj_mukh 7y ago
    Most of these companies run closed-circuit tests (aka nerf-golf-carts) and 1000's of hours of simulation. In the end, the only test that matters is the real-test. Real speed (sensor/computation latency et al.), real sensor feeds (lighting et al.) and real car size (momentum et al.). Source: Am Controls engineer. There WILL be bugs and un-modellable sources of error. The real hedging in these situations is the safety driver. The death of Elaine Herzberg is very regrettable, but the fault ultimately lies with the safety driver and the training that was offered to her. She was on her phone, like 1000's of drivers are now.
    - carapace 7y ago
      > Most of these companies run closed-circuit tests (aka nerf-golf-carts) and 1000's of hours of simulation. I don't think we're talking about the same thing. I mean build a machine that, in the real world, can't hurt people. Make it light. Make it soft. Program it to limit its speed such that it can always stop before colliding with whatever (whoever) might leap out in front of it. If the top speed is five miles per hour, so be it. The safety driver is wrong too. But she was there because Uber wanted to put car-shaped robots onto public streets. Really the insane thing is that we mix car and pedestrian traffic at all in the first place. Oddly enough, it's the result of a deliberate campaign of propaganda: https://www.youtube.com/watch?v=-AFn7MiJz_s https://www.youtube.com/watch?v=-AFn7MiJz_s "Adam Ruins Everything - Why Jaywalking Is a Crime" > Adam reveals the derogatory origins of jaywalking and explains how the auto industry made it illegal.
      - pj_mukh 7y ago
        "If the top speed is five miles per hour, so be it." You are not really testing though. The whole point is to build a machine that can go the speed limit. You can test all you want at 5mph, but let me assure you, most of the real issues will show up when you go 45 in the real world.
        carapace 7y ago
        > The whole point is to build a machine that can go the speed limit. Sure, eventually, when the machines and sensors and algorithms and so on are good enough. When the infrastructure can be rebuilt to accommodate them (e.g. Boring company tunnels for auto-trucking, sensors and comms in the roads and signage, &c.) The rush to market is the whole problem. Not the ultimate concept. I want a machine that can take my mother to her doctors appointment now that the dementia has gotten to the point where she shouldn't take the bus on her own anymore. Let the experiences with the toy cars guide the incremental graceful adoption of faster machines. > most of the real issues will show up when you go 45 in the real world. Right! So don't go 45.
        7y ago
  - torpfactory 7y ago
    Speed is an important parameter: you can’t do freeway driving in a golf cart. IMO regular vehicles can be safe enough with appropriate supervision. It’s really once you get to the first runs without safety drivers where your backup goes away. Hopefully that will be only after your intervention rate is zero.
    - carapace 7y ago
      We can work our way to that point (freeway driving) without killing people. Really, the problem is the rush to market not the idea itself.
      - torpfactory 7y ago
        That’s the heart of the original argument I posed on this thread: we really should rush to market because the status quo is quite unsafe.
        carapace 7y ago
        The status quo is insane, IMO. I call our mixed ped/car transportation networks the "mayhem lottery" (As in, every time you go out there you're taking a chance that you won't come back with all your limbs, or your life.) It was years ago, but a neighbors son was crossing the street in a crosswalk when someone ran a stop sign and knocked him fifty feet. I've twice seen little old ladies lying dead in the street from hit and run drivers. Making bad imitations of KITT from Knight Rider is not the solution here. https://en.wikipedia.org/wiki/Traffic_calming https://en.wikipedia.org/wiki/Traffic_calming is cheaper and easier, for example. Look I want robot cars, okay? I even want them ASAP. As soon as possible without testing robots on public ways. Build a fake city, populate it with people who have signed waivers, test there. Sure it's expensive but at least you don't risk killing more innocent people with your experimental car-shaped robots. I mean, what if I built a television with H.E.M? (Human Eradication Mode) If it escapes and kills someone, isn't that my fault? Maybe if I put a human in there, give her a phone to distract her, and call her "Safety Driver", I can deflect any blame for my robot's killing spree onto her and get away with murder scott free! It's the perfect crime! Okay, sorry, I got a little carried away there. But I hope my point is clear: Safety: Yes. Robot Cars: also yes, but obeying the First Law of Robotics. Testing robots cars on innocent people: hard no.
AndrewKemendo 7y ago
You can't learn to operate in environments you don't train in. It would be great if we had a solution to the out-of-distribution inference/reward problem, but I don't think it really exists. I'm firmly in the "Perfect for freight, questionable value for consumers" camp WRT autonomous cars. I also think it's irresponsible to do this but the reality is, they are doing all the socially "appropriate" things, like get approval from the city.
- carapace 7y ago
  It's one of those things where I wonder if I'm not being too much of a curmudgeon. I'm sure the case could be made that these things will reduce overall traffic deaths long before they become perfect drivers, eh
  - AndrewKemendo 7y ago
    Yeah I don't buy that argument. These systems would have to be at least human level and also so pervasive that it would be almost like an inoculation. At that point there are cheaper and easier ways to do that, which by the way are already happening. If you buy a modern car they have very impressive look ahead/smart cruise and lane keeping systems.
    - carapace 7y ago
      I agree with you, I think. To me it seems that you pretty much need AGI to drive IRL. And there are a lot of cheaper and easier things to do in the meantime. Cheers
stale2002 7y ago
Have you ever driven a car, or been in a car? If so, why? Do you realize that by doing so, you are putting other people in danger? Maybe you are the one who is being just as irresponsible by being a person who has even been in a car, ever.
anigbrowl 7y ago
In the video, which was recorded with a safety driver behind the wheel I can live with it. Human drivers annoy me so much that throwing the dice on autonomous cars is not a big stressor to me.
KKKKkkkk1 7y ago
I think you're missing the narrative that the self-driving industry is pushing here. They "solved the problem" and their fleets driving around "autonomously" is being done in order to demonstrate this to the public. A golf cart is obviously unsuitable for that purpose. I think this narrative has run out of steam at this point, by the way. Waymo's valuation has gone from $175B to $105B to $30B since 2018. Zoox specifically is now laying off engineers.
- yalogin 7y ago
  Wait, Zoox was valued at 175B at one point? Is this true? Can you point me to any resources on this?
  - Xorlev 7y ago
    GP said: > Waymo's valuation has gone from $175B to $105B to $30B since 2018. Zoox specifically is now laying off engineers. Waymo.
  - z3rgl1ng 7y ago
    They said Waymo had that valuation.
- reitzensteinm 7y ago
  Waymo has had a single round with outside money, and that valued it at $30bn.
z3rgl1ng 7y ago
I agree.
Dumblydorr 7y ago
Those were already made years ago at the start of SDC innovation. A few companies are way beyond the worst human drivers now, there's already a massive amount of motor vehicle death caused by intoxication that we should worry about. Not fantasy robodeaths that we can count on one hand.
- def8cefe 7y ago
  > fantasy robodeaths They're not a fantasy, they've occurred. The reason we can count them on one hand is because few cars currently drive autonomously and there is a fail-safe human at the wheel who (most of the time) is paying attention to the road. Even if autonomous cars are better than human drivers they will still inevitably strike and kill pedestrians and vehicle occupants; they are not a magical solution to vehicle collisions.
baby 7y ago
> It's bad enough that people are driving cars all over the place, car collisions have killed more Americans than all the wars we've fought put together. Do you want this to stop? Then we're going to have this people test their self-driving cars in a real environment. The more we delay this, the more people are dying because of car accidents.
wtvanhest 7y ago
I used to live in north beach on grant and Union, and these cars were driving by constantly. I knew right away that they would show it there. I forgot how annoying it was.
the-dude 7y ago
So you are bothered by this, but then casually compare it to 'all the wars' you have fought? Are you counting US casualties only, or all humans who perished in these adventures?
lonelappde 7y ago
Elaine Herzberg was killed by a human driver not watching the road.
- carapace 7y ago
  My understanding is that the software in the car detected Herzberg and could have stopped the car in time to avoid the collision but that that subroutine had been disabled due to to many false positives. The "safety driver" (in quotes because she turned out to be both unsafe and not actually driving at the time) is also at fault. Certainly, Elaine Herzberg wouldn't have been killed by that car if it wasn't there, eh? Don't test killer robots on the public.

torpfactory 7y ago

There is a different perspective that you could use (and I’m not necessarily advocating for it; hear me out): Human driven cars are dangerous to the tune of ~36,000 deaths per year. Every year without the implementation of full self driving we pay some large percentage of that number in lives. Self driving cars won’t make it out of the lab without real driving on real roads in real scenarios. Taking appropriate precautions (a human safety driver, maybe two) and testing in the real world might save more lives overall than keeping the vehicles in a more lab-like setting for longer, and missing some of the complexity of the real thing.

carapace 7y ago
What about the nerf-golf-cart idea?
- pj_mukh 7y ago
  Most of these companies run closed-circuit tests (aka nerf-golf-carts) and 1000's of hours of simulation. In the end, the only test that matters is the real-test. Real speed (sensor/computation latency et al.), real sensor feeds (lighting et al.) and real car size (momentum et al.). Source: Am Controls engineer. There WILL be bugs and un-modellable sources of error. The real hedging in these situations is the safety driver. The death of Elaine Herzberg is very regrettable, but the fault ultimately lies with the safety driver and the training that was offered to her. She was on her phone, like 1000's of drivers are now.
  - carapace 7y ago
    > Most of these companies run closed-circuit tests (aka nerf-golf-carts) and 1000's of hours of simulation. I don't think we're talking about the same thing. I mean build a machine that, in the real world, can't hurt people. Make it light. Make it soft. Program it to limit its speed such that it can always stop before colliding with whatever (whoever) might leap out in front of it. If the top speed is five miles per hour, so be it. The safety driver is wrong too. But she was there because Uber wanted to put car-shaped robots onto public streets. Really the insane thing is that we mix car and pedestrian traffic at all in the first place. Oddly enough, it's the result of a deliberate campaign of propaganda: https://www.youtube.com/watch?v=-AFn7MiJz_s https://www.youtube.com/watch?v=-AFn7MiJz_s "Adam Ruins Everything - Why Jaywalking Is a Crime" > Adam reveals the derogatory origins of jaywalking and explains how the auto industry made it illegal.
    - pj_mukh 7y ago
      "If the top speed is five miles per hour, so be it." You are not really testing though. The whole point is to build a machine that can go the speed limit. You can test all you want at 5mph, but let me assure you, most of the real issues will show up when you go 45 in the real world.
      - carapace 7y ago
        > The whole point is to build a machine that can go the speed limit. Sure, eventually, when the machines and sensors and algorithms and so on are good enough. When the infrastructure can be rebuilt to accommodate them (e.g. Boring company tunnels for auto-trucking, sensors and comms in the roads and signage, &c.) The rush to market is the whole problem. Not the ultimate concept. I want a machine that can take my mother to her doctors appointment now that the dementia has gotten to the point where she shouldn't take the bus on her own anymore. Let the experiences with the toy cars guide the incremental graceful adoption of faster machines. > most of the real issues will show up when you go 45 in the real world. Right! So don't go 45.
        7y ago
- torpfactory 7y ago
  Speed is an important parameter: you can’t do freeway driving in a golf cart. IMO regular vehicles can be safe enough with appropriate supervision. It’s really once you get to the first runs without safety drivers where your backup goes away. Hopefully that will be only after your intervention rate is zero.
  - carapace 7y ago
    We can work our way to that point (freeway driving) without killing people. Really, the problem is the rush to market not the idea itself.
    - torpfactory 7y ago
      That’s the heart of the original argument I posed on this thread: we really should rush to market because the status quo is quite unsafe.
      - carapace 7y ago
        The status quo is insane, IMO. I call our mixed ped/car transportation networks the "mayhem lottery" (As in, every time you go out there you're taking a chance that you won't come back with all your limbs, or your life.) It was years ago, but a neighbors son was crossing the street in a crosswalk when someone ran a stop sign and knocked him fifty feet. I've twice seen little old ladies lying dead in the street from hit and run drivers. Making bad imitations of KITT from Knight Rider is not the solution here. https://en.wikipedia.org/wiki/Traffic_calming https://en.wikipedia.org/wiki/Traffic_calming is cheaper and easier, for example. Look I want robot cars, okay? I even want them ASAP. As soon as possible without testing robots on public ways. Build a fake city, populate it with people who have signed waivers, test there. Sure it's expensive but at least you don't risk killing more innocent people with your experimental car-shaped robots. I mean, what if I built a television with H.E.M? (Human Eradication Mode) If it escapes and kills someone, isn't that my fault? Maybe if I put a human in there, give her a phone to distract her, and call her "Safety Driver", I can deflect any blame for my robot's killing spree onto her and get away with murder scott free! It's the perfect crime! Okay, sorry, I got a little carried away there. But I hope my point is clear: Safety: Yes. Robot Cars: also yes, but obeying the First Law of Robotics. Testing robots cars on innocent people: hard no.

AndrewKemendo 7y ago

You can't learn to operate in environments you don't train in. It would be great if we had a solution to the out-of-distribution inference/reward problem, but I don't think it really exists. I'm firmly in the "Perfect for freight, questionable value for consumers" camp WRT autonomous cars. I also think it's irresponsible to do this but the reality is, they are doing all the socially "appropriate" things, like get approval from the city.

carapace 7y ago
It's one of those things where I wonder if I'm not being too much of a curmudgeon. I'm sure the case could be made that these things will reduce overall traffic deaths long before they become perfect drivers, eh
- AndrewKemendo 7y ago
  Yeah I don't buy that argument. These systems would have to be at least human level and also so pervasive that it would be almost like an inoculation. At that point there are cheaper and easier ways to do that, which by the way are already happening. If you buy a modern car they have very impressive look ahead/smart cruise and lane keeping systems.
  - carapace 7y ago
    I agree with you, I think. To me it seems that you pretty much need AGI to drive IRL. And there are a lot of cheaper and easier things to do in the meantime. Cheers

stale2002 7y ago

Have you ever driven a car, or been in a car? If so, why? Do you realize that by doing so, you are putting other people in danger? Maybe you are the one who is being just as irresponsible by being a person who has even been in a car, ever.

anigbrowl 7y ago

In the video, which was recorded with a safety driver behind the wheel I can live with it. Human drivers annoy me so much that throwing the dice on autonomous cars is not a big stressor to me.

KKKKkkkk1 7y ago

I think you're missing the narrative that the self-driving industry is pushing here. They "solved the problem" and their fleets driving around "autonomously" is being done in order to demonstrate this to the public. A golf cart is obviously unsuitable for that purpose. I think this narrative has run out of steam at this point, by the way. Waymo's valuation has gone from $175B to $105B to $30B since 2018. Zoox specifically is now laying off engineers.

yalogin 7y ago
Wait, Zoox was valued at 175B at one point? Is this true? Can you point me to any resources on this?
- Xorlev 7y ago
  GP said: > Waymo's valuation has gone from $175B to $105B to $30B since 2018. Zoox specifically is now laying off engineers. Waymo.
- z3rgl1ng 7y ago
  They said Waymo had that valuation.
reitzensteinm 7y ago
Waymo has had a single round with outside money, and that valued it at $30bn.

z3rgl1ng 7y ago

I agree.

Dumblydorr 7y ago

Those were already made years ago at the start of SDC innovation. A few companies are way beyond the worst human drivers now, there's already a massive amount of motor vehicle death caused by intoxication that we should worry about. Not fantasy robodeaths that we can count on one hand.

def8cefe 7y ago
> fantasy robodeaths They're not a fantasy, they've occurred. The reason we can count them on one hand is because few cars currently drive autonomously and there is a fail-safe human at the wheel who (most of the time) is paying attention to the road. Even if autonomous cars are better than human drivers they will still inevitably strike and kill pedestrians and vehicle occupants; they are not a magical solution to vehicle collisions.

baby 7y ago

> It's bad enough that people are driving cars all over the place, car collisions have killed more Americans than all the wars we've fought put together. Do you want this to stop? Then we're going to have this people test their self-driving cars in a real environment. The more we delay this, the more people are dying because of car accidents.

wtvanhest 7y ago

I used to live in north beach on grant and Union, and these cars were driving by constantly. I knew right away that they would show it there. I forgot how annoying it was.

the-dude 7y ago

So you are bothered by this, but then casually compare it to 'all the wars' you have fought? Are you counting US casualties only, or all humans who perished in these adventures?

lonelappde 7y ago

Elaine Herzberg was killed by a human driver not watching the road.

carapace 7y ago
My understanding is that the software in the car detected Herzberg and could have stopped the car in time to avoid the collision but that that subroutine had been disabled due to to many false positives. The "safety driver" (in quotes because she turned out to be both unsafe and not actually driving at the time) is also at fault. Certainly, Elaine Herzberg wouldn't have been killed by that car if it wasn't there, eh? Don't test killer robots on the public.