3 ms·

They have a fairly big weakness. If someone takes over their control these taxis are basically weapons. You have to admit that the question is not if this is po

They have a fairly big weakness. If someone takes over their control these taxis are basically weapons. You have to admit that the question is not if this is possible but how will it pan out.

huygens6363 2y ago
I agree, but that’s an argument against cars in general. Turns out cars are a bit too bulky and pricey to repurpose as shanks.
nmca 2y ago
This is already true of many consumer vehicles and thus less exacerbated by Waymo than you imply
greiskul 2y ago
People can literally just go to a car rental service right now and do that as well. Or use a car service app. Or rob someone elses car. Most people in society don't really have a desire to run amok.
- fire_lake 2y ago
  I think OP means remotely control the car.
  - tialaramex 2y ago
    The problem is that you can't "remotely control the car". If you have the same level of access as Waymo's actual humans you can suggest to a Waymo driver (ie the machine) hey, the way to resolve this situation you are in might be to inch forward into this position - and maybe in some cases you can trick it into causing harm this way. However I'd guess in the vast majority of cases all you do is trap it so that it gets into a situation where it can't see any action it's allowed to take which gets closer to its goals - so it gives up and just sits there. The most crucial insight for Self Driving Cars is that this is not the trolley problem. "I give up, stop where I am" is a valid answer. We actually have built automation where "just give up" isn't a valid answer. CAT IIIc autoland (on a jet liner) has "Fail Active" scenarios where the machine concludes just before touch down that it no longer has confidence in its position due to one or more sudden sensor failures but the human pilots can't possibly intercede quickly enough to be safe and under IIIc conditiosn they can't see anything anyway, so, although the aeroplane will tell the human pilots that a failure occurred, it will nevertheless attempt to continue the now unsafe landing in this edge case. Most likely despite the reduced sensor validity, this is successful and everybody lives, and if not it's not as though the humans could have reacted in time anyway. But self driving isn't like that, the plane is flying, if it were to just stop flying everybody dies. In contrast a car can just stop and it's merely annoying.
    - fire_lake 2y ago
      > The problem is that you can't "remotely control the car". The assumption is that the highly networked car has some kind of security vulnerability that allows malicious users to take control and perform acts of terror. Since this is in software, the attacker can theoretically scale the attack to involve all cars on the road with the same vulnerabilities, which could be millions of vehicles. This problem is not unique to self driving cars - just any highly networked car with software control of key systems, like a modern Tesla. However, a full self driving car may not manual overrides that allow the passenger to stop the vehicle.
Mashimo 2y ago
In theory you could also hack the many self driving trains that exists around the globe.
- tialaramex 2y ago
  Two things about the self-driving trains make that even less appealing than a Waymo 1. They're mostly even more local. The Waymo Driver is in your car, the "driver" for say a DLR train is inside the train too. Unlike Waymo they aren't running multiple live feeds to remote oversight, even the emergency human intervention is literally on board with you. There's somebody wearing a uniform telling those tourists that no, Abbey Road is an outer suburb with a sewage pumping station, they're on the wrong train for the famous Beatles photograph. The person in the uniform is trained to drive the train if there's some reason the automation can't do it, nobody can do that from a control room miles away. In the even higher (and rare) GoA systems where nobody aboard can drive the train even if they need to, remote oversight still may need to dispatch a specialist to rescue a failed train. 2. They're mostly "grade separated" that is, they're either underground or suspended in the air, or maybe in fenced off ground-level areas, so you can't use a "hacked" train to hurt anybody except its passengers or maybe, in some cases, passengers on a nearby train.
  - Mashimo 2y ago
    > so you can't use a "hacked" train to hurt anybody except its passengers or maybe, in some cases, passengers on a nearby train. Yes, is that not enough?

They have a fairly big weakness. If someone takes over their control these taxis are basically weapons. You have to admit that the question is not if this is possible but how will it pan out.

huygens6363 2y ago

I agree, but that’s an argument against cars in general. Turns out cars are a bit too bulky and pricey to repurpose as shanks.

nmca 2y ago

This is already true of many consumer vehicles and thus less exacerbated by Waymo than you imply

greiskul 2y ago

People can literally just go to a car rental service right now and do that as well. Or use a car service app. Or rob someone elses car. Most people in society don't really have a desire to run amok.

fire_lake 2y ago
I think OP means remotely control the car.
- tialaramex 2y ago
  The problem is that you can't "remotely control the car". If you have the same level of access as Waymo's actual humans you can suggest to a Waymo driver (ie the machine) hey, the way to resolve this situation you are in might be to inch forward into this position - and maybe in some cases you can trick it into causing harm this way. However I'd guess in the vast majority of cases all you do is trap it so that it gets into a situation where it can't see any action it's allowed to take which gets closer to its goals - so it gives up and just sits there. The most crucial insight for Self Driving Cars is that this is not the trolley problem. "I give up, stop where I am" is a valid answer. We actually have built automation where "just give up" isn't a valid answer. CAT IIIc autoland (on a jet liner) has "Fail Active" scenarios where the machine concludes just before touch down that it no longer has confidence in its position due to one or more sudden sensor failures but the human pilots can't possibly intercede quickly enough to be safe and under IIIc conditiosn they can't see anything anyway, so, although the aeroplane will tell the human pilots that a failure occurred, it will nevertheless attempt to continue the now unsafe landing in this edge case. Most likely despite the reduced sensor validity, this is successful and everybody lives, and if not it's not as though the humans could have reacted in time anyway. But self driving isn't like that, the plane is flying, if it were to just stop flying everybody dies. In contrast a car can just stop and it's merely annoying.
  - fire_lake 2y ago
    > The problem is that you can't "remotely control the car". The assumption is that the highly networked car has some kind of security vulnerability that allows malicious users to take control and perform acts of terror. Since this is in software, the attacker can theoretically scale the attack to involve all cars on the road with the same vulnerabilities, which could be millions of vehicles. This problem is not unique to self driving cars - just any highly networked car with software control of key systems, like a modern Tesla. However, a full self driving car may not manual overrides that allow the passenger to stop the vehicle.

Mashimo 2y ago

In theory you could also hack the many self driving trains that exists around the globe.

tialaramex 2y ago
Two things about the self-driving trains make that even less appealing than a Waymo 1. They're mostly even more local. The Waymo Driver is in your car, the "driver" for say a DLR train is inside the train too. Unlike Waymo they aren't running multiple live feeds to remote oversight, even the emergency human intervention is literally on board with you. There's somebody wearing a uniform telling those tourists that no, Abbey Road is an outer suburb with a sewage pumping station, they're on the wrong train for the famous Beatles photograph. The person in the uniform is trained to drive the train if there's some reason the automation can't do it, nobody can do that from a control room miles away. In the even higher (and rare) GoA systems where nobody aboard can drive the train even if they need to, remote oversight still may need to dispatch a specialist to rescue a failed train. 2. They're mostly "grade separated" that is, they're either underground or suspended in the air, or maybe in fenced off ground-level areas, so you can't use a "hacked" train to hurt anybody except its passengers or maybe, in some cases, passengers on a nearby train.
- Mashimo 2y ago
  > so you can't use a "hacked" train to hurt anybody except its passengers or maybe, in some cases, passengers on a nearby train. Yes, is that not enough?