3 ms·

I would hope that the vehicles trust their own sensors before trusting the claims of other cars. The most a car should be able to lie convincingly about should

I would hope that the vehicles trust their own sensors before trusting the claims of other cars. The most a car should be able to lie convincingly about should be its intentions since they're unobservable. If car A says "I'm going to let you take this left" and car B tries taking the left and then car A tries to ram them, car B should ideally take evasive action (braking or speeding up) as it would from a human driver just randomly charging an intersection. tl;dr: don't trust the client (or other drivers).

georgespencer 10y ago
Those are emergency situations, though. I'm talking specifically about a model for cooperative decision making between AI wherein the AI has access to data from both sets of sensors (in its most limited example), and is therefore able to make a superior decision to if it had only its own data. Simple scenario to illustrate this: car X enters a narrow bidirectional road which is lined with cars. There is enough room only for one car to pass at a time, safely. Car Y enters from the other end. Several more cars enter behind car Y (we'll call these Y1, Y2, Y3, Y4). One car must reverse back down the road from the midway point, in order for any cars to pass through. Car X and its driver cannot see what is behind car Y, but for car Y to reverse it must rely on Y1, Y2, Y3, and Y4 reversing. In order for this to happen, Y4 must first reverse despite not being able to see why it needs to reverse (either through meat or tech sensors). The optimal solution is that sensor data reviewed in the aggregate by each car leads to a cooperative decision that car X should reverse until it can pull over, and allow the other cars to pass before proceeding. In an emergency situation it's also possible to envisage scenarios in which shared data analysis and cooperative decision making are optimal. For example, consider cars X and Y now destined for a high speed, head-on collision. The right lane of the road (car X's right, car Y's left) is clear and the left lane (car X's left, car Y's right) is a deep trench. A primitive or non collaborative AI might suggest that both cars swerve to the same direction to avoid collision, which results in a collision of similar magnitude. A better solution than swerving the two cars into each other might be to swerve one into the ditch and one into the road. The optimal solution is likely to swerve only one car and hard stop the other, knowing that the other car is going to move its direction of travel significantly. This can only be done by giving the cars the ability to make decisions collaboratively.
- BWStearns 10y ago
  Those are valid concerns. For the emergency scenario I have to wonder if it is possible to never trust another car to the point where it can induce a less optimal solution than an independent estimate could produce. For instance if the OtherCar says "Don't worry, I'll swerve into the ditch, just keep driving" and both MyCar and OtherCar keeps driving, we both die, so maybe MyCar says "he said he'd swerve, but I'm going to hard brake instead of driving" because impacting a ditch at speed x is roughly similar to getting hit while stopped at speed x and that's the worst case scenario for braking, whereas the both cars moving at speed x at each other is worse. Not as tight an example as yours I'm afraid, but humans make cooperative decisions all the time where we use information and continue to be suspicious of it and hedge against lies. I think any realistic cooperative tech system where there are untrusted components needs to stay skeptical as well.
  - georgespencer 10y ago
    I completely understand what you mean and agree it's a valid concern. I'm probably super naïve, but I think there's probably a way of digitally signing hardware components so that we can trust sensor data from other cars.

georgespencer 10y ago

Those are emergency situations, though. I'm talking specifically about a model for cooperative decision making between AI wherein the AI has access to data from both sets of sensors (in its most limited example), and is therefore able to make a superior decision to if it had only its own data. Simple scenario to illustrate this: car X enters a narrow bidirectional road which is lined with cars. There is enough room only for one car to pass at a time, safely. Car Y enters from the other end. Several more cars enter behind car Y (we'll call these Y1, Y2, Y3, Y4). One car must reverse back down the road from the midway point, in order for any cars to pass through. Car X and its driver cannot see what is behind car Y, but for car Y to reverse it must rely on Y1, Y2, Y3, and Y4 reversing. In order for this to happen, Y4 must first reverse despite not being able to see why it needs to reverse (either through meat or tech sensors). The optimal solution is that sensor data reviewed in the aggregate by each car leads to a cooperative decision that car X should reverse until it can pull over, and allow the other cars to pass before proceeding. In an emergency situation it's also possible to envisage scenarios in which shared data analysis and cooperative decision making are optimal. For example, consider cars X and Y now destined for a high speed, head-on collision. The right lane of the road (car X's right, car Y's left) is clear and the left lane (car X's left, car Y's right) is a deep trench. A primitive or non collaborative AI might suggest that both cars swerve to the same direction to avoid collision, which results in a collision of similar magnitude. A better solution than swerving the two cars into each other might be to swerve one into the ditch and one into the road. The optimal solution is likely to swerve only one car and hard stop the other, knowing that the other car is going to move its direction of travel significantly. This can only be done by giving the cars the ability to make decisions collaboratively.

BWStearns 10y ago
Those are valid concerns. For the emergency scenario I have to wonder if it is possible to never trust another car to the point where it can induce a less optimal solution than an independent estimate could produce. For instance if the OtherCar says "Don't worry, I'll swerve into the ditch, just keep driving" and both MyCar and OtherCar keeps driving, we both die, so maybe MyCar says "he said he'd swerve, but I'm going to hard brake instead of driving" because impacting a ditch at speed x is roughly similar to getting hit while stopped at speed x and that's the worst case scenario for braking, whereas the both cars moving at speed x at each other is worse. Not as tight an example as yours I'm afraid, but humans make cooperative decisions all the time where we use information and continue to be suspicious of it and hedge against lies. I think any realistic cooperative tech system where there are untrusted components needs to stay skeptical as well.
- georgespencer 10y ago
  I completely understand what you mean and agree it's a valid concern. I'm probably super naïve, but I think there's probably a way of digitally signing hardware components so that we can trust sensor data from other cars.