4 ms·
It's the loss of agency to machines, not the raw numbers that scares people. Why do you think so many people are unduly spooked after airline accidents due to m
by lr4444lr 6y ago
It's the loss of agency to machines, not the raw numbers that scares people. Why do you think so many people are unduly spooked after airline accidents due to mechanical or computer error?
- waterhouse 6y agoI would say that, when I think about it occasionally, I am kind of scared by the possibility that someone will hack into all the self-driving machines (from a particular vendor) and make them all do something terrible at once—before anyone has time to deploy countermeasures.
- contingencies 6y agoIt went as an unspoken, unquestionable assumption that telephony was the right model for data networking. - Van Jacobson Maybe a single 'driver' agent in a free-moving ground vehicle in close proximity to vulnerable actors and objects is not the right model for scalable, efficient, safe personal transport?
- colechristensen 6y agoIt’s the fundamentally different kind of danger from machines. They make mistakes we can’t understand. I can understand a person being distracted, reacting slowly, not seeing me, etc. I can also mitigate this risk. Crossing the street I can make eye contact with a person and be really confident that they aren’t going to run me over. Walking in front of a robot? I have no internal model for the kind of mistakes they make. It seems equally likely in any circumstance that they’ll make the mistake of running me down, and there is nothing i can do to avoid it.
- jfoster 6y agoYou do develop an internal model for the behaviour of machines, though. Examples: Elevators where the doors take extremely long to close on their own. Elevators where the doors close too soon after opening. Elevators where the limb sensors are not very responsive. Cars that take a couple of tries to start in cold weather. Automatic doors that are a little bit slow to open. Etc.
- gotostatement 6y agobut those dont happen at 60+ miles per hour
- jfoster 6y agoOkay, let's use another example: lifts don't move until the doors are closed. If the lift thinks the doors are open, it won't move. If it did, it would be an out of control situation just as quickly as a car misbehaving at 60 mph is. The expectation is that autonomous cars must be cautious.
- gotostatement 6y agomy point is when we develop expectations and mental models of how machines work, we can use those to adapt and feel safe and in control. but that's less possible when the machine is typically moving at 60 mph and the mistake can happen without warning
- jfoster 6y agoLike elevators where the brakes fail. If it's infrequent enough, we tend not to think about it even if it does still happen from time to time.
- gotostatement 6y agoI see your point but for me at least, my gut feeling is the autonomous cars are just scarier because they're moving at 60 miles per hour. Maybe it's irrational but this whole discussion isn't about rationality its about gut feeling
- colechristensen 6y agoPowered doors are one dimensional machines with perhaps one binary sensor and a motor driver, a person could implement a state of the set automatic door in an afternoon. You would also have to be rather clever in order to get an automatic door to injure you. A car is also a rather easily understood dynamic system, but that isn’t an issue either. We are all quite frequently in the position of being at between a few seconds and one heartbeat away from a car killing us... walking down a sidewalk, crossing the street, driving on the highway... We understand pretty well the dynamics of a human driver. Even the drivers that make terrible mistakes we understand something of hope and why. A robot driver though... the terrible mistakes it makes are mysterious and terrifying. They make the kinds of mistakes a human never would (and few mistakes a human would make).
- shermanmccoy 6y agoYes! You can control for human risks in the tail scenarios. You can't control for the tail scenarios in AI because you don't know what on earth they are, they are a nasty surprise, thrown up by a model which, in the tail, we as engineers don't fully understand
- letitbeirie 6y agoWhich is ironic, because this is the kind of automation they've been building into planes for years and consequently air travel is safer than it's ever been. 737 Max fiasco notwithstanding - it's worth mentioning that MCAS had nothing to do with flying the plane, only making it feel more like the 737NG to its human pilots
- sofixa 6y ago> it's worth mentioning that MCAS had nothing to do with flying the plane, only making it feel more like the 737NG to its human pilots Not entirely true, the original issue is that the engines are bigger and the center of gravity is moved forward ( because the 737's are too low), thus the planes tend to stall. MCAS is there to correct the stall so that they feel more like 737NGs, but the issue is forcing bigger engines on an obsolete design.
- mrguyorama 6y agoHow much machine learning and black box decision making does autopilot in a plane do? My understanding is none, or nearly none. VNAV follows GPS waypoints with well described, manually designed behaviors for things like climb rate, bank angle, etc. Autoland isn't machine learning either, and has such a large list of requirements to be allowed that it's basically like designing a self driving computer for a train.