9 ms·

It strikes me that a useful analogy here is the adoption of automatic elevators in buildings. In some ways, it's amazing that pretty much everyone in industrial

It strikes me that a useful analogy here is the adoption of automatic elevators in buildings. In some ways, it's amazing that pretty much everyone in industrialized countries is OK with being locked in a windowless box controlled entirely by a computer, hanging over a hundreds of foot deep shaft, and in fact many people were terrified of that when Elevator operators were first replaced with computers. Some places even had operators employed to simply stand there and push the buttons to provide confidence that an trained expert was there, even though they didn't actually contribute to safety. Eventually, autonomous elevators got common enough that people will look at you really funny if you're not willing to ride in one, even though they are still responsible for ~20-30 deaths per year.

dekervin 6y ago
there is some kind of implied machine capability that a layman assign to a computer. If a task is culturally ( through movies , series, ... ) though to be within that implied capability people will be comfortable. ( cf automatic trains, ... )
ggreer 6y ago
Can you provide more info about people being terrified of automatic elevators? I searched around and everything I found seems to cite one NPR article from 2015.[1] The interviewee's book is out of print and costs $100[2], so that's where that trail stops. If public sentiment against automatic elevators was as strong as described, it seems like there would be more historical evidence available. It's easier for me to find articles disparaging self-checkout systems than for automated elevators. I realize the change in elevators happened long ago, making articles harder to find, but you'd think at least one of them would have gotten digitized and indexed. 1. https://www.npr.org/2015/07/31/427990392/remembering-when-driverless-elevators-drew-skepticism https://www.npr.org/2015/07/31/427990392/remembering-when-dr... 2. https://www.amazon.com/Ascending-Rooms-Express-Elevators-Passenger/dp/1937689107 https://www.amazon.com/Ascending-Rooms-Express-Elevators-Pas...
- HALtheWise 6y ago
  I just spent some time digging, but finding original sources from the ~1950s is really hard without a NYTimes subscription. These sorts of things look like promising primary sources, but I can't access the full text. https://www.nytimes.com/1949/01/12/archives/city-gets-elevator-bill-isaacs-would-require-operator-in-buildings.html https://www.nytimes.com/1949/01/12/archives/city-gets-elevat... https://www.nytimes.com/1949/02/03/archives/tenants-want-a-doorman-suit-against-landlord-demands-elevators-with.html https://www.nytimes.com/1949/02/03/archives/tenants-want-a-d... https://www.nytimes.com/1928/11/11/archives/elevator-law-change-oppose-employment-of-operators-on-automatic.html https://www.nytimes.com/1928/11/11/archives/elevator-law-cha...
  - SomewhatLikely 6y ago
    I have a subscription. The first two seem to be more about tenants feeling that an operator raises the quality of life. The third one talks of several deaths of children. Apparently in these cases there was a gap of 8 to ten inches between the car doors and the landing doors which accidentally trapped the children and then the car was requested from someone on another level. An ordinance had been proposed to require operators, but opposition claims that rules requiring doors to be flush will prevent such cases. So this would likely be in the category of cases that would probably never happen with a manual operator. Two solutions were proposed: keep manual operators, or improve the safety of the automatic system.
    - tialaramex 6y ago
      > Two solutions were proposed: keep manual operators, or improve the safety of the automatic system. After Southall [1] the drivers union felt the correct approach was to double man trains. It is surely not a coincidence this would also have meant more members and thus more power and income for the union. That accident would have been prevented by ATP (fitted to the train but disabled because the driver had not been trained) or AWS (fitted to all British trains but disabled by the driver due to a fault). The ATP system would have braked to a halt to avoid entering an occupied track section. AWS would have audibly alerted the driver to the signals (showing a Preliminary Caution, an ordinary Caution and then Danger) and if the driver did not react, braked to a halt. With neither it appears the driver simply did not see the signals because they weren't looking and so did not react until far too late. The correct fix of course is to require trains to have working AWS/ ATP and use it. Operating trains with more humans is a worthwhile work around to get a faulty train back to a repair yard as empty stock, but makes no sense in passenger service. [1] https://en.wikipedia.org/wiki/Southall_rail_crash https://en.wikipedia.org/wiki/Southall_rail_crash
RcouF1uZ4gsC 6y ago
The difference with elevators is that the safety systems are actually in large parts independent of the control. Since the Otis safety elevator, you could go so far as to cut the elevator cord and it would still be ok. With self-driving cars, you don't have those type of backup safety systems.
- a_t48 6y ago
  You don't have those sorts of systems for regular human drivers, either. If a human driver passes out (diabetic episode, heart attack, drunkeness), you are just as SOL. With a self driving system there's various levels of redundancy you can put in, up to some sort of dead man's switch on an embedded controller that slams the brakes (or maybe just a gentle stop in combination with existing ADAS) if all of the "higher level" systems stop responding.
  - hypertele-Xii 6y ago
    The problem is, what if the higher level systems are responding, but are confused by the data they see? We'd have to have redundant decision-making like in the space industry - two of each computer and sensor that have to agree on the result, otherwise safety systems stop the vehicle.
    - ntsplnkv2 6y ago
      We're going to end up at the trolley problem again. "Should I do action X and kill 3 or do action Y and maybe kill 2?"
      - Dylan16807 6y ago
        Don't waste too much time on that. These are cars. They can just brake, it's good enough.
        ntsplnkv2 6y ago
        Let me know when we reach the point that autonomous cars can bend the laws of physics too, that would truly be a hoot!
Guthur 6y ago
The analogy with elevators largely doesn't work. In the main the only individuals at risk from an elevator accident are those that chose to enter the elevator, this is vastly different for automotive accidents where the individuals affecting by the autonomous car are not those that chose to use one.
- aeternum 6y ago
  If anything, that should be more of an argument for autonomous cars being allowed once they reach only 1x of the safety of human-driven-vehicles. Almost by definition, more pedestrians will die if we wait for the 4-5x safety threshold proposed in the article.
  - studius 6y ago
    The 4-5x is based on statistics of a sample with a high ratio of human drivers to autonomous drivers, if done on the road today. Autonomous won't be known to be N times as safe as human driving until there are sufficient autonomous cars on the road and the varying ratios of autonomous to human drivers have been at equilibrium for a some sufficient amount of time to gather sufficient data to compare to human driving safety statistics. So- 4-5x times is a lie. It could be much more dangerous with only autonomous drivers or much less dangerous. They don't know yet.
    - Dylan16807 6y ago
      > So- 4-5x times is a lie. It could be much more dangerous with only autonomous drivers or much less dangerous. They don't know yet. "could be"? Statistics? Aren't these studies asking about theoretical cars that are by definition a certain amount safer?
      - studius 6y ago
        e.g. "This autonomous car is 4-5x safer than your human-driven car." When people hear numbers like that, they may assume it's based on sufficient data and is true, even if it's theory. But how could it be true or acceptable, given that we're only just starting to have more autonomous vehicles on the road, and the safety could be dependent on the ratio of autonomous vehicles to human drivers?
ntsplnkv2 6y ago
I don't think a self driving car that has to make rapid decisions in quickly changing circumstances with tons of moving parts outside of its control is comparable to a specific use case with much fewer variables like an elevator.
- RhodesianHunter 6y ago
  Why not? The relative complexity of an automatic elevator at the time considering their level of tech was probably comparable to the complexity of an autonomous vehicle today.
  - kaba0 6y ago
    Lol. At the level of tech? We are barely better than a lego mindstorm that goes forward until something is not closer than x centimeters.. (with a bit of sarcasm) An autonomous car can’t really exit a parking building (I’m not really sure what are they called, but those multi-level parking thingies) because it would require them to understand signs and the like and those may often be missing.
    - RhodesianHunter 6y ago
      I rode around Vegas in an automated car at reinvent last year. It understood signs just fine. they have thousands and thousands of miles of autonomous driving across California and Nevada.
kaba0 6y ago
I know it’s an analogy, but elevators are dead-simple concepts with many fail-safety mechanisms independent of the controlling ‘program’. Self-driving cars are ridiculously complicated black-box program with at most software fail-safes that are itself more complicated than one can reason about so the only thing remaining is empirical testing in real-world situations. And every single bugfix have to be retested substantially as it can just as easily make something else worse - and I don’t feel confident about today’s “self-driving” cars

dekervin 6y ago

there is some kind of implied machine capability that a layman assign to a computer. If a task is culturally ( through movies , series, ... ) though to be within that implied capability people will be comfortable. ( cf automatic trains, ... )

ggreer 6y ago

Can you provide more info about people being terrified of automatic elevators? I searched around and everything I found seems to cite one NPR article from 2015.[1] The interviewee's book is out of print and costs $100[2], so that's where that trail stops. If public sentiment against automatic elevators was as strong as described, it seems like there would be more historical evidence available. It's easier for me to find articles disparaging self-checkout systems than for automated elevators. I realize the change in elevators happened long ago, making articles harder to find, but you'd think at least one of them would have gotten digitized and indexed. 1. https://www.npr.org/2015/07/31/427990392/remembering-when-driverless-elevators-drew-skepticism https://www.npr.org/2015/07/31/427990392/remembering-when-dr... 2. https://www.amazon.com/Ascending-Rooms-Express-Elevators-Passenger/dp/1937689107 https://www.amazon.com/Ascending-Rooms-Express-Elevators-Pas...

HALtheWise 6y ago
I just spent some time digging, but finding original sources from the ~1950s is really hard without a NYTimes subscription. These sorts of things look like promising primary sources, but I can't access the full text. https://www.nytimes.com/1949/01/12/archives/city-gets-elevator-bill-isaacs-would-require-operator-in-buildings.html https://www.nytimes.com/1949/01/12/archives/city-gets-elevat... https://www.nytimes.com/1949/02/03/archives/tenants-want-a-doorman-suit-against-landlord-demands-elevators-with.html https://www.nytimes.com/1949/02/03/archives/tenants-want-a-d... https://www.nytimes.com/1928/11/11/archives/elevator-law-change-oppose-employment-of-operators-on-automatic.html https://www.nytimes.com/1928/11/11/archives/elevator-law-cha...
- SomewhatLikely 6y ago
  I have a subscription. The first two seem to be more about tenants feeling that an operator raises the quality of life. The third one talks of several deaths of children. Apparently in these cases there was a gap of 8 to ten inches between the car doors and the landing doors which accidentally trapped the children and then the car was requested from someone on another level. An ordinance had been proposed to require operators, but opposition claims that rules requiring doors to be flush will prevent such cases. So this would likely be in the category of cases that would probably never happen with a manual operator. Two solutions were proposed: keep manual operators, or improve the safety of the automatic system.
  - tialaramex 6y ago
    > Two solutions were proposed: keep manual operators, or improve the safety of the automatic system. After Southall [1] the drivers union felt the correct approach was to double man trains. It is surely not a coincidence this would also have meant more members and thus more power and income for the union. That accident would have been prevented by ATP (fitted to the train but disabled because the driver had not been trained) or AWS (fitted to all British trains but disabled by the driver due to a fault). The ATP system would have braked to a halt to avoid entering an occupied track section. AWS would have audibly alerted the driver to the signals (showing a Preliminary Caution, an ordinary Caution and then Danger) and if the driver did not react, braked to a halt. With neither it appears the driver simply did not see the signals because they weren't looking and so did not react until far too late. The correct fix of course is to require trains to have working AWS/ ATP and use it. Operating trains with more humans is a worthwhile work around to get a faulty train back to a repair yard as empty stock, but makes no sense in passenger service. [1] https://en.wikipedia.org/wiki/Southall_rail_crash https://en.wikipedia.org/wiki/Southall_rail_crash

RcouF1uZ4gsC 6y ago

The difference with elevators is that the safety systems are actually in large parts independent of the control. Since the Otis safety elevator, you could go so far as to cut the elevator cord and it would still be ok. With self-driving cars, you don't have those type of backup safety systems.

a_t48 6y ago
You don't have those sorts of systems for regular human drivers, either. If a human driver passes out (diabetic episode, heart attack, drunkeness), you are just as SOL. With a self driving system there's various levels of redundancy you can put in, up to some sort of dead man's switch on an embedded controller that slams the brakes (or maybe just a gentle stop in combination with existing ADAS) if all of the "higher level" systems stop responding.
- hypertele-Xii 6y ago
  The problem is, what if the higher level systems are responding, but are confused by the data they see? We'd have to have redundant decision-making like in the space industry - two of each computer and sensor that have to agree on the result, otherwise safety systems stop the vehicle.
  - ntsplnkv2 6y ago
    We're going to end up at the trolley problem again. "Should I do action X and kill 3 or do action Y and maybe kill 2?"
    - Dylan16807 6y ago
      Don't waste too much time on that. These are cars. They can just brake, it's good enough.
      - ntsplnkv2 6y ago
        Let me know when we reach the point that autonomous cars can bend the laws of physics too, that would truly be a hoot!

Guthur 6y ago

The analogy with elevators largely doesn't work. In the main the only individuals at risk from an elevator accident are those that chose to enter the elevator, this is vastly different for automotive accidents where the individuals affecting by the autonomous car are not those that chose to use one.

aeternum 6y ago
If anything, that should be more of an argument for autonomous cars being allowed once they reach only 1x of the safety of human-driven-vehicles. Almost by definition, more pedestrians will die if we wait for the 4-5x safety threshold proposed in the article.
- studius 6y ago
  The 4-5x is based on statistics of a sample with a high ratio of human drivers to autonomous drivers, if done on the road today. Autonomous won't be known to be N times as safe as human driving until there are sufficient autonomous cars on the road and the varying ratios of autonomous to human drivers have been at equilibrium for a some sufficient amount of time to gather sufficient data to compare to human driving safety statistics. So- 4-5x times is a lie. It could be much more dangerous with only autonomous drivers or much less dangerous. They don't know yet.
  - Dylan16807 6y ago
    > So- 4-5x times is a lie. It could be much more dangerous with only autonomous drivers or much less dangerous. They don't know yet. "could be"? Statistics? Aren't these studies asking about theoretical cars that are by definition a certain amount safer?
    - studius 6y ago
      e.g. "This autonomous car is 4-5x safer than your human-driven car." When people hear numbers like that, they may assume it's based on sufficient data and is true, even if it's theory. But how could it be true or acceptable, given that we're only just starting to have more autonomous vehicles on the road, and the safety could be dependent on the ratio of autonomous vehicles to human drivers?

ntsplnkv2 6y ago

I don't think a self driving car that has to make rapid decisions in quickly changing circumstances with tons of moving parts outside of its control is comparable to a specific use case with much fewer variables like an elevator.

RhodesianHunter 6y ago
Why not? The relative complexity of an automatic elevator at the time considering their level of tech was probably comparable to the complexity of an autonomous vehicle today.
- kaba0 6y ago
  Lol. At the level of tech? We are barely better than a lego mindstorm that goes forward until something is not closer than x centimeters.. (with a bit of sarcasm) An autonomous car can’t really exit a parking building (I’m not really sure what are they called, but those multi-level parking thingies) because it would require them to understand signs and the like and those may often be missing.
  - RhodesianHunter 6y ago
    I rode around Vegas in an automated car at reinvent last year. It understood signs just fine. they have thousands and thousands of miles of autonomous driving across California and Nevada.

kaba0 6y ago

I know it’s an analogy, but elevators are dead-simple concepts with many fail-safety mechanisms independent of the controlling ‘program’. Self-driving cars are ridiculously complicated black-box program with at most software fail-safes that are itself more complicated than one can reason about so the only thing remaining is empirical testing in real-world situations. And every single bugfix have to be retested substantially as it can just as easily make something else worse - and I don’t feel confident about today’s “self-driving” cars