4 ms·
There was a great paper[1] I read about the human components in complex technical systems which argues that one of the roles of the human is to take the blame w
by aeturnum 3y ago
There was a great paper[1] I read about the human components in complex technical systems which argues that one of the roles of the human is to take the blame when the entire system fails. This does real valuable work for the companies involved and helps them avoid needing to answer the most uncomfortable questions.
[1] Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction by Madeline Clare Elish
- jacquesm 3y agoThis goes for self driving cars just the same.
- aeturnum 3y agoAn example used in the paper are the human drivers of Uber self-driving cars - one of which killed a pedestrian in Arizona.
- TaylorAlexander 3y agoYeah it leads to real problems for fully driverless cars, as there is no longer a human to blame. Also note that blaming the human has been Tesla's strategy for avoiding responsibility on their software drive system failures.
- dev1n 3y agoMost car - pedestrian crashes in the USA never end up with the driver at fault. Not sure this will change when cars are autonomous.
- TaylorAlexander 3y agoWell I am talking about all crashes, not just car-pedestrian crashes. For example Teslas crash in to other cars and then the company blames the human driver.
- jxf 3y ago"Moral crumple zones" is an incredible description.
- _ea1k 3y agoThe flip side to this is that every system involving software seems to inevitably devolve into a situation where the human is expected to no longer be responsible. Oh, you floored it while the car was pointed at the wall? It has cameras, why didn't the car disable the go pedal? This is happening more and more with cars, and it seems inevitable that it will happen in other spaces as well, as software is expected to protect us from ourselves.
- underwater 3y agoThere is a big difference between software being the safety net for humans, and humans being the safety net for software.
- _ea1k 3y agoIn the extreme, yes. But it looks like it is actually a continuum, with a shockingly blurry line forming the threshold.