8 ms·

Also known as the "Level 3 is inherently unsafe" argument. Waymo noticed this back in 2012: https://driverless.wonderhowto.com/news/waymo-was-right-why-every-ca

by snaily 9y ago

Also known as the "Level 3 is inherently unsafe" argument. Waymo noticed this back in 2012: https://driverless.wonderhowto.com/news/waymo-was-right-why-every-car-maker-should-skip-level-3-0178497/ https://driverless.wonderhowto.com/news/waymo-was-right-why-...

digi_owl 9y ago
In case others are wondering about the details about the levels: https://en.wikipedia.org/wiki/Autonomous_car#Classification https://en.wikipedia.org/wiki/Autonomous_car#Classification
stale2002 9y ago
Well, so level 3 can be safe under certain conditions and still be world changing. For example, imagine a self driving truck that works perfectly, but only during the day, on highways, when it is not raining. Such a system would still put millions out of work, as firing half of the truck drivers is possible if something works when it is sunny, on the highway.
- rco8786 9y ago
  I’ve been pretty convinced for a while that this is effectively how it will play out. Effectively the “iRobot” model of designated areas/highways that are autonomous and highly controlled/protected and surface streets will remain human controlled for a long time.
- tjoff 9y ago
  At least level 4 is required for that.
- twblalock 9y ago
  Truck drivers do a lot more than just drive trucks: http://marginalrevolution.com/marginalrevolution/2018/02/will-truckers-automated-comments.html http://marginalrevolution.com/marginalrevolution/2018/02/wil...
  - tjoff 9y ago
    That wasn't convincing. Obviously someone else (or a machine) would have to load/unload. Sure there will be routes that doesn't automate well, where the destination don't have the knowledge or scale for taking care of loading etc. but for the most parts it would be trivial compared to getting the truck to drive.
    - jjeaff 9y ago
      The ice road truckers will still have a job!
    - Twisell 9y ago
      This kind of reaction is totally baffling for me. I’ve done quite a bit of inventory manual work during college/university Holliday’s and even with such a limited experience I can’t even begin to explain how wrong you are because it would go on for hours. In a neighbor country wich is pretty much considered as a leader in industrial automation (Germany) it’s almost mandatory for engineers to experience months of low level jobs to get a grasp of what practical real world problems are when working. This is pure Silicon Valley overconfidence. Software "move fast and break things" approach really don’t translate well for hardware. And in this case I think Uber will learn this lesson the hard way.
      - tjoff 9y ago
        Could you at least try to make your point? All I said was that the act of loading/unloading etc. would be done by different people than the truck driver. Also, we already have semi-automated warehouses, it is not unimaginable that the drones driving around with boxes could be adapted to able to help load a truck. Probably with the help of another machine. Not saying that this process needs to be void of any human at all - just that the truck driver wouldn't be part of it. I really didn't say that the inventory work would just disappear - it would just be moved from the truck driver.
        Twisell 9y ago
        Well first research field of automated warehouse is hudge and so it’s very reductive to think that what happens after the truck is the easiest part. Secondly truck driver usually are in charge of the final delivery to shop or customer and have to deliver good to buildings that are not by any mean standardized. Having a human on board that know how to operate the truck and how to properly unload it is a feature that will probably be missed out. As It would be pretty unwise IMHO to rely on external people to "not fuck up" when handling heavy objects around a costly automated truck. So maybe I am totally wrong, but if we concentrate our current R&D on building expensive automated truck so that we can reduce cost by firing driver... Instead of focusing our R&D on producing more economic and ecological truck and alternative transportation... we might as well be fucked up as a specie.
- femto 9y ago
  In the physical world there's no such thing as a system with preconditions that can "work perfectly". There will always be a boundary region where is is hard to say whether the precondition is present and the system is unpredictable. What we really mean when we say a system is reliable/perfect is that the conditions which can cause failure are so rare that they can be neglected. Any system that depends on things that are inside normal experience: rain, day/night, errant pedestrians, earthquakes, and so on can never be trusted to act reliably/perfectly. A "reliable" system might be excused for messing up under conditions such as meteor strikes or falling space stations.
  - stale2002 9y ago
    The pedanty here is insane. Imagine if instead of saying the word "perfectly" I said "works better than humans". Do you reject the idea that there are situations, such as sunny highways, that are much much easier than everything else? And that if we can solve this much easier problem, it would still be world changing? Why couldnt that be trusted to work better than humans? It is a much easier problem.
    - femto 9y ago
      > The pedanty here is insane. One person's pedant is another person's clear conscience. There's massive disagreement in the self-driving debate as to the extent to which we are dealing with safety critical systems. As you can no doubt tell, I'm firmly on the safety critical side. I've a fair bit of experience with designing and shipping critical infrastructure for emergency services. One has to ship, but in my experience the majority of people don't have the necessary eye for detail to make a safe system. This is why we have process: to allow fallible people to ship safe systems (and to avoid delusion in the minority who do have an eye for detail). The self-driving field is still developing their processes, so they have to be conservative with their shipping. Many of those arguing for self-driving are under estimating the safety of a typical human driver. The bar that a self-driving system has to meet is very high. If conditions are easy for a self-driving car then they are probably easy and safe for a human as well. Accidents are skewed towards a subset of poor conditions involving speed, alcohol and fatigue. Is it fair to replace a safe driver that avoids speed, alcohol and fatigue with a system that whilst safer than average is still less safe than the specific circumstance? Maybe an interim use of self-driving technology is to have the person driving, and the self-driving system in a co-driver/critique mode acting as an "unsafe driving detector"?
- xiphias 9y ago
  What you're describing is Level 4: full self driving in limited conditions. The Waymo fleet is a great example of that.
- gaius 9y ago
  For example, imagine a self driving truck that works perfectly, but only during the day, on highways, when it is not raining. LOL. The other day I was in Mexico on a cave diving trip. It rained, short sudden downpour, out of nowhere. The carnage on the highway just in the few miles from the dive shop to the cenote was insane. No fair-weather robot would stand a chance in those conditions.
sah2ed 9y ago
Granted, but this wasn't some grand discovery by Waymo. The human side of automation [0] is an issue that has been known and written about since at least 1983 by Lisanne Bainbridge and Don Norman in 1990. [0] https://www.jnd.org/dn.mss/the_human_side_of_au.html https://www.jnd.org/dn.mss/the_human_side_of_au.html

by snaily 9y ago

digi_owl 9y ago

In case others are wondering about the details about the levels: https://en.wikipedia.org/wiki/Autonomous_car#Classification https://en.wikipedia.org/wiki/Autonomous_car#Classification

stale2002 9y ago

Well, so level 3 can be safe under certain conditions and still be world changing. For example, imagine a self driving truck that works perfectly, but only during the day, on highways, when it is not raining. Such a system would still put millions out of work, as firing half of the truck drivers is possible if something works when it is sunny, on the highway.

rco8786 9y ago
I’ve been pretty convinced for a while that this is effectively how it will play out. Effectively the “iRobot” model of designated areas/highways that are autonomous and highly controlled/protected and surface streets will remain human controlled for a long time.
tjoff 9y ago
At least level 4 is required for that.
twblalock 9y ago
Truck drivers do a lot more than just drive trucks: http://marginalrevolution.com/marginalrevolution/2018/02/will-truckers-automated-comments.html http://marginalrevolution.com/marginalrevolution/2018/02/wil...
- tjoff 9y ago
  That wasn't convincing. Obviously someone else (or a machine) would have to load/unload. Sure there will be routes that doesn't automate well, where the destination don't have the knowledge or scale for taking care of loading etc. but for the most parts it would be trivial compared to getting the truck to drive.
  - jjeaff 9y ago
    The ice road truckers will still have a job!
  - Twisell 9y ago
    This kind of reaction is totally baffling for me. I’ve done quite a bit of inventory manual work during college/university Holliday’s and even with such a limited experience I can’t even begin to explain how wrong you are because it would go on for hours. In a neighbor country wich is pretty much considered as a leader in industrial automation (Germany) it’s almost mandatory for engineers to experience months of low level jobs to get a grasp of what practical real world problems are when working. This is pure Silicon Valley overconfidence. Software "move fast and break things" approach really don’t translate well for hardware. And in this case I think Uber will learn this lesson the hard way.
    - tjoff 9y ago
      Could you at least try to make your point? All I said was that the act of loading/unloading etc. would be done by different people than the truck driver. Also, we already have semi-automated warehouses, it is not unimaginable that the drones driving around with boxes could be adapted to able to help load a truck. Probably with the help of another machine. Not saying that this process needs to be void of any human at all - just that the truck driver wouldn't be part of it. I really didn't say that the inventory work would just disappear - it would just be moved from the truck driver.
      - Twisell 9y ago
        Well first research field of automated warehouse is hudge and so it’s very reductive to think that what happens after the truck is the easiest part. Secondly truck driver usually are in charge of the final delivery to shop or customer and have to deliver good to buildings that are not by any mean standardized. Having a human on board that know how to operate the truck and how to properly unload it is a feature that will probably be missed out. As It would be pretty unwise IMHO to rely on external people to "not fuck up" when handling heavy objects around a costly automated truck. So maybe I am totally wrong, but if we concentrate our current R&D on building expensive automated truck so that we can reduce cost by firing driver... Instead of focusing our R&D on producing more economic and ecological truck and alternative transportation... we might as well be fucked up as a specie.
femto 9y ago
In the physical world there's no such thing as a system with preconditions that can "work perfectly". There will always be a boundary region where is is hard to say whether the precondition is present and the system is unpredictable. What we really mean when we say a system is reliable/perfect is that the conditions which can cause failure are so rare that they can be neglected. Any system that depends on things that are inside normal experience: rain, day/night, errant pedestrians, earthquakes, and so on can never be trusted to act reliably/perfectly. A "reliable" system might be excused for messing up under conditions such as meteor strikes or falling space stations.
- stale2002 9y ago
  The pedanty here is insane. Imagine if instead of saying the word "perfectly" I said "works better than humans". Do you reject the idea that there are situations, such as sunny highways, that are much much easier than everything else? And that if we can solve this much easier problem, it would still be world changing? Why couldnt that be trusted to work better than humans? It is a much easier problem.
  - femto 9y ago
    > The pedanty here is insane. One person's pedant is another person's clear conscience. There's massive disagreement in the self-driving debate as to the extent to which we are dealing with safety critical systems. As you can no doubt tell, I'm firmly on the safety critical side. I've a fair bit of experience with designing and shipping critical infrastructure for emergency services. One has to ship, but in my experience the majority of people don't have the necessary eye for detail to make a safe system. This is why we have process: to allow fallible people to ship safe systems (and to avoid delusion in the minority who do have an eye for detail). The self-driving field is still developing their processes, so they have to be conservative with their shipping. Many of those arguing for self-driving are under estimating the safety of a typical human driver. The bar that a self-driving system has to meet is very high. If conditions are easy for a self-driving car then they are probably easy and safe for a human as well. Accidents are skewed towards a subset of poor conditions involving speed, alcohol and fatigue. Is it fair to replace a safe driver that avoids speed, alcohol and fatigue with a system that whilst safer than average is still less safe than the specific circumstance? Maybe an interim use of self-driving technology is to have the person driving, and the self-driving system in a co-driver/critique mode acting as an "unsafe driving detector"?
xiphias 9y ago
What you're describing is Level 4: full self driving in limited conditions. The Waymo fleet is a great example of that.
gaius 9y ago
For example, imagine a self driving truck that works perfectly, but only during the day, on highways, when it is not raining. LOL. The other day I was in Mexico on a cave diving trip. It rained, short sudden downpour, out of nowhere. The carnage on the highway just in the few miles from the dive shop to the cenote was insane. No fair-weather robot would stand a chance in those conditions.

sah2ed 9y ago

Granted, but this wasn't some grand discovery by Waymo. The human side of automation [0] is an issue that has been known and written about since at least 1983 by Lisanne Bainbridge and Don Norman in 1990. [0] https://www.jnd.org/dn.mss/the_human_side_of_au.html https://www.jnd.org/dn.mss/the_human_side_of_au.html