4 ms·

In the physical world there's no such thing as a system with preconditions that can "work perfectly". There will always be a boundary region where is is hard t

by femto 9y ago

In the physical world there's no such thing as a system with preconditions that can "work perfectly". There will always be a boundary region where is is hard to say whether the precondition is present and the system is unpredictable. What we really mean when we say a system is reliable/perfect is that the conditions which can cause failure are so rare that they can be neglected. Any system that depends on things that are inside normal experience: rain, day/night, errant pedestrians, earthquakes, and so on can never be trusted to act reliably/perfectly. A "reliable" system might be excused for messing up under conditions such as meteor strikes or falling space stations.

stale2002 9y ago
The pedanty here is insane. Imagine if instead of saying the word "perfectly" I said "works better than humans". Do you reject the idea that there are situations, such as sunny highways, that are much much easier than everything else? And that if we can solve this much easier problem, it would still be world changing? Why couldnt that be trusted to work better than humans? It is a much easier problem.
- femto 9y ago
  > The pedanty here is insane. One person's pedant is another person's clear conscience. There's massive disagreement in the self-driving debate as to the extent to which we are dealing with safety critical systems. As you can no doubt tell, I'm firmly on the safety critical side. I've a fair bit of experience with designing and shipping critical infrastructure for emergency services. One has to ship, but in my experience the majority of people don't have the necessary eye for detail to make a safe system. This is why we have process: to allow fallible people to ship safe systems (and to avoid delusion in the minority who do have an eye for detail). The self-driving field is still developing their processes, so they have to be conservative with their shipping. Many of those arguing for self-driving are under estimating the safety of a typical human driver. The bar that a self-driving system has to meet is very high. If conditions are easy for a self-driving car then they are probably easy and safe for a human as well. Accidents are skewed towards a subset of poor conditions involving speed, alcohol and fatigue. Is it fair to replace a safe driver that avoids speed, alcohol and fatigue with a system that whilst safer than average is still less safe than the specific circumstance? Maybe an interim use of self-driving technology is to have the person driving, and the self-driving system in a co-driver/critique mode acting as an "unsafe driving detector"?
  - LoSboccacc 9y ago
    ome can just look at the state of autonomous robots in industrial plants. they do a handful of functions, they have limited reach and they have predictable patterns and yet when you add humans to the mix all kind of dangerous conditions arise to the point most robots sharing space with humans need to be caged this is our current state of the art with automation and safety and yet we allow companies to slap a radar and a pretend safety driver and hurl a ton of metal down the pubblic roads autonomous vehicle will undoubtedly at some point be safer than human, but shortcutting known limitations like radars vs dodging static objects because 'drivers are the backup' is lunacy and the recent deaths are just a symptom of people never having worked in safety system design playing fast and loose with human lifes, and the companies which enable that should be made an example of by courts. some years ago serious automaker did their analisys and come up with the concept of designing the anticollision system first and use those as a backup to autonomous driving technology, that somehow has been lost in the race for profitability.
    - CaptainZapp 9y ago
      "this is our current state of the art with automation and safety and yet we allow companies to slap a radar and a pretend safety driver and hurl a ton of metal down the pubblic roads" This is what bugs me most: Who on earth ever thought it's a good idea to allow a company like Uber with their history of corner cutting and shady business to test such dangerous technology on public roads? The mind boggles.
- nubbins 9y ago
  I’m confused about this hypothetical- when ot starts raining or in any other adverse condition thousands of truck loads are just parked and delayed until it improves? I don’t see how this would put anyone out of work.
  - stale2002 9y ago
    Imagine this on the highway in the desert. Is it ever going to start raining then? Or imagine if you just look up the weather ahead of time and only run the self driving trucks if there is a less than 5% chance of rain. These situations are still very common. You could replace all trucks on the highway in any desert climate, for example. And that's still many billions of dollars.
- boulos 9y ago
  > The pedanty here is insane. You mean pedantry :). -- Admiral Pedant.
- darpa_escapee 9y ago
  What's an acceptable time period for autonomous cars to fail catastrophically on the road until they are better than humans at driving safely?
- shanghaiaway 9y ago
  What's the point of a technology that only works under the least challenging conditions, that is probably less than one percent of the actual occurring conditions - and where the current solution, human beings, work better anyway because they don't have to stop for rain? What you're imagining is a car on a fixed track - a railcar. We already have trains but the US does need to build out their railway, yeah.
  - stale2002 9y ago
    Because it is not 1% of circumstances. It would be more like 50% of circumstances. There are lots of places where it is sunny, and not raining and it is daytime. Just don't run the trucks on self driving mode during the other times where they don't work, and we could still save up to 50% in labor costs. Obviously, though, the market for self driving taxis is different than for self driving trucks, but once again, we could just only use them for trucks and still save Trillions.
nerfhammer 9y ago
There is going to be a higher threshold of self-driving vehicles, because random accidents by random people is not narrative-inspiring but for self-driving vehicles all accidents will be the fault of The Corporation. Every single individual accident will be the focus of world media fascination and speculation to derive how the motives of the corporation's singular supermind chose to allow it to happen, which will turn out to be that it was too greedy, scheming, uncaring, lazy, etc. There will be intense focus on a trickle of rare accidents because they are caused by a powerful corporate overmind that is potentially betraying us all, and little interest in a greater number of random accidents caused by no one of particular recognition for no discernable reason. Which is exactly what we've seen so far.
- JumpCrisscross 9y ago
  > random accidents by random people is not narrative-inspiring but for self-driving vehicles all accidents will be the fault of The Corporation I think we can conclude, absent conditions of wilful neglect, that this hypothesis is false. I thought this, too. But people are reacting reasonably to Tesla and Uber’s, erm, faux pas.
  - jon_richards 9y ago
    I'm inclined to say "too reasonably" given the fact that the statistics seem to imply a much higher risk from the self-driving software than a human driver (Uber doesn't have many miles driven and Tesla is using overall-telsa statistics to hide the auto-pilot statistics).

by femto 9y ago

stale2002 9y ago

The pedanty here is insane. Imagine if instead of saying the word "perfectly" I said "works better than humans". Do you reject the idea that there are situations, such as sunny highways, that are much much easier than everything else? And that if we can solve this much easier problem, it would still be world changing? Why couldnt that be trusted to work better than humans? It is a much easier problem.

femto 9y ago
> The pedanty here is insane. One person's pedant is another person's clear conscience. There's massive disagreement in the self-driving debate as to the extent to which we are dealing with safety critical systems. As you can no doubt tell, I'm firmly on the safety critical side. I've a fair bit of experience with designing and shipping critical infrastructure for emergency services. One has to ship, but in my experience the majority of people don't have the necessary eye for detail to make a safe system. This is why we have process: to allow fallible people to ship safe systems (and to avoid delusion in the minority who do have an eye for detail). The self-driving field is still developing their processes, so they have to be conservative with their shipping. Many of those arguing for self-driving are under estimating the safety of a typical human driver. The bar that a self-driving system has to meet is very high. If conditions are easy for a self-driving car then they are probably easy and safe for a human as well. Accidents are skewed towards a subset of poor conditions involving speed, alcohol and fatigue. Is it fair to replace a safe driver that avoids speed, alcohol and fatigue with a system that whilst safer than average is still less safe than the specific circumstance? Maybe an interim use of self-driving technology is to have the person driving, and the self-driving system in a co-driver/critique mode acting as an "unsafe driving detector"?
- LoSboccacc 9y ago
  ome can just look at the state of autonomous robots in industrial plants. they do a handful of functions, they have limited reach and they have predictable patterns and yet when you add humans to the mix all kind of dangerous conditions arise to the point most robots sharing space with humans need to be caged this is our current state of the art with automation and safety and yet we allow companies to slap a radar and a pretend safety driver and hurl a ton of metal down the pubblic roads autonomous vehicle will undoubtedly at some point be safer than human, but shortcutting known limitations like radars vs dodging static objects because 'drivers are the backup' is lunacy and the recent deaths are just a symptom of people never having worked in safety system design playing fast and loose with human lifes, and the companies which enable that should be made an example of by courts. some years ago serious automaker did their analisys and come up with the concept of designing the anticollision system first and use those as a backup to autonomous driving technology, that somehow has been lost in the race for profitability.
  - CaptainZapp 9y ago
    "this is our current state of the art with automation and safety and yet we allow companies to slap a radar and a pretend safety driver and hurl a ton of metal down the pubblic roads" This is what bugs me most: Who on earth ever thought it's a good idea to allow a company like Uber with their history of corner cutting and shady business to test such dangerous technology on public roads? The mind boggles.
nubbins 9y ago
I’m confused about this hypothetical- when ot starts raining or in any other adverse condition thousands of truck loads are just parked and delayed until it improves? I don’t see how this would put anyone out of work.
- stale2002 9y ago
  Imagine this on the highway in the desert. Is it ever going to start raining then? Or imagine if you just look up the weather ahead of time and only run the self driving trucks if there is a less than 5% chance of rain. These situations are still very common. You could replace all trucks on the highway in any desert climate, for example. And that's still many billions of dollars.
boulos 9y ago
> The pedanty here is insane. You mean pedantry :). -- Admiral Pedant.
darpa_escapee 9y ago
What's an acceptable time period for autonomous cars to fail catastrophically on the road until they are better than humans at driving safely?
shanghaiaway 9y ago
What's the point of a technology that only works under the least challenging conditions, that is probably less than one percent of the actual occurring conditions - and where the current solution, human beings, work better anyway because they don't have to stop for rain? What you're imagining is a car on a fixed track - a railcar. We already have trains but the US does need to build out their railway, yeah.
- stale2002 9y ago
  Because it is not 1% of circumstances. It would be more like 50% of circumstances. There are lots of places where it is sunny, and not raining and it is daytime. Just don't run the trucks on self driving mode during the other times where they don't work, and we could still save up to 50% in labor costs. Obviously, though, the market for self driving taxis is different than for self driving trucks, but once again, we could just only use them for trucks and still save Trillions.

nerfhammer 9y ago

There is going to be a higher threshold of self-driving vehicles, because random accidents by random people is not narrative-inspiring but for self-driving vehicles all accidents will be the fault of The Corporation. Every single individual accident will be the focus of world media fascination and speculation to derive how the motives of the corporation's singular supermind chose to allow it to happen, which will turn out to be that it was too greedy, scheming, uncaring, lazy, etc. There will be intense focus on a trickle of rare accidents because they are caused by a powerful corporate overmind that is potentially betraying us all, and little interest in a greater number of random accidents caused by no one of particular recognition for no discernable reason. Which is exactly what we've seen so far.

JumpCrisscross 9y ago
> random accidents by random people is not narrative-inspiring but for self-driving vehicles all accidents will be the fault of The Corporation I think we can conclude, absent conditions of wilful neglect, that this hypothesis is false. I thought this, too. But people are reacting reasonably to Tesla and Uber’s, erm, faux pas.
- jon_richards 9y ago
  I'm inclined to say "too reasonably" given the fact that the statistics seem to imply a much higher risk from the self-driving software than a human driver (Uber doesn't have many miles driven and Tesla is using overall-telsa statistics to hide the auto-pilot statistics).