5 ms·

Couldn't agree more with every point listed. These are fantastic points. I do not know the author, but reading these points makes me suspect he/she is an exper

by jwr 7y ago

Couldn't agree more with every point listed. These are fantastic points. I do not know the author, but reading these points makes me suspect he/she is an experienced software engineer that has been doing this for many years now. I expect that especially his first point (about being humble in the face of software systems complexity) will provoke many hubris-filled comments. I fully agree with the author: we are incapable of building complex and correct software systems. This is why I am afraid of the hype behind self-driving vehicles.

barrkel 7y ago
I have a different expectation of correctness; absolute correctness is impossible, in practice correctness is not a binary condition. Even if something provably executes correctly according to spec, the spec may be wrong (and probably is, since everything interfaces with humans eventually). Everything is in a process of becoming; 100% correctness is not achievable, but it's not desirable either - it costs too much; it just needs to be better than the next best alternative.
- makach 7y ago
  Even with 100% correctness, your solution still has the capacity to fail spectacularly. The universe is infinite in its desire to mess with your expectations and assumptions of what correctness is! Oh, you thought that was 100% correct? Well, have a look: A platypus.
- AstralStorm 7y ago
  The proper goal is robustness. It is either achieved via clear indication of failure and guidance on alternative solutions - so that the user can handle it. Or by actually failing gracefully and handling as many error conditions in an always reasonable way. Usually letting user handle it is more general as long as the failures are rare enough. Internal correctness is perfectly achievable though. External (correct spec) is not. There are always some unhandled conditions, due to hardware or external component failures...
- bregma 7y ago
  Wow, that's along-winded way of saying "perfect is the enemy of good enough".
- mark-r 7y ago
  I once worked at a place where a bug in the spec was treated as impossible - if people couldn't get the results they wanted because the spec made it impossible, there was no way to get it fixed. Any attempt to log it as a bug was rejected.
bob33212 7y ago
autonomous driving is a great example here. It will never be "correct" because there will always be accidents. So the author is right about that. But it is very possible that those cars may be 10x safer than the average human driver some day.
- EugeneOZ 7y ago
  what is a measure for correctness in this case? There are incidents with humans drivers.
- est31 7y ago
  What if you are an above average driver? Maybe for most people it would be an improvement in security, but for you it'd be more dangerous to be in an autonomous car.
  - groby_b 7y ago
    If you're an above average driver, a below average driver will ram your car unexpectedly. And there will be nothing you can do, because your reaction times are human. Also, you're likely not as above average as you think yourself to be.
    - est31 7y ago
      > If you're an above average driver, a below average driver will ram your car unexpectedly. And there will be nothing you can do, because your reaction times are human. That doesn't require level 5 self driving cars, only brake assistants. > Also, you're likely not as above average as you think yourself to be. That doesn't mean that there aren't above average drivers. I'm not assuming I'm among them, but for them, driving in a self driving car would make matters worse.
      - groby_b 7y ago
        It requires more than brake assistants. People are, in the limit, idiots (at least when driving). If you drive long enough, you'll see lots of really hair rising things - and the benefit for the above average driver is that the below-average drivers are off the road. The benefit to self-driving cars is not only that they are driving (hopefully) better than humans - it's also that they minimize the risk of completely erratic behavior. People don't have accidents when they're doing average. They have accidents when they're at their worst. Last example I witnessed: At a traffic light, a car going ~30mph suddenly and without warning pulls into oncoming traffic because for whatever reason they thought "must left turn" - despite left turns being clearly disallowed. That wasn't just a "below average driver", because that doesn't happen every day. That was somebody who was very clearly way off their game. And that's the main source of accidents, doing something clearly wrong that you likely wouldn't even think about on a good day. Or being distracted. Taking these things out of the equation is way more important than "driving better than the best human on their best day". And this will benefit the above-average drivers as well. (Also, the average will likely move up, so "above average human" might suddenly not mean a lot)
prox 7y ago
Cooper: Hey TARS, what's your honesty parameter? TARS: 90 percent. Cooper: 90 percent? TARS: Absolute honesty isn't always the most diplomatic nor the safest form of communication with emotional beings. Cooper: Okay, 90 percent it is.
- faeyanpiraat 7y ago
  Self destruct in 10.. 9..
slumdev 7y ago
"Beware of bugs in the above code; I have only proved it correct, not tried it." - Donald Knuth
131012 7y ago
I would say he/she is an experienced "person", but many of these points, including some about software, can be understood simply by an honest understanding of human beings.
patja 7y ago
> reading these points makes me suspect he/she is an experienced software engineer that has been doing this for many years now. And yet, a cursory review indicates they are barely out of high school and has never worked on software in a commercial environment. Not casting stones (I agree with some of what they say too, and what I agree with I find insightful. How's that for confirmation bias?), just observing that this is in fact not based on much experience at all.
jimbokun 7y ago
> This is why I am afraid of the hype behind self-driving vehicles. Well, they don't have to be absolutely correct. Just a lot more correct than human drivers across a broad range of driving scenarios to justify replacing human drivers.

by jwr 7y ago

barrkel 7y ago

I have a different expectation of correctness; absolute correctness is impossible, in practice correctness is not a binary condition. Even if something provably executes correctly according to spec, the spec may be wrong (and probably is, since everything interfaces with humans eventually). Everything is in a process of becoming; 100% correctness is not achievable, but it's not desirable either - it costs too much; it just needs to be better than the next best alternative.

makach 7y ago
Even with 100% correctness, your solution still has the capacity to fail spectacularly. The universe is infinite in its desire to mess with your expectations and assumptions of what correctness is! Oh, you thought that was 100% correct? Well, have a look: A platypus.
AstralStorm 7y ago
The proper goal is robustness. It is either achieved via clear indication of failure and guidance on alternative solutions - so that the user can handle it. Or by actually failing gracefully and handling as many error conditions in an always reasonable way. Usually letting user handle it is more general as long as the failures are rare enough. Internal correctness is perfectly achievable though. External (correct spec) is not. There are always some unhandled conditions, due to hardware or external component failures...
bregma 7y ago
Wow, that's along-winded way of saying "perfect is the enemy of good enough".
mark-r 7y ago
I once worked at a place where a bug in the spec was treated as impossible - if people couldn't get the results they wanted because the spec made it impossible, there was no way to get it fixed. Any attempt to log it as a bug was rejected.

bob33212 7y ago

autonomous driving is a great example here. It will never be "correct" because there will always be accidents. So the author is right about that. But it is very possible that those cars may be 10x safer than the average human driver some day.

EugeneOZ 7y ago
what is a measure for correctness in this case? There are incidents with humans drivers.
est31 7y ago
What if you are an above average driver? Maybe for most people it would be an improvement in security, but for you it'd be more dangerous to be in an autonomous car.
- groby_b 7y ago
  If you're an above average driver, a below average driver will ram your car unexpectedly. And there will be nothing you can do, because your reaction times are human. Also, you're likely not as above average as you think yourself to be.
  - est31 7y ago
    > If you're an above average driver, a below average driver will ram your car unexpectedly. And there will be nothing you can do, because your reaction times are human. That doesn't require level 5 self driving cars, only brake assistants. > Also, you're likely not as above average as you think yourself to be. That doesn't mean that there aren't above average drivers. I'm not assuming I'm among them, but for them, driving in a self driving car would make matters worse.
    - groby_b 7y ago
      It requires more than brake assistants. People are, in the limit, idiots (at least when driving). If you drive long enough, you'll see lots of really hair rising things - and the benefit for the above average driver is that the below-average drivers are off the road. The benefit to self-driving cars is not only that they are driving (hopefully) better than humans - it's also that they minimize the risk of completely erratic behavior. People don't have accidents when they're doing average. They have accidents when they're at their worst. Last example I witnessed: At a traffic light, a car going ~30mph suddenly and without warning pulls into oncoming traffic because for whatever reason they thought "must left turn" - despite left turns being clearly disallowed. That wasn't just a "below average driver", because that doesn't happen every day. That was somebody who was very clearly way off their game. And that's the main source of accidents, doing something clearly wrong that you likely wouldn't even think about on a good day. Or being distracted. Taking these things out of the equation is way more important than "driving better than the best human on their best day". And this will benefit the above-average drivers as well. (Also, the average will likely move up, so "above average human" might suddenly not mean a lot)

prox 7y ago

Cooper: Hey TARS, what's your honesty parameter? TARS: 90 percent. Cooper: 90 percent? TARS: Absolute honesty isn't always the most diplomatic nor the safest form of communication with emotional beings. Cooper: Okay, 90 percent it is.

faeyanpiraat 7y ago
Self destruct in 10.. 9..

slumdev 7y ago

"Beware of bugs in the above code; I have only proved it correct, not tried it." - Donald Knuth

131012 7y ago

I would say he/she is an experienced "person", but many of these points, including some about software, can be understood simply by an honest understanding of human beings.

patja 7y ago

> reading these points makes me suspect he/she is an experienced software engineer that has been doing this for many years now. And yet, a cursory review indicates they are barely out of high school and has never worked on software in a commercial environment. Not casting stones (I agree with some of what they say too, and what I agree with I find insightful. How's that for confirmation bias?), just observing that this is in fact not based on much experience at all.

jimbokun 7y ago

> This is why I am afraid of the hype behind self-driving vehicles. Well, they don't have to be absolutely correct. Just a lot more correct than human drivers across a broad range of driving scenarios to justify replacing human drivers.