4 ms·

Because by "works" you mean "I tried it out a bunch of times and nothing bad happened, so must be production ready", because that's what you do when you build w

Because by "works" you mean "I tried it out a bunch of times and nothing bad happened, so must be production ready", because that's what you do when you build websites and desktop CRUD apps and nobody ever died cause the web server choked under load or the web page rendered a bit funny or the request took a full quarter second because the GC kicked off as a wave of requests came in. And then one day a one-in-ten-million event that never would've surfaced during testing does happen and the OS crashes or the interpreter hangs or some non-determinism causes a period of non-responsiveness or a bit gets mangled because you don't have any redundancies and solar rays are thing or ... and someone dies. And of course that didn't happen in testing, because all of those things are rare possibilities. And then it happens again. And again. And then you're in court being sued for millions. And a bunch of industry experts who do build their systems with redundancies and do design their systems with a safety-first mindset come to the stand and rip you apart for not taking even the most basic precautions. And when it comes out just how many best practices you completely ignored, you're mostly just hoping beyond hope that all of the legal problems stay on the civil side of the civil/criminal divide. Or, to say it in a sentence, "because the sort of exceptional circumstances that safety-critical software needs to handle with grace are very difficult or impossible to account for with a desktop machine running interpreted code on top of Linux."

hasbroslasher 10y ago
I like that you reduced that to a one-liner, and I understand your point. I sure as hell wouldn't hook my car up to this software. But I also feel like you're being a bit hasty. Obviously a Python script isn't going to turn into a Tesla overnight. But maybe it'll help you find a few bugs before you throw all that time and effort into building the real deal. When I look at a Github repo that claims to be a self-driving car I don't say to myself "yep, looks production ready," I say "Cool prototype, now let's break it." To me it seems entirely reasonable to get a working Python prototype 90% of the way there and then send it off to the OS programmers to design something that actually meets the concept of "production ready".
- throwaway729 10y ago
  Yes, I absolutely agree with you on that. Nothing wrong with prototyping in whatever setting is most convenient. I think w/ self-driving cars there is an interesting ethical question. The full auto cars on the roads today definitely aren't production ready, but they also have constant safety drivers. Probably even ACC systems are tested in the wild with a safety driver before production. Basically, "is the safety driver sufficient to justify running prototype software in the real world?"
  - hasbroslasher 10y ago
    Yeah, that's a tough question that we as a society will have to wrestle with. Knowing that humans are imperfect drivers as well doesn't make it any easier. Even cars today have crippling safety flaws -- remember that one Lexus model that had a sticky gas pedal? Or worse yet, the Ford Pinto. I'm genuinely interested to see how the governments of the world weigh in on this, if at all. Intuitively, though, I think that buying/using the software is tantamount to accepting its imperfections, so long as they are adequately (factually) presented to you beforehand. You're signing off your ability to make your own decisions, but are still responsible for them.
sapphireblue 10y ago
You have a lot of solid points, but note that Linux is currently being used by SpaceX in an even more safety-critical aerospace setting. Also note that interpreters have their place in safety-critical aerospace as well: some satellites run Forth.
- sangnoir 10y ago
  > You have a lot of solid points, but note that Linux is currently being used by SpaceX in an even more safety-critical aerospace setting I disagree with the 'more' - how many lives are at risk with a SpaceX failure vs. a self-driving car failure? This is even without multiplying by number of users.
  - Kliment 10y ago
    There are few things more destructive than a serious spacecraft crash.
nightski 10y ago
You are really going to freak out when you learn how deep neural networks work.
- throwaway729 10y ago
  I know how DNNs work -- I've even designed my own. The existence of a component that is difficult to analyze for safety doesn't justify ignoring well-established safety engineering techniques throughout the rest of the system. That attitude would have us throw out seat belts and snow tires just because the ACC system might be buggy sometimes.

hasbroslasher 10y ago

I like that you reduced that to a one-liner, and I understand your point. I sure as hell wouldn't hook my car up to this software. But I also feel like you're being a bit hasty. Obviously a Python script isn't going to turn into a Tesla overnight. But maybe it'll help you find a few bugs before you throw all that time and effort into building the real deal. When I look at a Github repo that claims to be a self-driving car I don't say to myself "yep, looks production ready," I say "Cool prototype, now let's break it." To me it seems entirely reasonable to get a working Python prototype 90% of the way there and then send it off to the OS programmers to design something that actually meets the concept of "production ready".

throwaway729 10y ago
Yes, I absolutely agree with you on that. Nothing wrong with prototyping in whatever setting is most convenient. I think w/ self-driving cars there is an interesting ethical question. The full auto cars on the roads today definitely aren't production ready, but they also have constant safety drivers. Probably even ACC systems are tested in the wild with a safety driver before production. Basically, "is the safety driver sufficient to justify running prototype software in the real world?"
- hasbroslasher 10y ago
  Yeah, that's a tough question that we as a society will have to wrestle with. Knowing that humans are imperfect drivers as well doesn't make it any easier. Even cars today have crippling safety flaws -- remember that one Lexus model that had a sticky gas pedal? Or worse yet, the Ford Pinto. I'm genuinely interested to see how the governments of the world weigh in on this, if at all. Intuitively, though, I think that buying/using the software is tantamount to accepting its imperfections, so long as they are adequately (factually) presented to you beforehand. You're signing off your ability to make your own decisions, but are still responsible for them.

sapphireblue 10y ago

You have a lot of solid points, but note that Linux is currently being used by SpaceX in an even more safety-critical aerospace setting. Also note that interpreters have their place in safety-critical aerospace as well: some satellites run Forth.

sangnoir 10y ago
> You have a lot of solid points, but note that Linux is currently being used by SpaceX in an even more safety-critical aerospace setting I disagree with the 'more' - how many lives are at risk with a SpaceX failure vs. a self-driving car failure? This is even without multiplying by number of users.
- Kliment 10y ago
  There are few things more destructive than a serious spacecraft crash.

nightski 10y ago

You are really going to freak out when you learn how deep neural networks work.

throwaway729 10y ago
I know how DNNs work -- I've even designed my own. The existence of a component that is difficult to analyze for safety doesn't justify ignoring well-established safety engineering techniques throughout the rest of the system. That attitude would have us throw out seat belts and snow tires just because the ACC system might be buggy sometimes.