11 ms·

What's really interesting about this is not just that Lattner is brilliant and liked, but that is highlights just how critical software correctness and reliabil

by abalone 10y ago

What's really interesting about this is not just that Lattner is brilliant and liked, but that is highlights just how critical software correctness and reliability is to autonomous vehicles. Naively one might have expected some machine learning expert to take over the reins at Tesla. But fast-moving Silicon Valley needs a fundamental shift in quality standards when it comes to safety-critical software, and if you look closely, Lattner has been leading this charge at the language and compiler level. His work has been distinguished by the melding of language safety, reliability and clarity, that is, not merely having sophisticated constructs that help the guarantee correctness, but also making code simple, beautiful and easy to read. Ultimately writing safe code depends on the ability of the programmer to comprehend it, so creating a programming environment that's successful on all fronts is a foundational achievement. A notable example: LLVM enabled ARC, a beautifully simple approach to memory management that removed much (not all) of the need for the developer to implement details in code, while providing high efficiency and, perhaps even more importantly, predictable performance (no garbage collection pauses). These are all essential for safety-critical realtime software.

Cyph0n 10y ago
That's not very surprising honestly. ML/AI algorithms alone are useless if they are written incorrectly or are not executed properly by the underlying OS/firmware.
tma-1 10y ago
Well considering that most of the autopilot software is driven by black box deep neural networks, I don't think hiring someone with a strong coding background is going to make that much of a difference when it comes to safety.
- tawaycozstoned 10y ago
  Sure it is. NNs can behave non-deterministic (i.e. they produce different output for same input because AFAIK some operations on GPUs are undeterministic) and it makes sense to invest into software systems that clearly take a closer look on software correctness. One idea to get correcter code is clean code layout (I heard good things about the LLVM code base), simple to read code and compilers that exploit theoretical knowledge we gained over years of compiler and type theory research. I think Chris Lattner has expertise in all of them (or at least knows about their importance, contributions and drawbacks), and if want to build a full-blown self-driving car it is important to have no indeterminism in your car, and advanced languages and compilers help to guarantee specific statements about your code. So it absolutely makes sense to invest in your compiler research team for safety, as our security, reliability and correctness expectations will rise (which is good) in particular for self-driving cars (they are not just DNNs).
  - deleted 10y ago
    [deleted]
  - tma-1 10y ago
    It ultimately depends on how much of the car's action is driven by DNNs, a miss-classification in the CNN can easily lead to a car crashing on a highway and causing a pileup. DNNs are so complex, you can't really write unit tests like you typically do for deterministic code. That's why some of the big banks flat out refuse to implement any form of deep learning for risk analytics. They're much more reliant on simpler ML models like random forests and logistic regression that are easier to analyse and diagnose by model governance teams.
    - tawaycozstoned 10y ago
      Yeah and it will happen. I am all for self-driving cars simply as they increase the life-quality for our civilization (more independency, cheaper ways of transportation, less fatalities), but there will be accidents and then you have to analyze them (which is good). In the end you want to know what did go wrong (the public will demand it and they are right) and it might be misclassification. But that is not enough. You want to know why did it classify situation X wrong? So the answer is because the inference network (which was created via the training network) computed its weights because the input was Y. Now you might throw your hands in the air and say "oh it's complicated, the network is undeterministic, blame NVidia", but you can also go farther and build your networks deterministic (which is possible and AFAIK not a performance penalty). Compiler research helps in at least guaranteeing that certain parts of code are deterministic which makes it easier to debug and maybe avoid complex NN misclassification scenarios, but the way to do it doesn't have to do much with NN's itself but more with language design and (real-time, in particular deterministic) OS research. So the statement oh, that's so complex, we do not no why we misclassified is no excusion, we can do better. For starters we have to publish NN papers with implementations that describe how to make a particular NN out of given trainigs data (provide that too). We already publish the code and network structure (see caffe, etc.), but often with pre-trained models that have been build on a cluster with many forms of training-data going through network structure, etc. Now at the moment you read a paper, head to the published code (often the case, again a desirable property of the ML community) and try to reproduce some examples by training on the data. However it is hard to say in the end, if your network is really as good as the published, as a simple $ diff my-net-binary-blob.dat tesla-net-binary-blob.dat might fail, if the stack (training etc.) to build the network is undeterministic. However if you have a good (i.e. deterministic) stack you might be able to reproduce NNs bit-for-bit, which makes it simpler to answer the question "why did we missclassify".
- simonh 10y ago
  So would you say that the reliability and correctness of the non-NN part of the system, and the code that implements the NN itself, is not worth bothering with because the system includes a black box NN in it's design? What a bizarre conclusion to draw!
devsquid 10y ago
ARC is a whole debate, but the one thing it is not is simple and I would argue its more error prone than a traditional GC. I've used it for most of my career and I have seen what sloppy/unaware coding can do to it. Lattner is a well known expert on compilers. Having used Swift since its inception, I would call into question the reliability of the Swift LLVM compiler. In its current state (3.0.2) its absolutely terrible and does not back up the sentiment; "But fast-moving Silicon Valley needs a fundamental shift in quality standards when it comes to safety-critical software, and if you look closely, Lattner has been leading this charge at the language and compiler level".
- 9935c101ab17a66 10y ago
  Can you explain why you call into question the reliability of the Swift LLVM compiler?
  - devsquid 10y ago
    Do you use Swift at all? -edit- I meant it as a serious question. But the person who responded to me sums up the issues.
    - chris_7 10y ago
      This is downvoted presumably for lack of information, but it's pretty much true. The Swift compiler segfaults very frequently. I do find this amusing in that it's the compiler for a theoretically largely-memory-safe language (yes the compiler is written in C++, it's still funny). The syntax highlighter in Xcode, which is driven by the same stuff, also crashes, which breaks autocompletion and even indentation. Using Xcode, you just have to get used to it. It frequently reports the wrong error message - just something that isn't even close to related. Sometimes object files just become 0 bytes and you either need to clean (and experience the Swift compiler's blazing performance again) or go and modify that file so that the incremental compiler will pick it up. I've found most of these to be triggered by using a lot of closures and possibly type inference. Shaking out the incorrect errors or segfaults is... not fun.
gigatexal 10y ago
software correctness eh? so build the autopilot and other systems in haskell?
freyir 10y ago
Swift, XCode, and Apple software in general don't scream reliability to me. No doubt Lattner is a smart guy, but I'd be more comfortable with a smart guy from NASA or somewhere similar with experience developing survival-critical software systems.
- devsquid 10y ago
  I know the first thing I thought about was does this mean Tesla's going to start writing its software in Swift. If so they are fucked :P. (BTW I write non-critical consumer software in Swift as my day job)
  - webjprgm 10y ago
    Swift has its problems but keeps getting better. I only get to use it for my at-home projects, since at work the portion of the codebase that is for iOS is Objective-C with no plans to switch any time soon. Old Obj-C too, and started by people used to programming in VB on MS platforms. So count your blessings.
    - devsquid 10y ago
      I find Swift to be getting worse each year. While sure i'm blessed that I don't have to write applications in assemble, I have grown jaded towards Swift. If it wasn't because I am an iOS developer, I would happily not use the language. I have been slowly positioning myself away from doing iOS development. I feel like a massive corporation like Apple can provide better tools to write apps for their walled garden than what they are providing me.
      - 72deluxe 10y ago
        What would you recommend instead? C#? C++? C++ has become incredibly great recently after languishing in the C++2003 period for too long.
        nicky0 10y ago
davidgay 10y ago
I like LLVM. But in the world of programming languages/compilers, someone like Xavier Leroy (a real, proved correct C compiler) is much more a name that I would associate with a drive for "software correctness and reliability".
pjmlp 10y ago
> LLVM enabled ARC, a beautifully simple approach to memory management that removed much (not all) of the need for the developer to implement details in code, while providing high efficiency and, perhaps even more importantly, predictable performance (no garbage collection pauses). These are all essential for safety-critical realtime software. A GC algorithm known since the early days of Lisp GC research and used in languages like Mesa/Cedar in the late 70's. I can think of lots of other examples, like how VB, Delphi and C++ Builder interfaced with COM in the mid-90's. Yes, ARC PR made automatic memory management to those without background in compiler research more easy to accept, specially if they weren't aware of the stability issues that trying to implement a tracing GC in Objective-C semantics meant. However it goes without saying, that Lattner is really brilliant and his work on LLVM as a compiler building tool, as well as, clang in regards to improving the status quo of C static analysis tooling and compiler error messages, is really notable. In regards to ARC, maybe also in terms of PR, as now there is a whole generation of developers that thinks reference counting isn't garbage collection.
- vintagedave 10y ago
  > I can think of lots of other examples, like how VB, Delphi and C++ Builder interfaced with COM in the mid-90's. One curious thing is how similar Swift is to modern Pascal, ie Delphi. It has many other influences, and it is certainly not Pascal with another syntax, but reading the Apple guide when Swift was released gave frequent moments of deja-vu. On the subject of COM, yes. If you think COM means ATL you should try it in a language designed to work with interfaces and reference counting inbuilt - Delphi has COM interfaces as first-class language primitives, and a heap of classes and other code making COM quite straightforward. Much of this even spills into C++Builder, though since it's a different language it's not as clean as in Delphi. Still miles past ATL though. Re ARC: I think the thing that makes it appealing is it's conceptually simple, completely deterministic, and can be traced by reading code rather than understanding an environment's implementation. Delphi does ARC now too, and if you've ever wanted ARC in C++, C++Builder optionally supports it for some classes. ARC is not yet on Windows for either language, we're talking just iOS, Android and soon Linux here. (Disclosure: I recently started working at Embarcadero on C++Builder. This is a personal comment only. But liking the languages Embarcadero makes was one main reason I joined.)
PinguTS 10y ago
If you want to talk about safety and reliability in automotive, then we talk about Autosar[1]. One of the concepts is to prevent the use of any dynamic behavior, which means you don't need any garbage collection at all. Because garbage collection is not predictable. It can be good and bad, but even a good one is not predictable. Embedded design and development is completely different to classical IT. For IT that sounds archaic. But it always proved right at the end. [1] https://en.wikipedia.org/wiki/AUTOSAR https://en.wikipedia.org/wiki/AUTOSAR
- Matthias247 10y ago
  True, AUTOSAR is the current standard for embedded automotive systems. Regarding safety I heard different things about it, some parties would only use it for up to ASIL A, others to C, others also to D. My personal opinion is that AUTOSAR (and even it's basics like C as a programming language and the OSEK OS) are not the best solutions that we should come up with for all the safety critical tasks in the autonomous driving world. Someone like Lattner would clearly have the potential to improve the current state of the art there a lot - but it's uncertain if Tesla would share any advantages with the remaining industry. And regarding memory allocation in reliable automotive systems: Yes, the best practice would be not to allocate at all to get to some deterministic behavior. However I've seen lots of projects where "don't allocate" is implemented as "don't allocate with malloc", and you find dozens of custom memory allocators and pools throughout the code. Some of those designs are probably less reliable and safe than using a garbage collected language would be.
  - therockhead 10y ago
    I often wonder would Erlang be a good language for embedded automotive systems. It's seems to have the right traits - no GC pauses, built for reliability etc.
    - Matthias247 10y ago
      I think it would fit great from the semantic. Message based communication is exactly what a lot of automotive systems are about (CAN, FlexRay, ...). The downside of the actor model from an implementation point of view is that you need potentially unbounded memory for the incoming message queue of each actor. For each message that is sent to an actor you need to allocate a message, populate it and place in that that actors queue. If the actor can't process fast enough the queue will fill hap. It might work out with a very careful system design, but determining statically how much queued messages and memory is needed seems like a very hard task.
thatfrenchguy 10y ago
"predictable performance (no garbage collection pauses)" No garbage collection means more fragmentation though which can be an issue at some point perf-wise, there's no free lunch.

by abalone 10y ago

Cyph0n 10y ago

That's not very surprising honestly. ML/AI algorithms alone are useless if they are written incorrectly or are not executed properly by the underlying OS/firmware.

tma-1 10y ago

Well considering that most of the autopilot software is driven by black box deep neural networks, I don't think hiring someone with a strong coding background is going to make that much of a difference when it comes to safety.

tawaycozstoned 10y ago
Sure it is. NNs can behave non-deterministic (i.e. they produce different output for same input because AFAIK some operations on GPUs are undeterministic) and it makes sense to invest into software systems that clearly take a closer look on software correctness. One idea to get correcter code is clean code layout (I heard good things about the LLVM code base), simple to read code and compilers that exploit theoretical knowledge we gained over years of compiler and type theory research. I think Chris Lattner has expertise in all of them (or at least knows about their importance, contributions and drawbacks), and if want to build a full-blown self-driving car it is important to have no indeterminism in your car, and advanced languages and compilers help to guarantee specific statements about your code. So it absolutely makes sense to invest in your compiler research team for safety, as our security, reliability and correctness expectations will rise (which is good) in particular for self-driving cars (they are not just DNNs).
- deleted 10y ago
  [deleted]
- tma-1 10y ago
  It ultimately depends on how much of the car's action is driven by DNNs, a miss-classification in the CNN can easily lead to a car crashing on a highway and causing a pileup. DNNs are so complex, you can't really write unit tests like you typically do for deterministic code. That's why some of the big banks flat out refuse to implement any form of deep learning for risk analytics. They're much more reliant on simpler ML models like random forests and logistic regression that are easier to analyse and diagnose by model governance teams.
  - tawaycozstoned 10y ago
    Yeah and it will happen. I am all for self-driving cars simply as they increase the life-quality for our civilization (more independency, cheaper ways of transportation, less fatalities), but there will be accidents and then you have to analyze them (which is good). In the end you want to know what did go wrong (the public will demand it and they are right) and it might be misclassification. But that is not enough. You want to know why did it classify situation X wrong? So the answer is because the inference network (which was created via the training network) computed its weights because the input was Y. Now you might throw your hands in the air and say "oh it's complicated, the network is undeterministic, blame NVidia", but you can also go farther and build your networks deterministic (which is possible and AFAIK not a performance penalty). Compiler research helps in at least guaranteeing that certain parts of code are deterministic which makes it easier to debug and maybe avoid complex NN misclassification scenarios, but the way to do it doesn't have to do much with NN's itself but more with language design and (real-time, in particular deterministic) OS research. So the statement oh, that's so complex, we do not no why we misclassified is no excusion, we can do better. For starters we have to publish NN papers with implementations that describe how to make a particular NN out of given trainigs data (provide that too). We already publish the code and network structure (see caffe, etc.), but often with pre-trained models that have been build on a cluster with many forms of training-data going through network structure, etc. Now at the moment you read a paper, head to the published code (often the case, again a desirable property of the ML community) and try to reproduce some examples by training on the data. However it is hard to say in the end, if your network is really as good as the published, as a simple $ diff my-net-binary-blob.dat tesla-net-binary-blob.dat might fail, if the stack (training etc.) to build the network is undeterministic. However if you have a good (i.e. deterministic) stack you might be able to reproduce NNs bit-for-bit, which makes it simpler to answer the question "why did we missclassify".
simonh 10y ago
So would you say that the reliability and correctness of the non-NN part of the system, and the code that implements the NN itself, is not worth bothering with because the system includes a black box NN in it's design? What a bizarre conclusion to draw!

devsquid 10y ago

ARC is a whole debate, but the one thing it is not is simple and I would argue its more error prone than a traditional GC. I've used it for most of my career and I have seen what sloppy/unaware coding can do to it. Lattner is a well known expert on compilers. Having used Swift since its inception, I would call into question the reliability of the Swift LLVM compiler. In its current state (3.0.2) its absolutely terrible and does not back up the sentiment; "But fast-moving Silicon Valley needs a fundamental shift in quality standards when it comes to safety-critical software, and if you look closely, Lattner has been leading this charge at the language and compiler level".

9935c101ab17a66 10y ago
Can you explain why you call into question the reliability of the Swift LLVM compiler?
- devsquid 10y ago
  Do you use Swift at all? -edit- I meant it as a serious question. But the person who responded to me sums up the issues.
  - chris_7 10y ago
    This is downvoted presumably for lack of information, but it's pretty much true. The Swift compiler segfaults very frequently. I do find this amusing in that it's the compiler for a theoretically largely-memory-safe language (yes the compiler is written in C++, it's still funny). The syntax highlighter in Xcode, which is driven by the same stuff, also crashes, which breaks autocompletion and even indentation. Using Xcode, you just have to get used to it. It frequently reports the wrong error message - just something that isn't even close to related. Sometimes object files just become 0 bytes and you either need to clean (and experience the Swift compiler's blazing performance again) or go and modify that file so that the incremental compiler will pick it up. I've found most of these to be triggered by using a lot of closures and possibly type inference. Shaking out the incorrect errors or segfaults is... not fun.

gigatexal 10y ago

software correctness eh? so build the autopilot and other systems in haskell?

freyir 10y ago

Swift, XCode, and Apple software in general don't scream reliability to me. No doubt Lattner is a smart guy, but I'd be more comfortable with a smart guy from NASA or somewhere similar with experience developing survival-critical software systems.

devsquid 10y ago
I know the first thing I thought about was does this mean Tesla's going to start writing its software in Swift. If so they are fucked :P. (BTW I write non-critical consumer software in Swift as my day job)
- webjprgm 10y ago
  Swift has its problems but keeps getting better. I only get to use it for my at-home projects, since at work the portion of the codebase that is for iOS is Objective-C with no plans to switch any time soon. Old Obj-C too, and started by people used to programming in VB on MS platforms. So count your blessings.
  - devsquid 10y ago
    I find Swift to be getting worse each year. While sure i'm blessed that I don't have to write applications in assemble, I have grown jaded towards Swift. If it wasn't because I am an iOS developer, I would happily not use the language. I have been slowly positioning myself away from doing iOS development. I feel like a massive corporation like Apple can provide better tools to write apps for their walled garden than what they are providing me.
    - 72deluxe 10y ago
      What would you recommend instead? C#? C++? C++ has become incredibly great recently after languishing in the C++2003 period for too long.
      - nicky0 10y ago

davidgay 10y ago

I like LLVM. But in the world of programming languages/compilers, someone like Xavier Leroy (a real, proved correct C compiler) is much more a name that I would associate with a drive for "software correctness and reliability".

pjmlp 10y ago

> LLVM enabled ARC, a beautifully simple approach to memory management that removed much (not all) of the need for the developer to implement details in code, while providing high efficiency and, perhaps even more importantly, predictable performance (no garbage collection pauses). These are all essential for safety-critical realtime software. A GC algorithm known since the early days of Lisp GC research and used in languages like Mesa/Cedar in the late 70's. I can think of lots of other examples, like how VB, Delphi and C++ Builder interfaced with COM in the mid-90's. Yes, ARC PR made automatic memory management to those without background in compiler research more easy to accept, specially if they weren't aware of the stability issues that trying to implement a tracing GC in Objective-C semantics meant. However it goes without saying, that Lattner is really brilliant and his work on LLVM as a compiler building tool, as well as, clang in regards to improving the status quo of C static analysis tooling and compiler error messages, is really notable. In regards to ARC, maybe also in terms of PR, as now there is a whole generation of developers that thinks reference counting isn't garbage collection.

vintagedave 10y ago
> I can think of lots of other examples, like how VB, Delphi and C++ Builder interfaced with COM in the mid-90's. One curious thing is how similar Swift is to modern Pascal, ie Delphi. It has many other influences, and it is certainly not Pascal with another syntax, but reading the Apple guide when Swift was released gave frequent moments of deja-vu. On the subject of COM, yes. If you think COM means ATL you should try it in a language designed to work with interfaces and reference counting inbuilt - Delphi has COM interfaces as first-class language primitives, and a heap of classes and other code making COM quite straightforward. Much of this even spills into C++Builder, though since it's a different language it's not as clean as in Delphi. Still miles past ATL though. Re ARC: I think the thing that makes it appealing is it's conceptually simple, completely deterministic, and can be traced by reading code rather than understanding an environment's implementation. Delphi does ARC now too, and if you've ever wanted ARC in C++, C++Builder optionally supports it for some classes. ARC is not yet on Windows for either language, we're talking just iOS, Android and soon Linux here. (Disclosure: I recently started working at Embarcadero on C++Builder. This is a personal comment only. But liking the languages Embarcadero makes was one main reason I joined.)

PinguTS 10y ago

If you want to talk about safety and reliability in automotive, then we talk about Autosar[1]. One of the concepts is to prevent the use of any dynamic behavior, which means you don't need any garbage collection at all. Because garbage collection is not predictable. It can be good and bad, but even a good one is not predictable. Embedded design and development is completely different to classical IT. For IT that sounds archaic. But it always proved right at the end. [1] https://en.wikipedia.org/wiki/AUTOSAR https://en.wikipedia.org/wiki/AUTOSAR

Matthias247 10y ago
True, AUTOSAR is the current standard for embedded automotive systems. Regarding safety I heard different things about it, some parties would only use it for up to ASIL A, others to C, others also to D. My personal opinion is that AUTOSAR (and even it's basics like C as a programming language and the OSEK OS) are not the best solutions that we should come up with for all the safety critical tasks in the autonomous driving world. Someone like Lattner would clearly have the potential to improve the current state of the art there a lot - but it's uncertain if Tesla would share any advantages with the remaining industry. And regarding memory allocation in reliable automotive systems: Yes, the best practice would be not to allocate at all to get to some deterministic behavior. However I've seen lots of projects where "don't allocate" is implemented as "don't allocate with malloc", and you find dozens of custom memory allocators and pools throughout the code. Some of those designs are probably less reliable and safe than using a garbage collected language would be.
- therockhead 10y ago
  I often wonder would Erlang be a good language for embedded automotive systems. It's seems to have the right traits - no GC pauses, built for reliability etc.
  - Matthias247 10y ago
    I think it would fit great from the semantic. Message based communication is exactly what a lot of automotive systems are about (CAN, FlexRay, ...). The downside of the actor model from an implementation point of view is that you need potentially unbounded memory for the incoming message queue of each actor. For each message that is sent to an actor you need to allocate a message, populate it and place in that that actors queue. If the actor can't process fast enough the queue will fill hap. It might work out with a very careful system design, but determining statically how much queued messages and memory is needed seems like a very hard task.

thatfrenchguy 10y ago

"predictable performance (no garbage collection pauses)" No garbage collection means more fragmentation though which can be an issue at some point perf-wise, there's no free lunch.