11 ms·

Teaching Compilers Backward

271 points by azhenley 6y ago

toolslive 6y ago
The bottom up order is the right one. The classic paper: An Incremental Approach to Compiler Construction (Abdulaziz Ghuloum) https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&ved=2ahUKEwjC2Kjrr4DvAhUTHOwKHXg1C4sQFjACegQIBRAD&url=http%3A%2F%2Fwww.schemeworkshop.org%2F2006%2F11-ghuloum.pdf&usg=AOvVaw1i0Fyy4oDI8CRVKTGicR7n https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&c...
- Mathnerd314 6y ago
  They link this paper in the bottom of the article as a different approach from the one they're using.
- tom_mellior 6y ago
  That's not really the same approach. It's not "backward", it's more "sideways": It starts with an end-to-end compiler for a tiny language, then builds out that end-to-end compiler incrementally. The featured article describes an approach where no such end-to-end compiler is available from the beginning. It's only end-to-end after the final exercise.
agnokapathetic 6y ago
This approach has the added benefit of teaching "mechanical sympathy". We're often told "Let the complier do the work" without really understanding what "work" the compiler is doing. Learning concepts like branch prediction, out-of-order execution, pipelining, register planning, caches, memory management is far far more important than Lexing/Parsing/Intermediate Representations and SSA.
- deleted 6y ago
  [deleted]
- jfoutz 6y ago
  I have a different take. Compilers are usually the first really big project students encounter. There's a lot of complexity around keeping things organized and still efficient. There's a lot of real world hassle around both ends - the input can be anything, and the implementer has to deal with that. The output is subtle with lots of gnarly little details that have to be exactly right. Compilers also have this wonderful side effect of eliminating all the magic. after a compiler class, there aren't many mysteries left about how computers work. they take a string and do stuff. Personally, I haven't gotten much milage out of shift reduce parsing, but parsing in general has been super important. Intermediate representations, coming from complexity to something canonical, and then transforming that back to complexity has probably given me the biggest bang for my time. If you squint your eyes, frameworks and dsls are kinda just intermediate representations. Sticking the parts together in a coherent way helped me. I've never been in a situation where out-of-order execution really mattered to my code. but my compiler course taught me enough to know valgrind, so I'd go look for whatever the modern equivalent is. Everybody is different, but compilers for me is where everything really jelled. There is so much complexity you just eat, day after day. Memorization only got me so far, at some point, I had to have real organization and clarity. I think, if I had to give a compiler class, I'd start in the middle. tools for manipulating trees, tools to build that tree up, tools to tear that tree down. I know it's kind of a breezy approach, but the clarity seems like a prerequisite for dealing with complexity. the tree is the one thing the implementer owns, and is free to structure (and restructure) however they wish.
- 908B64B197 6y ago
  These belong in a separate class about low level systems (CPU architecture and assembly).
- konjin 6y ago
  >Learning concepts like branch prediction, out-of-order execution, pipelining, register planning, caches, memory management is far far more important than Lexing/Parsing/Intermediate Representations and SSA. These are incidents of technology. If this were 1970s you'd be talking about in place sorting because we're stuck with tapes for anything past a toy compiler. Something that would be useless in a decade. The point of education is to teach you how to learn, not how to collect stamps.
dominicjj 6y ago
This is pretty cool. I'm currently teaching myself compiler theory so I can write a toy compiler for a C-like language targeting x86-64 and I've also started at the bottom: re-acquiring my assembly skills so I can then map common idioms back to some sort of IR. The disadvantage is you can't perform high level optimizations that you could do when transforming from SSA to the IR but that is a long way away for now.
smasher164 6y ago
Much to the chagrin of a lot of educators, I think this approach is the way to go. Too many compilers classes get bogged down in grammar classifications and parsing. Some argue that parsing is a microcosm of the rest of the compiler, since it requires one to transform a program from one representation to another. However, this message doesn't really come across when you're operating on highly unstructured input, and not doing any simplification passes. I think execution is really important for visualizing the effects of your work.
- pjmlp 6y ago
  Agreed, we did it all the way, but that was just in the course of a 5 years degree there was room to spend three semesters having compiler design related lectures, split between parser theory, denotational semantics, programming language history and implementing our own toy compiler.
- JoelMcCracken 6y ago
  This is what happened in my compilers class. We spent, like, the whole thing talking about grammars etc. Which is mind-numbingly boring (to me), and especially given a significant number of industry parsers are hand-rolled/recursive descent style, its not terribly useful information.
  - JoelMcCracken 6y ago
    At the moment, some of us in the PGH FP group are working through https://www.cs.cmu.edu/~rwh/pfpl/ https://www.cs.cmu.edu/~rwh/pfpl/, and so far I vastly prefer it. It is math heavy, which I like for various personal reasons which are irrelevant here, but it also skips parsing!
  - ahartmetz 6y ago
    Same here, at FU Berlin. We spent several weeks on LR and LALR parsers and such and, IIRC, even more than a week on regular expressions. Somehow we talked about two-address and three-address instructions, but never seemed to do any code generation, and certainly no optimization. It was very disappointing.
mamcx 6y ago
Interesting, this is how I actually make progres building my own lang in the side (https://tablam.org https://tablam.org). I discover "early" that parsing will be at mercy of each change in direction on the internals, making a costing rewrite each time. Is like premature, aimless, unit-testing, where the cost of testing dominate the actual code to write. So, eventually I split my efforts in 2: On the side I just write programs in the imaginary syntax and change it even in each line. But the whole action was from AST->Execution. I don't bother to parsing until very late in the game, and I think I could have deferred even more. This is not because I believe the syntax "don't matter", I think the opposite!, but you can't know which syntax its the best until you nail the semantics. If the parsing/syntax is introduced too early you could going backwards to support THAT syntax bolted on the semantics, instead of left the semantics guide you. ie: Is like UX/UI: If the program is not well defined, the UI will distort it.
travisgriggs 6y ago
Good it be loosely posited, that they're teaching decompilation in the forward sense?
p4bl0 6y ago
The idea is so simple it seems obvious once it has been told, which the sign of a fantastic idea. It's too late for this year but I will try this approach next year in my compilers course!
harperlee 6y ago
In a similar vein, networking is typically taught bottom-up, but some people swear by the backwards top-down approach: https://www.goodreads.com/book/show/83847.Computer_Networking https://www.goodreads.com/book/show/83847.Computer_Networkin...
- renox 6y ago
  Yep, I had two classes about network the first was 'top to bottom' and let me really confused, the second was 'bottom to top' and I understood the whole lesson without difficulty.. That said, it could be me: I was never able to get inheritance until I learned how it was implemented..
nahuel0x 6y ago
This direction is also analogous to the historical development of programming languages, from bytecodes to textual assembly instructions to (something like) macro-assembler.
Mathnerd314 6y ago
They call it backward or bottom-up but really it's middle-out: they start with valid AST, learn how to compile it out, then add a frontend that generates ASTs.
waynesonfire 6y ago
Can anyone recommend resources that teach compilers in reverse?
- CalChris 6y ago
  https://github.com/aalhour/awesome-compilers https://github.com/aalhour/awesome-compilers
hardwaregeek 6y ago
I learned how to compile to WebAssembly by getting nice and intimate with the WebAssembly spec, learning the ins and outs, then writing a basic emitter for the bytecode, then writing a code generator for basic operations like arithmetic, then extending it to locals, then first class functions, etc. So kind of backwards. I did start with a parser and typechecker, but if anything that made it too difficult. I had all of these extremely high level concepts that I struggled to map to WebAssembly (pattern matching, closures, etc.) and discouraged me in the process. If I had started with WebAssembly and moved to my high level language, I'd probably have restricted myself to a much more practical set of language features. On the flip side, my language would probably be influenced by how easy it is to code generate to WASM, which isn't necessarily what I want.
carapace 6y ago
Yeah, FWIW, I think you should start at the bottom[1] with bits, bytes, RAM, and a wee CPU (like Wirth's RISC for Project Oberon[2]) and then META-II [3] seems like the best way forward to conventional languages. (Or go with Forth and Joy and head right past Lambda Calculus and straight to Category Theory[4].) On the other hand you could start with predicate logic and teach Prolog as a calculator for same, then develop all the low- and mid-level software (your emulator and compiler) using Prolog, then implement a Prolog interpreter in your language to close the loop. [1] http://phoenixbureau.github.io/PigeonComputer/curiculum.html http://phoenixbureau.github.io/PigeonComputer/curiculum.html [2] https://pythonoberon.readthedocs.io/en/latest/ https://pythonoberon.readthedocs.io/en/latest/ [3] https://en.wikipedia.org/wiki/META_II https://en.wikipedia.org/wiki/META_II [3] http://conal.net/papers/compiling-to-categories/ http://conal.net/papers/compiling-to-categories/
orthoxerox 6y ago
A project that was all about this approach: https://speakerdeck.com/nineties/creating-a-language-using-only-assembly-language https://speakerdeck.com/nineties/creating-a-language-using-o...
- dominicjj 6y ago
  That's an inspiring slide deck - thanks for sharing.
recursivedoubts 6y ago
I prefer the traditional lexer -> parser -> optimization -> code generator path. If I were going to change anything about teaching compilers, it would be to teach recursive descent, which makes the recursive nature of grammars a lot more obvious than parser generator approaches do. This is a great book/website a contributor to Dart, which explains the algorithm: https://craftinginterpreters.com/ https://craftinginterpreters.com/
- thomaslkjeldsen 6y ago
  Elaborating from https://en.wikipedia.org/wiki/Dart_(programming_language) https://en.wikipedia.org/wiki/Dart_(programming_language) > Dart was unveiled at the GOTO conference in Aarhus, Denmark, October 10–12, 2011. The project was founded by Lars Bak and Kasper Lund.
nickysielicki 6y ago
Disagree. I think by the time that most students get to the point of taking a compilers course, they can understand (at a high level) what each of these stages is doing with no motivation needed. When I took my compilers course, we went over the stages of a compiler in the first 10 minutes of the first lecture, and I think everyone "got it" right away.
deleted 6y ago
[deleted]
jl2718 6y ago
I was unaware of, and still confused by the other way, but now I understand all the bad ideas people come into practice with regarding compilers.
tester756 6y ago
>implementing passes over the AST using visitors, Why Visitor Pattern is so popular around ASTs? I always feel like this particular patterns adds too much boilerplate and is not really needed in "modern" languages.
- the-smug-one 6y ago
  Because the visitor pattern is just pattern matching in OO clothing.
wrycoder 6y ago
nand2tetris teaches compilers backwards. For that matter, they teach the whole computer backwards. Students start with NAND gates and implement a computer in a hardware definition language. They then implement an assembler for that hardware. The next step is to implement a virtual stack machine, written in that assembler. Then the students implement a compiler for an object-oriented language similar to Java. The target for the compiler is the stack machine language. This is done in two passes: lexing and a PEG compiler. Finally, a simple OS is implemented. In most courses, if you copy back what has been covered, you will have no problem completing the class. But in nand2tetris, some of the key material is omitted from each stage. You have to figure it out yourself. https://www.nand2tetris.org/ https://www.nand2tetris.org/ https://www.coursera.org/learn/build-a-computer https://www.coursera.org/learn/build-a-computer
ajfapoiawejf 6y ago
CMU's compiler's course teaches neither "forward" nor "backward". It teaches in increasing complexity. You start by writing the whole pipeline for straight line code. Then you add branches, and structs, and functions, in succession. This bypasses the whole "which direction?" debate. https://www.cs.cmu.edu/afs/cs/academic/class/15411-f20/www/assignments.html https://www.cs.cmu.edu/afs/cs/academic/class/15411-f20/www/a...
- lifthrasiir 6y ago
  Yes, very much yes! Compiler pipeline is highly decomposable and can be taught in pretty much any order. Which one to teach first is really the matter of importance (like, I learned physics by starting with dynamics and as a result I have a good understanding of dynamics but not much for subsequent topics like thermodynamics, optics or electomagnetism...). I do agree the importance of parsing is currently overstated, but its solution has to be topic-agnostic as you can't build compilers only with parsing or codegen.
pitterpatter 6y ago
I want to see new compiler paradigms. Most compilers end up working the same way but is there room to innovate? I know the Rust compiler is kinda moving away from "passes" to "queries".
- tikhonj 6y ago
  Program synthesis seems like the most promising area of research for that—treat compilation as a search problem or even a series of search problems rather than as a pipeline of (mostly) deterministic steps. Couple a compiler with search-based optimization, some kind of lightweight verification and a more interactive interface with the programmer and you have a programming environment that can aggressively use today's powerful compute resources to provide a qualitatively different experience.

271 points by azhenley 6y ago

The bottom up order is the right one. The classic paper: An Incremental Approach to Compiler Construction (Abdulaziz Ghuloum) https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&ved=2ahUKEwjC2Kjrr4DvAhUTHOwKHXg1C4sQFjACegQIBRAD&url=http%3A%2F%2Fwww.schemeworkshop.org%2F2006%2F11-ghuloum.pdf&usg=AOvVaw1i0Fyy4oDI8CRVKTGicR7n https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&c...

Mathnerd314 6y ago
They link this paper in the bottom of the article as a different approach from the one they're using.
tom_mellior 6y ago
That's not really the same approach. It's not "backward", it's more "sideways": It starts with an end-to-end compiler for a tiny language, then builds out that end-to-end compiler incrementally. The featured article describes an approach where no such end-to-end compiler is available from the beginning. It's only end-to-end after the final exercise.

agnokapathetic 6y ago

This approach has the added benefit of teaching "mechanical sympathy". We're often told "Let the complier do the work" without really understanding what "work" the compiler is doing. Learning concepts like branch prediction, out-of-order execution, pipelining, register planning, caches, memory management is far far more important than Lexing/Parsing/Intermediate Representations and SSA.

deleted 6y ago
[deleted]
jfoutz 6y ago
I have a different take. Compilers are usually the first really big project students encounter. There's a lot of complexity around keeping things organized and still efficient. There's a lot of real world hassle around both ends - the input can be anything, and the implementer has to deal with that. The output is subtle with lots of gnarly little details that have to be exactly right. Compilers also have this wonderful side effect of eliminating all the magic. after a compiler class, there aren't many mysteries left about how computers work. they take a string and do stuff. Personally, I haven't gotten much milage out of shift reduce parsing, but parsing in general has been super important. Intermediate representations, coming from complexity to something canonical, and then transforming that back to complexity has probably given me the biggest bang for my time. If you squint your eyes, frameworks and dsls are kinda just intermediate representations. Sticking the parts together in a coherent way helped me. I've never been in a situation where out-of-order execution really mattered to my code. but my compiler course taught me enough to know valgrind, so I'd go look for whatever the modern equivalent is. Everybody is different, but compilers for me is where everything really jelled. There is so much complexity you just eat, day after day. Memorization only got me so far, at some point, I had to have real organization and clarity. I think, if I had to give a compiler class, I'd start in the middle. tools for manipulating trees, tools to build that tree up, tools to tear that tree down. I know it's kind of a breezy approach, but the clarity seems like a prerequisite for dealing with complexity. the tree is the one thing the implementer owns, and is free to structure (and restructure) however they wish.
908B64B197 6y ago
These belong in a separate class about low level systems (CPU architecture and assembly).
konjin 6y ago
>Learning concepts like branch prediction, out-of-order execution, pipelining, register planning, caches, memory management is far far more important than Lexing/Parsing/Intermediate Representations and SSA. These are incidents of technology. If this were 1970s you'd be talking about in place sorting because we're stuck with tapes for anything past a toy compiler. Something that would be useless in a decade. The point of education is to teach you how to learn, not how to collect stamps.

dominicjj 6y ago

This is pretty cool. I'm currently teaching myself compiler theory so I can write a toy compiler for a C-like language targeting x86-64 and I've also started at the bottom: re-acquiring my assembly skills so I can then map common idioms back to some sort of IR. The disadvantage is you can't perform high level optimizations that you could do when transforming from SSA to the IR but that is a long way away for now.

smasher164 6y ago

Much to the chagrin of a lot of educators, I think this approach is the way to go. Too many compilers classes get bogged down in grammar classifications and parsing. Some argue that parsing is a microcosm of the rest of the compiler, since it requires one to transform a program from one representation to another. However, this message doesn't really come across when you're operating on highly unstructured input, and not doing any simplification passes. I think execution is really important for visualizing the effects of your work.

pjmlp 6y ago
Agreed, we did it all the way, but that was just in the course of a 5 years degree there was room to spend three semesters having compiler design related lectures, split between parser theory, denotational semantics, programming language history and implementing our own toy compiler.
JoelMcCracken 6y ago
This is what happened in my compilers class. We spent, like, the whole thing talking about grammars etc. Which is mind-numbingly boring (to me), and especially given a significant number of industry parsers are hand-rolled/recursive descent style, its not terribly useful information.
- JoelMcCracken 6y ago
  At the moment, some of us in the PGH FP group are working through https://www.cs.cmu.edu/~rwh/pfpl/ https://www.cs.cmu.edu/~rwh/pfpl/, and so far I vastly prefer it. It is math heavy, which I like for various personal reasons which are irrelevant here, but it also skips parsing!
- ahartmetz 6y ago
  Same here, at FU Berlin. We spent several weeks on LR and LALR parsers and such and, IIRC, even more than a week on regular expressions. Somehow we talked about two-address and three-address instructions, but never seemed to do any code generation, and certainly no optimization. It was very disappointing.

mamcx 6y ago

Interesting, this is how I actually make progres building my own lang in the side (https://tablam.org https://tablam.org). I discover "early" that parsing will be at mercy of each change in direction on the internals, making a costing rewrite each time. Is like premature, aimless, unit-testing, where the cost of testing dominate the actual code to write. So, eventually I split my efforts in 2: On the side I just write programs in the imaginary syntax and change it even in each line. But the whole action was from AST->Execution. I don't bother to parsing until very late in the game, and I think I could have deferred even more. This is not because I believe the syntax "don't matter", I think the opposite!, but you can't know which syntax its the best until you nail the semantics. If the parsing/syntax is introduced too early you could going backwards to support THAT syntax bolted on the semantics, instead of left the semantics guide you. ie: Is like UX/UI: If the program is not well defined, the UI will distort it.

travisgriggs 6y ago

Good it be loosely posited, that they're teaching decompilation in the forward sense?

p4bl0 6y ago

The idea is so simple it seems obvious once it has been told, which the sign of a fantastic idea. It's too late for this year but I will try this approach next year in my compilers course!

harperlee 6y ago

In a similar vein, networking is typically taught bottom-up, but some people swear by the backwards top-down approach: https://www.goodreads.com/book/show/83847.Computer_Networking https://www.goodreads.com/book/show/83847.Computer_Networkin...

renox 6y ago
Yep, I had two classes about network the first was 'top to bottom' and let me really confused, the second was 'bottom to top' and I understood the whole lesson without difficulty.. That said, it could be me: I was never able to get inheritance until I learned how it was implemented..

nahuel0x 6y ago

This direction is also analogous to the historical development of programming languages, from bytecodes to textual assembly instructions to (something like) macro-assembler.

Mathnerd314 6y ago

They call it backward or bottom-up but really it's middle-out: they start with valid AST, learn how to compile it out, then add a frontend that generates ASTs.

waynesonfire 6y ago

Can anyone recommend resources that teach compilers in reverse?

CalChris 6y ago
https://github.com/aalhour/awesome-compilers https://github.com/aalhour/awesome-compilers

hardwaregeek 6y ago

I learned how to compile to WebAssembly by getting nice and intimate with the WebAssembly spec, learning the ins and outs, then writing a basic emitter for the bytecode, then writing a code generator for basic operations like arithmetic, then extending it to locals, then first class functions, etc. So kind of backwards. I did start with a parser and typechecker, but if anything that made it too difficult. I had all of these extremely high level concepts that I struggled to map to WebAssembly (pattern matching, closures, etc.) and discouraged me in the process. If I had started with WebAssembly and moved to my high level language, I'd probably have restricted myself to a much more practical set of language features. On the flip side, my language would probably be influenced by how easy it is to code generate to WASM, which isn't necessarily what I want.

carapace 6y ago

Yeah, FWIW, I think you should start at the bottom[1] with bits, bytes, RAM, and a wee CPU (like Wirth's RISC for Project Oberon[2]) and then META-II [3] seems like the best way forward to conventional languages. (Or go with Forth and Joy and head right past Lambda Calculus and straight to Category Theory[4].) On the other hand you could start with predicate logic and teach Prolog as a calculator for same, then develop all the low- and mid-level software (your emulator and compiler) using Prolog, then implement a Prolog interpreter in your language to close the loop. [1] http://phoenixbureau.github.io/PigeonComputer/curiculum.html http://phoenixbureau.github.io/PigeonComputer/curiculum.html [2] https://pythonoberon.readthedocs.io/en/latest/ https://pythonoberon.readthedocs.io/en/latest/ [3] https://en.wikipedia.org/wiki/META_II https://en.wikipedia.org/wiki/META_II [3] http://conal.net/papers/compiling-to-categories/ http://conal.net/papers/compiling-to-categories/

orthoxerox 6y ago

A project that was all about this approach: https://speakerdeck.com/nineties/creating-a-language-using-only-assembly-language https://speakerdeck.com/nineties/creating-a-language-using-o...

dominicjj 6y ago
That's an inspiring slide deck - thanks for sharing.

recursivedoubts 6y ago

I prefer the traditional lexer -> parser -> optimization -> code generator path. If I were going to change anything about teaching compilers, it would be to teach recursive descent, which makes the recursive nature of grammars a lot more obvious than parser generator approaches do. This is a great book/website a contributor to Dart, which explains the algorithm: https://craftinginterpreters.com/ https://craftinginterpreters.com/

thomaslkjeldsen 6y ago
Elaborating from https://en.wikipedia.org/wiki/Dart_(programming_language) https://en.wikipedia.org/wiki/Dart_(programming_language) > Dart was unveiled at the GOTO conference in Aarhus, Denmark, October 10–12, 2011. The project was founded by Lars Bak and Kasper Lund.

nickysielicki 6y ago

Disagree. I think by the time that most students get to the point of taking a compilers course, they can understand (at a high level) what each of these stages is doing with no motivation needed. When I took my compilers course, we went over the stages of a compiler in the first 10 minutes of the first lecture, and I think everyone "got it" right away.

deleted 6y ago

[deleted]

jl2718 6y ago

I was unaware of, and still confused by the other way, but now I understand all the bad ideas people come into practice with regarding compilers.

tester756 6y ago

>implementing passes over the AST using visitors, Why Visitor Pattern is so popular around ASTs? I always feel like this particular patterns adds too much boilerplate and is not really needed in "modern" languages.

the-smug-one 6y ago
Because the visitor pattern is just pattern matching in OO clothing.

wrycoder 6y ago

nand2tetris teaches compilers backwards. For that matter, they teach the whole computer backwards. Students start with NAND gates and implement a computer in a hardware definition language. They then implement an assembler for that hardware. The next step is to implement a virtual stack machine, written in that assembler. Then the students implement a compiler for an object-oriented language similar to Java. The target for the compiler is the stack machine language. This is done in two passes: lexing and a PEG compiler. Finally, a simple OS is implemented. In most courses, if you copy back what has been covered, you will have no problem completing the class. But in nand2tetris, some of the key material is omitted from each stage. You have to figure it out yourself. https://www.nand2tetris.org/ https://www.nand2tetris.org/ https://www.coursera.org/learn/build-a-computer https://www.coursera.org/learn/build-a-computer

ajfapoiawejf 6y ago

CMU's compiler's course teaches neither "forward" nor "backward". It teaches in increasing complexity. You start by writing the whole pipeline for straight line code. Then you add branches, and structs, and functions, in succession. This bypasses the whole "which direction?" debate. https://www.cs.cmu.edu/afs/cs/academic/class/15411-f20/www/assignments.html https://www.cs.cmu.edu/afs/cs/academic/class/15411-f20/www/a...

lifthrasiir 6y ago
Yes, very much yes! Compiler pipeline is highly decomposable and can be taught in pretty much any order. Which one to teach first is really the matter of importance (like, I learned physics by starting with dynamics and as a result I have a good understanding of dynamics but not much for subsequent topics like thermodynamics, optics or electomagnetism...). I do agree the importance of parsing is currently overstated, but its solution has to be topic-agnostic as you can't build compilers only with parsing or codegen.

pitterpatter 6y ago

I want to see new compiler paradigms. Most compilers end up working the same way but is there room to innovate? I know the Rust compiler is kinda moving away from "passes" to "queries".

tikhonj 6y ago
Program synthesis seems like the most promising area of research for that—treat compilation as a search problem or even a series of search problems rather than as a pipeline of (mostly) deterministic steps. Couple a compiler with search-based optimization, some kind of lightweight verification and a more interactive interface with the programmer and you have a programming environment that can aggressively use today's powerful compute resources to provide a qualitatively different experience.