6 ms·

Anders Hejlsberg on Modern Compiler Construction (2016) [video]

187 points by ehaskins 7y ago

pbiggar 7y ago
This is kinda how the compiler-stuff in Dark is written. Everything - the editor, semantic analysis, version control, execution engine, everything - all use the same data structures (the same abstract syntax tree). We use functional data structure everywhere and we do functional updates within the AST all the time; that's even how text entry in the editor updates the program.
- chrisseaton 7y ago
  > Everything - the editor, semantic analysis, version control, execution engine, everything - all use the same data structures (the same abstract syntax tree). Is that data structure suitable for all those purposes though? How do you do optimisations like GVN on an AS->T<-?
  - pbiggar 7y ago
    It's not suitable for all purposes, but it's suitable for all editor purposes (including an in-editor execution engine). We don't do GVN, or any optimizations really, right now - I'm sure when we have a compiler of sorts we'll have other structures, SSA, etc.
dang 7y ago
A thread from 2018: https://news.ycombinator.com/item?id=17280589 https://news.ycombinator.com/item?id=17280589 Discussed at the time: https://news.ycombinator.com/item?id=11685317 https://news.ycombinator.com/item?id=11685317
W-Stool 7y ago
Turbo Pascal (which he authored) was quite the accomplishment back in the day - editor and compiler in 64K. Think about that for a minute.
- barrkel 7y ago
  The DOS TP compiler was still at the heart of Delphi 1 supported an inline assembler and reference-based class system, code generator, etc. in less than 43k lines of assembler (it was entirely written in assembler). (32-bit Delphi 2 switched to a C-based compiler originally written by Peter Sollich. I maintained the front end on that compiler for 6 years; PS followed AH to MS.)
  - slowmovintarget 7y ago
    I got a lot of use out of that in my early career. Thank you.
    - barrkel 7y ago
      To be clear, I was only involved in 2006-12 time period. I got a lot of use out of TP 6 and Delphi 2 when I was learning, which is a significant part of the reason I ended up at Borland.
  - mika9090 7y ago
    Cool. I love Delphi to this day. Nothing beats that for Windows native apps IMHO. Any interesting stories to share? I read somewhere that now the compile is a million lines of undocumented code.
- mixmastamyk 7y ago
  ^and it was awesome.
- yesenadam 7y ago
  That was my first language after BASIC - I loved it! I may be remembering this wrong, but it seemed virtually free - it was about $100 when its competitors and most software was $500+. That was in about 1986. The only way to get a manual was to buy the software. Another world..
  - tomcam 7y ago
tybit 7y ago
This is a great talk, I was struck by how similar some of the high level design goals are to LLVM https://www.aosabook.org/en/llvm.html https://www.aosabook.org/en/llvm.html I kind of hope Anders moves on from TypeScript soon, he’s done a fantastic job there considering the constraints of JavaScript, but I’d love to see him tackle something new.
- zackbrown 7y ago
  Between Delphi, C#, and TypeScript the man has already had a storied and impactful career.
  - pjmlp 7y ago
    You forgot about the previous 10 year's of the man's career with Turbo Pascal.
    - acqq 7y ago
      Also his first work in Microsoft: Windows Foundation Classes (WFC) and Visual J++ development system.
  - noir_lord 7y ago
    My three favourite languages ever (with Turbo Pascal as my first proper programming language) - The man has quite simply defined my career.
    - deleted 7y ago
      [deleted]
  - tybit 7y ago
    Yes, he’s fantastically accomplished and capable, and I think he could do amazing things in a new area. I’d love to see him take lessons learned from C# for a language designed for WASM from the get go.
    - 7y ago
bordercases 7y ago
It sounds like that content-addressed code as in the Unison language https://www.unisonweb.org/ https://www.unisonweb.org/ could end up facilitating the design of a very effective IDE since the language can naturally generate an immutable structure by default. I don't know enough to know if this hunch is correct, I'm wondering if anyone better informed can chime in.
- zubairq 7y ago
  Yes this is correct. We use content addressable code, inspired by Unison for yazz Pilot, and it makes so many things in the ide really simple, including dependency management and stuff like that
zubairq 7y ago
Very interesting, even though we don’t have a compiler in the traditional sense for yazz Pilot as a container image is our output format, many of the points are still valid. Also very interesting how rosalyn had the idea of compiler as an API, we went in the opposite direction and did not have any extensibility, instead we made the Code editor an API
octref 7y ago
One take away for me was: Language designers & compiler writers today need to consider editor support as one of their goals. On the top page there's another thread about rust-analyzer vs RLS. What Aleksey said[0] about RLS that "[RLS's] current architecture is not a good foundation for a perfect IDE long-term," feels similar to my coworker's conclusion in her effort to provide better editor support for PHP[1]. Parsers for compiling code into machine-executables vs parsers to serve LSP responses have different tradeoffs. For example, Anders mentioned TS parser need to have good error-recovery, can respond with completions/errors when one file changes inside a thousand-files-project. I vaguely remember TS had a goal to provide completions in < 50ms and errors in <150ms. Such goals are hard to achieve as after-thought. If your core compiler doesn't do error recovery, such as PHP, you need someone to write a new compiler from scratch for a language server implementation. If tools such as RLS rely on compiler to dump all JSON metadata and figure out LSP responses[0], it's too slow for editors. TS's good editor support doesn't come free. I think one of the most under-appreciated achievement of TS is how it took editor support seriously and designed its compiler infrastructure to do it well. That's why I don't believe in those hot-new-web-languages that try to take over TS by designing a better type system. TS brought the average developer's expectation of a language to having fast completion, fast error reporting, editor autofix, F2 to rename and renaming-a-file-in-editor-to-update-all-references. It's 2020 and people aren't going back to write code like in Notepad. [0]: https://ferrous-systems.com/blog/rust-analyzer-2019/ https://ferrous-systems.com/blog/rust-analyzer-2019/ [1]: https://github.com/microsoft/tolerant-php-parser https://github.com/microsoft/tolerant-php-parser --- EDIT: grammar.
- Kinrany 7y ago
  > One take away for me was: Language designers & compiler writers today need to consider editor support as one of their goals. In a way this is very intuitive: a programming language is a kind of a UI for the language's runtime. The IDE is just another UI layer on top of that.
oaiey 7y ago
At that time I wanted to read about compiler building. Missed that I'm University. That video made me skipping that. The standard books do not exist for this level of evolvement.
davidjnelson 7y ago
Something I’ve observed with languages he designs are that they end up with a ui form builder. I wonder if that will happen with typescript. It’s harder, because they’d either need to pick a framework, or design it for at least react, vue, angular.

187 points by ehaskins 7y ago

This is kinda how the compiler-stuff in Dark is written. Everything - the editor, semantic analysis, version control, execution engine, everything - all use the same data structures (the same abstract syntax tree). We use functional data structure everywhere and we do functional updates within the AST all the time; that's even how text entry in the editor updates the program.

chrisseaton 7y ago
> Everything - the editor, semantic analysis, version control, execution engine, everything - all use the same data structures (the same abstract syntax tree). Is that data structure suitable for all those purposes though? How do you do optimisations like GVN on an AS->T<-?
- pbiggar 7y ago
  It's not suitable for all purposes, but it's suitable for all editor purposes (including an in-editor execution engine). We don't do GVN, or any optimizations really, right now - I'm sure when we have a compiler of sorts we'll have other structures, SSA, etc.

dang 7y ago

A thread from 2018: https://news.ycombinator.com/item?id=17280589 https://news.ycombinator.com/item?id=17280589 Discussed at the time: https://news.ycombinator.com/item?id=11685317 https://news.ycombinator.com/item?id=11685317

W-Stool 7y ago

Turbo Pascal (which he authored) was quite the accomplishment back in the day - editor and compiler in 64K. Think about that for a minute.

barrkel 7y ago
The DOS TP compiler was still at the heart of Delphi 1 supported an inline assembler and reference-based class system, code generator, etc. in less than 43k lines of assembler (it was entirely written in assembler). (32-bit Delphi 2 switched to a C-based compiler originally written by Peter Sollich. I maintained the front end on that compiler for 6 years; PS followed AH to MS.)
- slowmovintarget 7y ago
  I got a lot of use out of that in my early career. Thank you.
  - barrkel 7y ago
    To be clear, I was only involved in 2006-12 time period. I got a lot of use out of TP 6 and Delphi 2 when I was learning, which is a significant part of the reason I ended up at Borland.
- mika9090 7y ago
  Cool. I love Delphi to this day. Nothing beats that for Windows native apps IMHO. Any interesting stories to share? I read somewhere that now the compile is a million lines of undocumented code.
mixmastamyk 7y ago
^and it was awesome.
yesenadam 7y ago
That was my first language after BASIC - I loved it! I may be remembering this wrong, but it seemed virtually free - it was about $100 when its competitors and most software was $500+. That was in about 1986. The only way to get a manual was to buy the software. Another world..
- tomcam 7y ago

tybit 7y ago

This is a great talk, I was struck by how similar some of the high level design goals are to LLVM https://www.aosabook.org/en/llvm.html https://www.aosabook.org/en/llvm.html I kind of hope Anders moves on from TypeScript soon, he’s done a fantastic job there considering the constraints of JavaScript, but I’d love to see him tackle something new.

zackbrown 7y ago
Between Delphi, C#, and TypeScript the man has already had a storied and impactful career.
- pjmlp 7y ago
  You forgot about the previous 10 year's of the man's career with Turbo Pascal.
  - acqq 7y ago
    Also his first work in Microsoft: Windows Foundation Classes (WFC) and Visual J++ development system.
- noir_lord 7y ago
  My three favourite languages ever (with Turbo Pascal as my first proper programming language) - The man has quite simply defined my career.
  - deleted 7y ago
    [deleted]
- tybit 7y ago
  Yes, he’s fantastically accomplished and capable, and I think he could do amazing things in a new area. I’d love to see him take lessons learned from C# for a language designed for WASM from the get go.
  - 7y ago

bordercases 7y ago

It sounds like that content-addressed code as in the Unison language https://www.unisonweb.org/ https://www.unisonweb.org/ could end up facilitating the design of a very effective IDE since the language can naturally generate an immutable structure by default. I don't know enough to know if this hunch is correct, I'm wondering if anyone better informed can chime in.

zubairq 7y ago
Yes this is correct. We use content addressable code, inspired by Unison for yazz Pilot, and it makes so many things in the ide really simple, including dependency management and stuff like that

zubairq 7y ago

Very interesting, even though we don’t have a compiler in the traditional sense for yazz Pilot as a container image is our output format, many of the points are still valid. Also very interesting how rosalyn had the idea of compiler as an API, we went in the opposite direction and did not have any extensibility, instead we made the Code editor an API

octref 7y ago

One take away for me was: Language designers & compiler writers today need to consider editor support as one of their goals. On the top page there's another thread about rust-analyzer vs RLS. What Aleksey said[0] about RLS that "[RLS's] current architecture is not a good foundation for a perfect IDE long-term," feels similar to my coworker's conclusion in her effort to provide better editor support for PHP[1]. Parsers for compiling code into machine-executables vs parsers to serve LSP responses have different tradeoffs. For example, Anders mentioned TS parser need to have good error-recovery, can respond with completions/errors when one file changes inside a thousand-files-project. I vaguely remember TS had a goal to provide completions in < 50ms and errors in <150ms. Such goals are hard to achieve as after-thought. If your core compiler doesn't do error recovery, such as PHP, you need someone to write a new compiler from scratch for a language server implementation. If tools such as RLS rely on compiler to dump all JSON metadata and figure out LSP responses[0], it's too slow for editors. TS's good editor support doesn't come free. I think one of the most under-appreciated achievement of TS is how it took editor support seriously and designed its compiler infrastructure to do it well. That's why I don't believe in those hot-new-web-languages that try to take over TS by designing a better type system. TS brought the average developer's expectation of a language to having fast completion, fast error reporting, editor autofix, F2 to rename and renaming-a-file-in-editor-to-update-all-references. It's 2020 and people aren't going back to write code like in Notepad. [0]: https://ferrous-systems.com/blog/rust-analyzer-2019/ https://ferrous-systems.com/blog/rust-analyzer-2019/ [1]: https://github.com/microsoft/tolerant-php-parser https://github.com/microsoft/tolerant-php-parser --- EDIT: grammar.

Kinrany 7y ago
> One take away for me was: Language designers & compiler writers today need to consider editor support as one of their goals. In a way this is very intuitive: a programming language is a kind of a UI for the language's runtime. The IDE is just another UI layer on top of that.

oaiey 7y ago

At that time I wanted to read about compiler building. Missed that I'm University. That video made me skipping that. The standard books do not exist for this level of evolvement.

davidjnelson 7y ago

Something I’ve observed with languages he designs are that they end up with a ui form builder. I wonder if that will happen with typescript. It’s harder, because they’d either need to pick a framework, or design it for at least react, vue, angular.