2 ms·

As someone who knows very little about compiler implementation but has some interest in it, I'm curious what makes the difference between the two in terms of ar

by _bxg1 7y ago

As someone who knows very little about compiler implementation but has some interest in it, I'm curious what makes the difference between the two in terms of architecture

pjmlp 7y ago
Basically the IDE one needs to take into account that your program is broken all the time, yet you want code completion for everything else that is actually correct. Also it needs to respond immediately after asking for completion, as anything beyond 2s is frustrating development experience. You also want to get real time errors and warnings, just for the parts that are actually broken, not a wall of text like many batch compilers that fail to understand the remaining of the file. Also you want to be able to do code refactorings, regardless of the compilation state. So basically you want a Smalltalk/Lisp Machines like experience. Anders has a nice interview about this, https://channel9.msdn.com/Blogs/Seth-Juarez/Anders-Hejlsberg-on-Modern-Compiler-Construction https://channel9.msdn.com/Blogs/Seth-Juarez/Anders-Hejlsberg...
- _bxg1 7y ago
  So would you typically maintain a separate compiler for each use-case, or try and make one that serves both?
  - steveklabnik 7y ago
    IMHO, one that serves both. The IDE case is a superset of the batch case.
estebank 7y ago
It not only informs the architecture but also the language design. For IDEs you want quick response even if it is not perfect, for a compiler you want correctness over everything else. The language having syntactic redundancy can help an IDE parser recover gracefully from a typo or missing tokens. Having semantic negative space can let an IDE recognize intent for code that looks correct from extrapolated understanding that is actually not semantically correct, and suggest solutions. rustc already tries to do all of these strategies, but blows the latency budget because it still prioritizes correctness over everything else.
dralley 7y ago
There's a decent explanation in here: https://www.infoq.com/news/2020/01/rust-analyser-ide-support/ https://www.infoq.com/news/2020/01/rust-analyser-ide-support...
oconnor663 7y ago
My understanding is that the "traditional way" is based on a number of "passes". In my head it looks like: 1) Parse all the code. 2) Assemble the set of all types. 3) Typecheck all the code. 4) Translate the parse tree into unoptimized machine code. 5) Optimize all that machine code. This is oversimplifying things, as in practice there are intermediate representations between the parse tree and the machine code. And optimization itself usually involves multiple passes of different kinds. But anyway, the key point here is that, if you change any of the code, you have to run all of these steps all over again. (I'm oversimplifying again. Maybe you only have to rerun them for a given "compilation unit", but that's bad enough.) This is the opposite of what you want for an IDE. There, you want to say "I just changed this function. Please recompile the absolute minimum necessary to tell me whether my change works." To answer a question like that efficiently, you have to rearchitect the whole compiler from being pass-based to being query-based, so you can give it instructions like "please update the type of just this expression".

by _bxg1 7y ago

As someone who knows very little about compiler implementation but has some interest in it, I'm curious what makes the difference between the two in terms of architecture

pjmlp 7y ago

Basically the IDE one needs to take into account that your program is broken all the time, yet you want code completion for everything else that is actually correct. Also it needs to respond immediately after asking for completion, as anything beyond 2s is frustrating development experience. You also want to get real time errors and warnings, just for the parts that are actually broken, not a wall of text like many batch compilers that fail to understand the remaining of the file. Also you want to be able to do code refactorings, regardless of the compilation state. So basically you want a Smalltalk/Lisp Machines like experience. Anders has a nice interview about this, https://channel9.msdn.com/Blogs/Seth-Juarez/Anders-Hejlsberg-on-Modern-Compiler-Construction https://channel9.msdn.com/Blogs/Seth-Juarez/Anders-Hejlsberg...

_bxg1 7y ago
So would you typically maintain a separate compiler for each use-case, or try and make one that serves both?
- steveklabnik 7y ago
  IMHO, one that serves both. The IDE case is a superset of the batch case.

estebank 7y ago

It not only informs the architecture but also the language design. For IDEs you want quick response even if it is not perfect, for a compiler you want correctness over everything else. The language having syntactic redundancy can help an IDE parser recover gracefully from a typo or missing tokens. Having semantic negative space can let an IDE recognize intent for code that looks correct from extrapolated understanding that is actually not semantically correct, and suggest solutions. rustc already tries to do all of these strategies, but blows the latency budget because it still prioritizes correctness over everything else.

dralley 7y ago

There's a decent explanation in here: https://www.infoq.com/news/2020/01/rust-analyser-ide-support/ https://www.infoq.com/news/2020/01/rust-analyser-ide-support...

oconnor663 7y ago

My understanding is that the "traditional way" is based on a number of "passes". In my head it looks like: 1) Parse all the code. 2) Assemble the set of all types. 3) Typecheck all the code. 4) Translate the parse tree into unoptimized machine code. 5) Optimize all that machine code. This is oversimplifying things, as in practice there are intermediate representations between the parse tree and the machine code. And optimization itself usually involves multiple passes of different kinds. But anyway, the key point here is that, if you change any of the code, you have to run all of these steps all over again. (I'm oversimplifying again. Maybe you only have to rerun them for a given "compilation unit", but that's bad enough.) This is the opposite of what you want for an IDE. There, you want to say "I just changed this function. Please recompile the absolute minimum necessary to tell me whether my change works." To answer a question like that efficiently, you have to rearchitect the whole compiler from being pass-based to being query-based, so you can give it instructions like "please update the type of just this expression".