3 ms·

A lot of ink is spilled on the details of how to make compilers go from taking text and turn it into fast executables, but I believe that the User Experience of

A lot of ink is spilled on the details of how to make compilers go from taking text and turn it into fast executables, but I believe that the User Experience of compilers is as important a subject as those, but there's very little information about it in the literature, outside of a handful of papers and the design documents of existing compilers that do focus on this.

dmitriid 6y ago
No idea why you are being downvoted, but it's true. Compiler UX is often horrible, but the new breed of languages tries to rectify parts of it: - non-crpytic error messages. Show exactly what failed, where it failed, how to solve it. - compiler-as-a-service. Tool makers will love you. - compilation speed. If I need to wait 2 minutes for a 10 LOC project to compile, your compiler is doing something wrong. - the compiler should be the only tool you need. If you rely on 15 different tools just to compile a project, you're doing something wrong
- estebank 6y ago
  As you allude to, treating a compiler as an isolated batch process is an outdated strategy. Any compiler started today should be designed such that it can be used in an IDE from the start, even if you don't implement it in that way until after you've reached a certain level of maturity. For error messages in particular is a very expansive topic because a compiler might have the following components: Lexer Parser Macro Expansion Name Resolution Lowering Type Checking Privacy Checking Lifetime Analysis Code Generation Linking and all of them can produce errors. If you introduce error recovery in your parser, then you better make sure you're carrying that information forward to name resolution and type checking. If you introduce a error type placeholder, better account for it during the rest of your analysis to avoid complaining about non-existing methods on something that wasn't resolved in the first place. Error deduplication. Exploring the grammar space to identify things that a human may attempt but isn't allowed for good reason but that will not blow up until some quasi-random stage. Now that I think about it you could write a whole book about these topic.
  - tomcam 6y ago
    What is lowering?
    - hctaw 6y ago
      Some languages require higher level internal representations ("IR"s) to be "lowered" to lower level representations before code generation - in other words like high level semantics or syntactic sugar and transform into simpler constructs to compile. For example, a language could support special iterator semantics that will be "lowered" into a simple for loop. Or a for loop itself may be lowered into a do-while loop, depending on the design of the lowered IR.
      - estebank 6y ago
        To give you an example, rustc has an AST, and a high-level intermediate representation tree called the HIR. The former can encode anything that is syntactically valid (and some things that aren't), while the HIR only encodes things that are valid and transforms things like `if foo {} else {}` to `match foo { true => {} false => {}}`. That makes this internal representation more fundamental, there are fewer constructs to care about, and allows unifying some validations that would otherwise have to be written for all the alternative user visible constructs. It also makes later stages of the compiler not have to care about how code was written, just about this higher level representation.
- turminal 6y ago
  > - non-crpytic error messages. Show exactly what failed, where it failed, how to solve it. I'm working on a compiler in my free time and I agree with you, but in defense of compilers that suck at this: error reporting is really, really hard to do right.
- hctaw 6y ago
  I downvoted it because I think it's a meme that ignores where contemporary compiler engineering is happening. We don't even realize they exist anymore - V8, Graal, Roslyn, etc are all optimizing JIT compilers running in the background that can compile multiple languages on the fly. And they do a great job. Not everything is a C++ compiler, which is inherently difficult to compile correctly. Let alone quickly. I think the future of compilers is that everything will have an incremental parser and JIT compiler with optional AOT compilation for release builds. Hopefully LSP and DAP will take over the world of tooling integration. There are other fundamental mistakes to look at than error messages and performance, like pretending that build systems don't exist and that linking multiple compilation units isn't something the compiler can do. Compiling code is getting much easier, but building it is much harder. Compilers have the most information and internal structure to handle all our build problems, but don't do it.

dmitriid 6y ago

No idea why you are being downvoted, but it's true. Compiler UX is often horrible, but the new breed of languages tries to rectify parts of it: - non-crpytic error messages. Show exactly what failed, where it failed, how to solve it. - compiler-as-a-service. Tool makers will love you. - compilation speed. If I need to wait 2 minutes for a 10 LOC project to compile, your compiler is doing something wrong. - the compiler should be the only tool you need. If you rely on 15 different tools just to compile a project, you're doing something wrong

estebank 6y ago
As you allude to, treating a compiler as an isolated batch process is an outdated strategy. Any compiler started today should be designed such that it can be used in an IDE from the start, even if you don't implement it in that way until after you've reached a certain level of maturity. For error messages in particular is a very expansive topic because a compiler might have the following components: Lexer Parser Macro Expansion Name Resolution Lowering Type Checking Privacy Checking Lifetime Analysis Code Generation Linking and all of them can produce errors. If you introduce error recovery in your parser, then you better make sure you're carrying that information forward to name resolution and type checking. If you introduce a error type placeholder, better account for it during the rest of your analysis to avoid complaining about non-existing methods on something that wasn't resolved in the first place. Error deduplication. Exploring the grammar space to identify things that a human may attempt but isn't allowed for good reason but that will not blow up until some quasi-random stage. Now that I think about it you could write a whole book about these topic.
- tomcam 6y ago
  What is lowering?
  - hctaw 6y ago
    Some languages require higher level internal representations ("IR"s) to be "lowered" to lower level representations before code generation - in other words like high level semantics or syntactic sugar and transform into simpler constructs to compile. For example, a language could support special iterator semantics that will be "lowered" into a simple for loop. Or a for loop itself may be lowered into a do-while loop, depending on the design of the lowered IR.
    - estebank 6y ago
      To give you an example, rustc has an AST, and a high-level intermediate representation tree called the HIR. The former can encode anything that is syntactically valid (and some things that aren't), while the HIR only encodes things that are valid and transforms things like `if foo {} else {}` to `match foo { true => {} false => {}}`. That makes this internal representation more fundamental, there are fewer constructs to care about, and allows unifying some validations that would otherwise have to be written for all the alternative user visible constructs. It also makes later stages of the compiler not have to care about how code was written, just about this higher level representation.
turminal 6y ago
> - non-crpytic error messages. Show exactly what failed, where it failed, how to solve it. I'm working on a compiler in my free time and I agree with you, but in defense of compilers that suck at this: error reporting is really, really hard to do right.
hctaw 6y ago
I downvoted it because I think it's a meme that ignores where contemporary compiler engineering is happening. We don't even realize they exist anymore - V8, Graal, Roslyn, etc are all optimizing JIT compilers running in the background that can compile multiple languages on the fly. And they do a great job. Not everything is a C++ compiler, which is inherently difficult to compile correctly. Let alone quickly. I think the future of compilers is that everything will have an incremental parser and JIT compiler with optional AOT compilation for release builds. Hopefully LSP and DAP will take over the world of tooling integration. There are other fundamental mistakes to look at than error messages and performance, like pretending that build systems don't exist and that linking multiple compilation units isn't something the compiler can do. Compiling code is getting much easier, but building it is much harder. Compilers have the most information and internal structure to handle all our build problems, but don't do it.