4 ms·

Should a language be designed with these things in mind? Or is it completely orthogonal?

eptcyka 3y ago
Designing syntax and semantics that are easier to parse can also make the language easier to understand to humans, as long as optimizing for these goals doesn't impede on useful language features.
matklad 3y ago
Descriptively, yes, the shape of the language determines how you can structure an interactive compiler. There are at least three wildly different ways to do that: https://rust-analyzer.github.io/blog/2020/07/20/three-architectures-for-responsive-ide.html https://rust-analyzer.github.io/blog/2020/07/20/three-archit... Which one can work depends on the language in question. Java, Rust, C++ lead to different answers, and that’s not because of the “interesting” differences like borrowchecker vs gc, but rather due to the “boring” differences in name resolution and module system. Prescriptively, whether we _should_ do this (or, rather, how much), is unclear. My biased answer is that, while language designers talk a lot about making languages tooling friendly, there’s little of that actually happening (at least with “current” languages, “next” ones seem to fare better). Like, it was said that “rust macros are designed with tooling friendliness in mind”, but overall the language is pretty tooling-hostile, mostly for accidental reasons. It seems to me that if we _actually_ co-design a tooling-first language, without trying to innovate too much, but by just ensuring that existing techniques work robustly, we might arrive at a close, but meaningfully different point in the design space. Specifically, here’s my version of IDE-friendliness diff for language design: Push conditional compilation far further in the pipeline, such that it is done after all semantic analysis. This is a pre-requisite for making automated refactors which are _guaranteed_ to work. Similarly, push meta programming further down, such that code analysis doesn’t invoke user-defined code (which might be arbitrary slow), but, at the same time, meta parts can fully reflect on existing code, including resolved types. C#-style source generators are an interesting design here. Have strong signatures on micro and macro level. Macro, have well-defined compilation units with explicitly specified dag of dependencies and signature files. Micro, annotate types of the functions. Use tooling to reduce double-annotation burden. Ensure that each source file can be somewhat deeply analyzed in complete isolation. The last two points should unlock both embarrassingly parallel (distributed) compilation, and snappy completion.
- avgcorrection 3y ago
  Thanks for that. It seems that some newer languages (like in the last ten years) either care about (1) batch compilation speed or (2) language design mostly unconstrained from having to have fast batch compilation. It would be interesting to see a third way: language design that takes into account all kinds of future interactive tooling by laying a good groundwork. So maybe from-scratch batch compilation ends up slow, but then it turns out to be fine in practice since you can distribute builds and cache things so that you only pay at most a one-time large cost. And then what you mostly have to deal with is incremental stuff.
  - firstlink 3y ago
    Interactive proof languages already work that way, to a greater or lesser extent. But, for general-purpose languages, this revictimizes the docker addicts.
- the_mitsuhiko 3y ago
  > there’s little of that actually happening (at least with “current” languages, “next” ones seem to fare better). I'm not too involved but I think the languages that actually develop the tooling as part of the language design have traditionally done quite well at it in language design. Delphi/C#/TypeScript all had tooling built alongside the language and it shows. Rust sadly made a ton of choices in the language that just make a lot of things hard, but enabled the type of ecosystem that exists today. So it has different tradeoffs. I'm quite amazed how well RLS is doing considering how hostile the language is to fast iteration.
  - valenterry 3y ago
    Look at Kotlin to see the opposite. It was designed to be IDE friendly - in fact, it was developed by the IntelliJ folks. However, looking at it now, it seems that Kotlin is pretty much falling behind even Java in terms of features and functionality. But especially it also seems behind in a well-thought design. When looking at Rust, I think while there are still a lot of features that I would desire, it at least introduced major improvements like typeclasses (or "traits" in Rust lingo) which really help the ecosystem and libraries to flourish. If this makes it harder for the compiler and the IDE then I think it's a fair tradeoff. Kotlin had the chance to do the same, they deliberately didn't and I is already starting to hinder the language from thriving.
levodelellis 3y ago
I wrote my (prototype) LSP in a completely different way from the article and I got very good results (autocomplete took less than 10ms, IIRC was <5). I didn't intentionally design my language for it but I did have to write my compiler in a way that would keep the data around so I can support it

Designing syntax and semantics that are easier to parse can also make the language easier to understand to humans, as long as optimizing for these goals doesn't impede on useful language features.

matklad 3y ago

Descriptively, yes, the shape of the language determines how you can structure an interactive compiler. There are at least three wildly different ways to do that: https://rust-analyzer.github.io/blog/2020/07/20/three-architectures-for-responsive-ide.html https://rust-analyzer.github.io/blog/2020/07/20/three-archit... Which one can work depends on the language in question. Java, Rust, C++ lead to different answers, and that’s not because of the “interesting” differences like borrowchecker vs gc, but rather due to the “boring” differences in name resolution and module system. Prescriptively, whether we _should_ do this (or, rather, how much), is unclear. My biased answer is that, while language designers talk a lot about making languages tooling friendly, there’s little of that actually happening (at least with “current” languages, “next” ones seem to fare better). Like, it was said that “rust macros are designed with tooling friendliness in mind”, but overall the language is pretty tooling-hostile, mostly for accidental reasons. It seems to me that if we _actually_ co-design a tooling-first language, without trying to innovate too much, but by just ensuring that existing techniques work robustly, we might arrive at a close, but meaningfully different point in the design space. Specifically, here’s my version of IDE-friendliness diff for language design: Push conditional compilation far further in the pipeline, such that it is done after all semantic analysis. This is a pre-requisite for making automated refactors which are _guaranteed_ to work. Similarly, push meta programming further down, such that code analysis doesn’t invoke user-defined code (which might be arbitrary slow), but, at the same time, meta parts can fully reflect on existing code, including resolved types. C#-style source generators are an interesting design here. Have strong signatures on micro and macro level. Macro, have well-defined compilation units with explicitly specified dag of dependencies and signature files. Micro, annotate types of the functions. Use tooling to reduce double-annotation burden. Ensure that each source file can be somewhat deeply analyzed in complete isolation. The last two points should unlock both embarrassingly parallel (distributed) compilation, and snappy completion.

avgcorrection 3y ago
Thanks for that. It seems that some newer languages (like in the last ten years) either care about (1) batch compilation speed or (2) language design mostly unconstrained from having to have fast batch compilation. It would be interesting to see a third way: language design that takes into account all kinds of future interactive tooling by laying a good groundwork. So maybe from-scratch batch compilation ends up slow, but then it turns out to be fine in practice since you can distribute builds and cache things so that you only pay at most a one-time large cost. And then what you mostly have to deal with is incremental stuff.
- firstlink 3y ago
  Interactive proof languages already work that way, to a greater or lesser extent. But, for general-purpose languages, this revictimizes the docker addicts.
the_mitsuhiko 3y ago
> there’s little of that actually happening (at least with “current” languages, “next” ones seem to fare better). I'm not too involved but I think the languages that actually develop the tooling as part of the language design have traditionally done quite well at it in language design. Delphi/C#/TypeScript all had tooling built alongside the language and it shows. Rust sadly made a ton of choices in the language that just make a lot of things hard, but enabled the type of ecosystem that exists today. So it has different tradeoffs. I'm quite amazed how well RLS is doing considering how hostile the language is to fast iteration.
- valenterry 3y ago
  Look at Kotlin to see the opposite. It was designed to be IDE friendly - in fact, it was developed by the IntelliJ folks. However, looking at it now, it seems that Kotlin is pretty much falling behind even Java in terms of features and functionality. But especially it also seems behind in a well-thought design. When looking at Rust, I think while there are still a lot of features that I would desire, it at least introduced major improvements like typeclasses (or "traits" in Rust lingo) which really help the ecosystem and libraries to flourish. If this makes it harder for the compiler and the IDE then I think it's a fair tradeoff. Kotlin had the chance to do the same, they deliberately didn't and I is already starting to hinder the language from thriving.

levodelellis 3y ago

I wrote my (prototype) LSP in a completely different way from the article and I got very good results (autocomplete took less than 10ms, IIRC was <5). I didn't intentionally design my language for it but I did have to write my compiler in a way that would keep the data around so I can support it