20 ms·

I’m porting the TypeScript type checker tsc to Go

450 points by msoad 5y ago

andyfleming 5y ago
Here's a relevant GitHub thread where some conversation that has been happening: https://github.com/swc-project/swc/issues/571 https://github.com/swc-project/swc/issues/571
forty 5y ago
go means it's going to be easier to integrate to esbuild :) I'm happy to see this, as honestly I don't really understand why you would compile TS without type checking.
- eatonphil 5y ago
  > I'm happy to see this, as honestly I don't really understand why you would compile TS without type checking. I don't think I'm remotely alone in doing this but I'll share from my perspective. I have two modes while writing TypeScript stuff: 1) prototyping something where I don't want to spend time writing types (or maybe I write some types but I don't care if they are correct or line up yet) but I still want to have JavaScript generated so I can test the thing out manually and 2) getting types correct as part of long-term maintenance and edge-case removal. There are fast TypeScript -> JavaScript generators today like swc and esbuild. tsc is not fast. So I can speed up "1)" from above if I use swc and esbuild and only use tsc for "2)" from above. As people write faster implementations of tsc-the-type-checker though I'll be happy to use that instead and just not use tsc anymore.
  - m-hilgendorf 5y ago
    > 1) prototyping something where I don't want to spend time writing types It doesn't compile to JS, but this is one of the canonical use cases for Stanza 's (1) "optional" type system (1) http://lbstanza.org/optional_typing.html http://lbstanza.org/optional_typing.html
  - tshaddox 5y ago
    Out of curiosity, how often do you have a project in the first mode that gets large enough that the speed of compiling TS to JS is significant?
    - eatonphil 5y ago
      There is a huge difference even in small and mid-size projects. https://datastation.multiprocess.io/blog/2021-11-13-benchmarking-esbuild-swc-typescript-babel.html https://datastation.multiprocess.io/blog/2021-11-13-benchmar...
    - nicoburns 5y ago
      If you're writing UI code then even a few seconds is slow. You don't want to be waiting 5 seconds every time you move something a couple of pixels to the left.
  - forty 5y ago
    That makes sense, I would generally use JS directly for mode 1), but I understand the use case.
  - mixedCase 5y ago
    Have you tried doing Type-Driven Development? IME, most of the time constraints are determined through types first with the code being done later tends to replace the game of "what kind of complex and non-obvious types fit my code the most" with a "oh, the implementation is obvious" reaction. It also makes it easier to determine what behavior is worth testing and which ones are already covered by your types, if you tend to write tests last. If instead you're used to Test-Driven Development you can do types->tests->code.
    - eatonphil 5y ago
      I've spent a lot of time programming in Standard ML and yeah that's a prerequisite there. But no I don't prefer it to what I do in TypeScript.
- apazzolini 5y ago
  We use SWC with ts-node to strip types in dev for faster iteration loops and run the full tsc typechecker in CI (and optionally as a separate process on your local machine).
- postalrat 5y ago
  You compile ts without type checking when the options to do so are 10x+ faster than with type checking.
  - pjmlp 5y ago
    I really don't get this, when I save my TS files they are already compiled into JS on the fly, ready to fed into a <script />.
    - recursive 5y ago
      By what? What if they were updated from an upstream repo, and never saved from an editor? What if you change branches without an upstream repo?
      - pjmlp 5y ago
        By VS and VSCode, the IDE/editor that belongs to the same company that does TypesScript. If I really need to call tsc explicitly, the few seconds that I have to wait, versus the minutes on some other languages, won't make me lose sleep over it.
- mattkrick 5y ago
  Personally, I let vscode do typechecking on open files & have a pre-commit git hook to typecheck changed files. When it comes to starting up a development server & building the client, there's a huge cost to repeatedly typechecking the same 1000+ files. By cutting out typechecking & only compiling, I can reduce the webpack client build from 40 seconds to 3 seconds (using sucrase).
  - forty 5y ago
    I'm answering to you but this is really for most siblings: I maintain a fairly large TS code base (nodejs backend). I think a full rebuild is 1:30 on my relatively recent laptop (i7-8650U, lots of RAM). But in practice I always use tsc -w and compile is mostly instant after editing a file (I do have to wait whenever I launch the command after I boot, but after that, it's fast enough). tsc now support incremental compilation too, though I haven't played with it too much as I'm happy with watch mode.
    - rezonant 5y ago
      Incremental compilation is _okay_ but it seems to perform full rebuilds if the previous build is _too_ old which is weird- meaning that more builds than expected take the entire time to complete. Nonetheless I use it for all of my builds. On the other hand this may just be because of the spiraling nature of type inference, where TS needs to re-examine a large amount of the program from a change that appears simple. Personally I only have a small number of projects that take more than 10-20 seconds to compile, but those ones are painful. I should probably do the same with -w for those.
  - morelisp 5y ago
    What if not everyone else on your team is paying attention to their editor's warnings? What if the LS's async response doesn't come fast enough? What if, god forbid, you have to change something on a different computer? If it's not in your CI/CD scripts, it's not actually getting checked. > I can reduce the webpack client build... to 3 seconds This is about 10x longer than any interaction with a computer should be.
- bastawhiz 5y ago
  If it takes one minute to type check a codebase and ten seconds to compile it to JavaScript, then whenever I simply need to produce a JavaScript file without type checking I'll save myself the fifty seconds. If my editor's TS language server does the checking and my bundler is just compiling for my dev environment, I don't want both tools duplicating each other's work.
- forty 5y ago
  Answering to myself, a colleague suggestion (from tsc doc actually): use tsc --noEmit (for type checking) and a fast compiler. tsc --noEmit duration is 2/3 of a normal build on our fairly large code base, so that can definitely speeds things up (for CI for example)
mariogintili 5y ago
k__ 5y ago
tsc relies in shared state and is slow, so they port it to a language that allows shared state. Strange reasoning.
- munificent 5y ago
  The reasoning is only strange if you assume that the cause of slowness is the shared state. CPUs have no problems with mutability. JS is slow because of dynamic typing, dispatch, flexible object layout, and an object model that requires lots and lots of heap allocation and pointer chasing.
- Yoric 5y ago
  (er, some text in the parent seems to have vanished) Sure, if you absolutely wish it to, in an `unsafe` block, you can define two writeable pointers to the same piece of data. I don't think I've ever seen anybody write such code in Rust (except perhaps when linking to a C library that required it?), nor have I ever felt the need to do so. Now, it is true that Rust is designed to discourage writing cyclic data structures. I can imagine that trying to port (rather than redesign) an algorithm that deeply requires cyclic data structures, that will make your life miserable.
  - wizzwizz4 5y ago
    You actually can't define two `&mut` references to the same data in Rust. It's invalid Rust code; if the compiler notices, it'll refuse to compile, and if it doesn't notice, your code will randomly break.
    - Yoric 5y ago
      You can define two `*mut`, though, in unsafe code.
      - anderskaseorg 5y ago
        A better option is to use Cell or RefCell; they allow completely safe shared mutability in single-threaded code. https://doc.rust-lang.org/std/cell/ https://doc.rust-lang.org/std/cell/
adamnemecek 5y ago
The way of dealing with cycles is generational arenas. Also where is the codebase? I would like to see the uses of unsafe.
jupp0r 5y ago
> tsc depends on shared mutability and has a cyclical mutable reference. Rust is designed to prevent this behavior. Having two references to the same data (shared mutability) is undefined behavior in Rust. This problem has numerous solutions in Rust - (A)Rc, RefCell etc. I don't understand how this caused the author of the article to stop considering Rust.
- petmon 5y ago
  Non-GC languages are bad in general at handling cyclical references (mutable or not), and Rust is especially ill suited for it because of the borrow checker. For example should an arbitrary reference in tsc become an rc::Rc or an rc::Weak? You have to tease out the implicit ownership model and one may not even exist. tsc was designed expecting GC and it makes perfect sense to port it to another fast GC language.
  - throw10920 5y ago
    I'm not a Rust fanboy (in fact, I rather despise the RESF, especially given Rust's mediocre tooling), but: I think that "You have to tease out the implicit ownership model and one may not even exist" suggests a problem with the program itself. I can't think of any reason why a program must have an ill-formed ownership model (unlike, say, the fact that many interesting programs must have an embedded dynamic type system (tagged unions etc) to do their thing). Now. The Rust language and existing tooling doesn't give you any help in figuring out the ownership model, but I can easily imagine a language and tools that would help with that. This isn't meant to refute anything you've said (you said "[existing] non-GC languages are bad in general" not that non-GC languages cannot be good) - just add clarity.
    - adamnemecek 5y ago
      > I rather despise the RESF, especially given Rust's mediocre tooling Which tooling are you talking about?
      - throw10920 5y ago
pjmlp 5y ago
The thing with all those projects is playing catch-up with upstream, and the possible variations from what is actually the official semantics depending on the TypeScript version.
- throwaway894345 5y ago
  Personally I wouldn't fret about keeping pace with upstream. Eventually the syntax will stabilize if it hasn't already. I'd be more worried about fidelity--does this produce the same result as the reference implementation? TFA says he's running against the "conformance" test suite, so that should keep things pretty tight and when bugs are found they'll be fixed in time and presumably the test suite will be updated accordingly.
  - wwwigham 5y ago
    The `conformance` suite is the "set of tests people added when a feature was first released", usually without the decade of following regression tests in the `compiler` suite. In TS itself, the `conformance` and `compiler` suite are considered the part of the same unit most of the time. (The compiler suite, however, is not helpfully subdivided by feature, since it oft deals with cross-cutting concerns!)
  - munificent 5y ago
    > Eventually the syntax will stabilize if it hasn't already. Every language in wide use today is still evolving and, if anything, it seems that the pace of language change is generally increasing.
    - throwaway894345 5y ago
      Some amount of evolution is fine, and I think your “pace of language change is increasing” may be true for some languages and time scales, but most languages don’t change more in their second five years than their first.
- cellularmitosis 5y ago
londons_explore 5y ago
At first I thought this was for type checking Go code... Ya know, all that interface{} stuff...
andreypopp 5y ago
Well... How many paid work hours spent on tsc? Then does TypeScript have a formal language specification? I'm pessimistic it's possible to rewrite tsc with full compat (bug-to-bug, otherwise it doesn't really make much sense) without MS support... but who knows... I'd like fast compiles too, so good luck!
- paavohtl 5y ago
  A formal specification would be basically useless, if nothing guarantees that the reference implementation actually follows it. Sharing & contributing to the same test suite is a much more practical approach.
  - quickthrower2 5y ago
    As a smoke you can swap it in for a lot of the npm ecosystem and run the tests of those projects on the compiled code.
  - andreypopp 5y ago
    Specification isn't being followed is of course useless. But yeah, tests are definitely useless and in the absence of the spec is the only sane way to approach such rewrite.
hardwaregeek 5y ago
This reminds me of when people try to make a new Ruby implementation. They get the basics down and think "this isn't so bad! I can do this!". Fast forward a few months and they're stuck on some nasty little edge case around eval and method_missing that they need to run Rails. Or some weird aspect of C extensions. TypeScript's compiler is an unholy, unsound mess, because JavaScript is an unholy, unsound mess. Sure the basics of type checking structs and variables is somewhat easy. But what about conditional types? What about weird generic constraints? What about indexing types? There's some really confusing stuff going on underneath the hood. It's also a mess that *does not have a spec*. Even if you somehow manage to make a type checker that implements most of the features, there's no way to ensure it has 1 to 1 parity with TypeScript. Plus that parity can be broken at any point. The only way I could see this happening is if the author somehow convinced Anders Hejlsberg or a significant portion of the TypeScript team to work on this new implementation. I'm not sure that's gonna happen.
- danenania 5y ago
  If they focus on the core features and make a significant enough improvement, perhaps the core TS team could be convinced to pull it in as a native extension and make calls to it for the operations it covers? If something like 62x improvement is really achievable, it's hard to imagine the team not being interested at all. I would personally be very happy to use a significantly slimmed down subset of TypeScript if it provided a 62x performance improvement. Slow type checking is currently the main drawback of TypeScript development in large projects.
  - travisd 5y ago
    To this point, many teams are already using subsets of TypeScript to improve compilation times. Often times it’s things like always declaring return types on functions to avoid the compiler having to infer it (especially between module boundaries). Calling this a “subset of typescript” might be strong wording, but it does suggest there is some desire in that general area.
    - fault1 5y ago
danenania 5y ago
If anyone happens to know: what are tsc's bottlenecks? I presume that if the bottleneck was just number of loop iterations, V8 would already be about as fast as C or any other language? Is it simply that accessing/mutating javascript data structures like objects, arrays, set, etc. is slow? Are those not fully optimizable by V8 for some reason? Or is it something else like the GC being slow?
- deleted 5y ago
  [deleted]
- cpcallen 5y ago
  Yes, I'm curious about this too. Given that gopherjs[1] can compile Go to JavaScript and in some cases thereby achieve better performance than the native Go compiler[2], it's not obvious to me that a JavaScript program ported mechanically to Go is likely to be any faster than the original. It seems to me that the effort might be better applied to optimising the performance of tsc itself. [1] https://github.com/gopherjs/gopherjs https://github.com/gopherjs/gopherjs [2] https://medium.com/gopherjs/surprises-in-gopherjs-performance-4a0a49b04ecd https://medium.com/gopherjs/surprises-in-gopherjs-performanc...
- newlisp 5y ago
  This is a problem that can take advantage of good parallelism, Go will give you that and at the same time be more memory efficient.
konart 5y ago
>shared mutability Sure but this goes against "Don't communicate by sharing memory", no?
- scottlamb 5y ago
  > Sure but this goes against "Don't communicate by sharing memory", no? Yes, but that's advice rather than something Go requires. It's reasonable to say it's outweighed by the desire to avoid significantly restructuring the existing logic. The author is explicitly choosing to match the upstream program's structure to limit the project scope, and whether that structure is good or not is almost besides the point. btw: especially in the Rust community, shared mutability doesn't always mean shared between threads/tasks. It's also a concern in single-threaded/non-concurrent code due to aliasing. This case does actually seem to be involving multithreading given that the author mentioned "using 8 threads".
  - newlisp 5y ago
    Curious, can you elaborate on what are the issues of mutability in single-threaded code?
    - scottlamb 5y ago
      It comes down to how a language construct allocates freedom to the programmer vs the optimizing compiler. * Rust references are strict. &mut references guarantee that nothing else can access the backing memory while the &mut is valid. & references guarantee that nothing (including &mut by the rule above, or unsafe code with raw pointers by programmer's pinkie swear) can change the backing memory while the &mut is valid. This allows the compiler to perform more optimizations behind the scene, like assuming a value loaded into a register is still up-to-date, even if there's been a function call between. * Rust raw pointers make no guarantees (but require the programmer to type "unsafe" to use). * C pointers are IMHO weirder. iirc, two pointers of the same type may alias (unless you use the rare "restrict" keyword), but two pointers of different type can't, unless one is "char*". * AFAIK, Go pointers are at least as permissive as C's. They might not make any aliasing guarantees at all. * Other languages vary...
Zababa 5y ago
I see a few people wanting to rewrite tsc in something else than Typescript because tsc itself is too slow. I don't remember seeing any "proof" or analysis that Typescript is the limiting factor in tsc. The basis often used is the performance of esbuild compared to other bundlers. From the esbuild FAQ, "Why is esbuild fast?": > Several reasons: > It's written in Go and compiles to native code. > Parallelism is used heavily. > Everything in esbuild is written from scratch. > Memory is used efficienty. Using another language will solve 1 and will maybe solve 2. From this FAQ too: > For example, many bundlers use the official TypeScript compiler as a parser. But it was built to serve the goals of the TypeScript compiler team and they do not have performance as a top priority. Their code makes pretty heavy use of megamorphic object shapes and unnecessary dynamic property accesses (both well-known JavaScript speed bumps). tsc itself seems like it could be optimized. By how much, I have no idea. But it sounds like it would be easier than writing something from scratch. esbuild, while a great tool, is limited in scope. Typechecking Typescript on the other hand is a moving target, and a fast one at that. All in all, I'm not convinced by this approach compared to "regular performance work" on tsc itself. I think people talk about that because it's easier to start your project than to change tsc. In that case, the problem sound political rather than technical. There's the x62 speed from the article, but there's no info on what features are supported. I feel like this is something where the devil is in the details. I'm happy being proven wrong on that subject, I fear that we're going increase the number of tools in the frontend ecosystem for no good reasons, and make every a bit harder again.
- n0w 5y ago
  I had the same thought. A direct port of tsc to a "faster" language doesn't guarantee a faster type checker. Sure it might be a bit faster, but my guess is that the large performance wins come from a different design that takes performance into consideration from the start. Maybe the thought process is that a 1:1 rewrite will handle compatibility and feature completeness? I suppose you could optimise from there?
- x-shadowban 5y ago
  How can you avoid megamorphizing an ast hierarchy?
marcus_holmes 5y ago
Given that ESBuild [0] is a way better replacement for Webpack already (and similarly written in Go) I can see how we could pretty soon have an entire toolchain written in Go for dealing with compiling JS. Hopefully the whole toolchain will be embeddable, too. Being able to do configuration-in-code and produce an executable that does the whole build step in milliseconds would be cool. [0] https://esbuild.github.io/ https://esbuild.github.io/
- deleted 5y ago
  [deleted]
- newlisp 5y ago
  The author of esbuild also started the implementation of esbuild in rust, found it painful too and switched to build it in go.
  - pjmlp 5y ago
    For me the ideal use case for Rust is really for places where automatic memory management is a hard sell, even if technically possible, kernels, drivers, hypervisors, firmware, GPGPU, ... For everything else I rather enjoy the productivity of automatic memory management.
    - ameliaquining 5y ago
      My general inclination is to go straight to Rust if I already tried a different reasonably performant language (like Node.js) and it wasn't fast enough—because only Rust (and C and C++ and a few more obscure languages) guarantees that basically any performance optimization that I might want to do will at least be possible, even if it might be inelegant or unsafe. I don't want to run the risk of hitting another performance wall that I can't climb over. That being said, porting this kind of codebase to Rust would indeed be really rough—definitely possible, but by no means pleasant. I like to think the platonic ideal Rust, which we're hopefully getting closer and closer to over time, is capable of elegantly handling cyclic data structures and the like even while encouraging a single-ownership model for the 95% of cases where that's the right design. I don't think the Rust we have today is there yet.
slig 5y ago
Cannot wait for their webpack replacement.
scottlamb 5y ago
The "why not port to Rust" part boils down to these paragraphs: > tsc uses a lot of shared mutability and many parts depend on garbage collection. Even though I’m an advocate and believer in Rust, it doesn’t feel like the right tool for the job here. Using Rust required me to use unsafe too much for this specific project. > tsc depends on shared mutability and has a cyclical mutable reference. Rust is designed to prevent this behavior. Having two references to the same data (shared mutability) is undefined behavior in Rust. I'd be curious to hear more. In particular: * Two references to the same data: were cells too awkward? * Actual garbage collection: my first instinct would be to use an arena for each compilation, destroying the whole thing at once rather than doing reference counting or garbage collection. I gather that's not cool in the age of language servers and incremental compilation, though. Does tsc support those?
- yazaddaruvala 5y ago
  All of that is valid for a rewrite, but not efficient if you’re porting the current implementation. As I understand it, in this case, Rust would be able to have a line to line port from JavaScript. I do still think your overall point holds. Why not use a library in the Rust ecosystem to help with this problem e.g. some GC<T> type.
  - scottlamb 5y ago
    > All of that is valid for a rewrite, but not efficient if you’re porting the current implementation. Help me see it? I can easily see how getting rid of the cyclical structure crosses from "line-to-line port" into "rewrite" territory. I'm not saying you're wrong, but I'm having more trouble seeing how "wrap some types in cells and pass along a lifetime and arena parameter" crosses that threshold. It might not be feasible at all, though, if tsc compilations are long-lived for language server + incremental compilation support. > Why not use a library in the Rust ecosystem to help with this problem e.g. some GC<T> type. iirc I've seen some experimental GC library mentioned. I think something like that is at least off the beaten path enough that I can see how the author might decide it's better to just use Go instead.
antihero 5y ago
It will be interesting to see if they create a language server so we can use it in VSCode.
BadOakOx 5y ago
Just from a week ago on HN: I'm making a Typescript type-checker in Rust: https://news.ycombinator.com/item?id=30033119 https://news.ycombinator.com/item?id=30033119
- epolanski 5y ago
  Next week it will be Zig then
  - udbhavs 5y ago
    https://bun.sh/ https://bun.sh/
nevir 5y ago
Super skeptical that porting this to a new language is going to have the kind of performance gains they expect ... assuming they want to replicate all of tsc's functionality
- eyelidlessness 5y ago
  It’s already proven (esbuild, SWC, Bun) to drastically improve performance of TypeScript’s other responsibility: compilation (tsc doesn’t offer an option for this, but ts-node does, for comparison). And while the type checker is certainly more complex, I have a hard time imagining it won’t show similar improvements in a native language.
  - resoluteteeth 5y ago
    Compiling typescript is basically just stripping the type definitions/annotations and other typescript-specific parts of the syntax out. That's a huge difference from actually type checking it.
    - eyelidlessness 5y ago
      And yet, it’s drastically slower in JavaScript than in Go, Rust and Zig. I realize different algorithms will optimize differently between JS and native code, but I sincerely doubt tsc is anywhere near optimal speed.
      - lucian1900 5y ago
        That's mostly because tsc's parser is used by the JS implementations. Writing a parser for the purpose would achieve performance closer to the other implementations.
  - strager 5y ago
    esbuild, SWC, and Bun are not ports of existing programs.
davepeck 5y ago
(In a different direction, Microsoft's pyright type checker for Python -- IMO the fastest and most correct type checker in the Python ecosystem -- is implemented in TypeScript!)
- zem 5y ago
  as someone who works on a competing typechecker, i'm definitely very impressed by pyright, especially since it has a single author! we have three people working on pytype and it definitely feels like too little; the way the pyright author manages such a great development velocity is mindblowing, and means they got the initial architecture very right.
wrs 5y ago
62X seems like a lot. It makes me wonder if someone has put serious effort into profiling and optimizing the JavaScript version. If not, that could be a better use of time than a port. I'd guess the TypeScript team may not be interested in maintaining a Go version, so this port will will always be behind the current TypeScript definition.
jkelleyrtp 5y ago
TSC doesn't need to "stick around", right? Just a run-once and the program is over? In those cases, https://github.com/fitzgen/bumpalo https://github.com/fitzgen/bumpalo works amazingly as an arena. You can pretty much forget about reference counting and have direct references everywhere in your graph. The disadvantage is that it's hard to modify your tree without leaving memory around. We use it extensively in http://github.com/dioxusLabs/dioxus http://github.com/dioxusLabs/dioxus and don't need to worry about Rc anywhere in the graph/diffing code.
- anonymousCar 5y ago
  I think --watch mode means it's running while developing
  - connor4312 5y ago
    Additionally, the language server used to power editors is long lived. TypeScript isn't just the compiler.
sam0x17 5y ago
Oof trust me from experience you do _not_ want to write a compiler or parser or anything like that in Go. The string support is so limited -- for example you have to implement basic things like reverse string from scratch instead of them (and much more) being in the standard library. Much better to use something like Crystal or Rust.
- ngrilly 5y ago
  The only string handling a compiler needs is usually during tokenization. After that the compiler only manipulates tokens, AST, SSA, IR, etc.
  - sam0x17 5y ago
    And then you face Go's usual awkward issues with having a lack of generics which becomes important when, say, traversing a tree.
    - pjmlp 5y ago
      Thankfully that will end next month.
      - arwineap 5y ago
        Truthfully I'm not looking forward to it.
    - constexpr 5y ago
      I'm the author of esbuild. I hadn't written Go before I started esbuild and I thought I would miss generics more, but I didn't. There is literally only one situation in the whole esbuild code base where I've found myself wishing for generics: sorting an array. The workaround is easy and involves implementing an interface with three methods, each of which is typically a single line: https://pkg.go.dev/sort#Interface https://pkg.go.dev/sort#Interface. And I only needed a few of these interfaces for a few different types. That wasn't nearly enough of a reason to not use Go, especially considering the other advantages.
smasher164 5y ago
I wonder how it would fare if instead of rewriting from scratch, they applied automated transformation from the Typescript source code to Go. It wouldn't have to capture all the semantics of TS/JS, but be opinionated enough to transform the Typescript source. Another possibility is to use something like GraalVM native-image to compile Typescript's type-checker into a binary. This might offer a speedup over its execution on V8.
tantalor 5y ago
This is a "rewrite" not a "port".
epolanski 5y ago
I'm not so positive about these projects. They are good willed but TypeScript doesn't have an official spec, it's an implementation-defined language. It's not C, it's not JavaScript, it's whatever each release wants it to be. This makes it very difficult to follow every version and evolution, you're consistently chasing whatever has been merged on release.
- peterangular 5y ago
  And it's the exact reason I'm really reluctant to fully buy into Typescript... I always feel like I'm wrong about the "not an official spec" part of things but you are 100% correct in my understanding. Also on "it's whatever each release wants it to be" - that is hell on earth if you're looking to target consistent software processes and it only gets worse the more you scale (both solution and team). I get the value of Typescript - hell I've used it extensively myself... it's just that I can't stop feeling like this is all an anti-pattern: trying to make a loosely-typed language a typed language (if you can call it that?). =/
  - lprd 5y ago
    I share the same concerns. It’s also a little irking that if I want to enjoy good tooling for typescript, I now have to install another language on my system since the language that typescript sits on is a hot mess. I don’t mean to cast shame on JavaScript, but I’d be remiss to not say that modern js development is a bit insane. The only reason we all go along with it is because we have to.
    - eropple 5y ago
      You don't have to install another language to use tools written in Golang (or Rust) unless you have to build binaries yourself.
  - deleted 5y ago
    [deleted]
silasb 5y ago
I'm surprised to see no one comment on Zig. It seems like the bun.sh developers are making some bold changes with such a low-level language.
- kbd 5y ago
  I'm super bullish on Zig, but it's a bit early to bank on for production things as the language is still changing/being developed. To give another example of where this came up: they were discussing using Zig for the Ruby JIT, but despite really liking Zig the team decided to go with Rust because of its maturity: https://bugs.ruby-lang.org/issues/18481#note-21 https://bugs.ruby-lang.org/issues/18481#note-21
- newlisp 5y ago
  I'm more curious about if the author research Ocaml as an alternative.
sjhuang26 5y ago
One thing that was not addressed in the post is the option of using a GC library in Rust. I'm not a Rust expert, but to me, it seems like this makes it quite feasible to quickly port a compiler. They could wrap most complex data structures in GC and optimize them as time goes by. That being said, I'd assume that the library's GC would be slower than Go, so perhaps that's a deal-breaker.
politician 5y ago
I'm sorrowful that such a significant percentage of this post was a preemptive apology for not using Rust.
- newlisp 5y ago
  Well this a blow for the "Rewrite it in Rust" movement.
constexpr 5y ago
I'm the author of esbuild. I think this is a very smart, practical choice and is one of the approaches that has the best chance of succeeding (the other being a semi-automated porting tool instead of a purely manual port). I'm very excited to see someone take this approach. This is how I would do it if I were to do it. Web developers around the world are hoping this works out :) The hard part about this project isn't doing the port but keeping up with the constant changes from the TypeScript team, especially if the TypeScript team isn't invested in helping you out. Staying as close as possible to the source language like this will be a big win there. One of the big benefits of a native language like Go aside from lack of a JIT is the ability to easily add shared-memory parallelism. I haven't studied the TypeScript compiler internals so I'm not sure how easy it is to parallelize their type checker. However, I assume that it would be relatively straightforward to parallelize the parsing of JavaScript files, so I'd hope for at least be a big speedup there.
- joshxyz 5y ago
  love esbuild, ditched webpack and rollpack since then. been a fan of golang too since then.
- vips7L 5y ago
  > the ability to easily add shared-memory parallelism I’m honestly not sure if there is any popular language where this isn’t true?
  - constexpr 5y ago
    The problem is that you can't use shared-memory parallelism in TypeScript. If you could, then the most straightforward way of speeding up the TypeScript compiler would be to parallelize the compiler, not to port it.
    - vips7L 5y ago
      I just don’t see how Go is more uniquely suited to do this than any other language with the same feature and admittedly I may have read too much into your comment.
sapphyrus 5y ago
Maybe tsc already does this, but why not just make extensive use of caching to get the compile times for "incremental builds" under control? Why would it have to re-check all 10,000 files in my project just because I modified a single file? That seems way more manageable (and less error-prone) than rewriting the entire thing.
kaba0 5y ago
Slightly off topic, but do I not get something about Go? It comes up waay to often nowadays here, perhaps even more than LISPs - last time Rust was the hyped one, but it seemed to have decreased a bit. But at least rust is an interesting language with some novel features, adding new value to the low level PL domain. I fail to see what is unique about Go, other than the perhaps least known non-blocking concurrency (based on the recent “invisible” article on this). I find the simplicity of it (with the often claimed reason being basically “code monkeys not hindering each other’s work”, quite offending) really off-puttingz
- gonzo41 5y ago
  Go is about getting things done. I think it's interesting to HN because it is enough engineering to solve a problem, and no more. I think the biggest feature of golang is the community process that considers change slowly, in contrast to the JS ecosystem, that's a distinct feature.
- Mawr 5y ago
  > I fail to see what is unique about Go That's because you're looking at it purely from only one point of view - the number of available features in the language itself. What's unique about Go is not so much about the language itself as the approach of its authors to creating it. Go hasn't meaningfully changed in the 10 years of its existence. That's not an accident - the bar to add anything is extremely high. A feature needs to be punching way above its cost to even be considered. If you look around the programming language space, you'll see the exact opposite attitudes, features get added into languages willy-nilly. Not much point in having different languages when they just trade features until they become indistinguishable from each other. There are only three other languages that I know of that at least sort-of follow a similar philosophy to Go: 1. C 2. Zig 3. Elixir You aren't going to be writing general-purpose code in C, so that's out. Zig is about a decade away from being a contender. Elixir is a valid choice, albeit much more niche and unfamiliar to the average programmer than Go is. So if you want a language that'll be there for you in the long haul, that will not invalidate what you've learned about it every 3 years or so, you're left with Go.
  - 5y ago
overmachine 5y ago
deleted 5y ago
[deleted]
xixixao 5y ago
> The more code you have, the longer it takes to compile. On medium-to-large-sized TypeScript projects, this compilation is extremely slow. Or you could, you know, use the JS typechecker already built with large scale performance in mind (and in native language) - Flow. I know Flow hasn’t had the bandwith to invest in supporting open source, and Typescript pretty much “won”, and there is now a large amount of library definitions without Flow support, but: If your codebase is large enough to suffer from TS’s architecture, you might have the resources to deal with the hurdles of adopting Flow. Flow still has some better designs such as nullables (?T). That said, Flow is now being heavily optimized for large, strictly typed codebase, which means there is now no typechecker geared towards checking JS with minimal type annotations. I think that would still be useful for the prototyping stage of a project.
- imcotton 5y ago
  Last time I heard news about Flow was facebook/jest replacing it with TypeScript.
pea 5y ago
Aside from preferences, is there a reason why C# or F# wouldn't be a more natural choice? I'd imagine porting TS to C# would be easier (especially for this specific use-case) and that the performance would be good.
npretto 5y ago
So this is a "This week vercel is sponsoring" episode...
derpthinker 5y ago
But why isn't Microsoft doing it? Leaving all the hard work to others?
lftp 5y ago
Is this something that could be automated with something like https://aws.github.io/jsii/ https://aws.github.io/jsii/?
mdwagner 5y ago
I'll throw this out there (even though I know the response I'll get), why not use Crystal-lang instead of Go-lang? Besides the obvious differences in funding, what does Go have that Crystal doesn't?

450 points by msoad 5y ago

Here's a relevant GitHub thread where some conversation that has been happening: https://github.com/swc-project/swc/issues/571 https://github.com/swc-project/swc/issues/571

forty 5y ago

go means it's going to be easier to integrate to esbuild :) I'm happy to see this, as honestly I don't really understand why you would compile TS without type checking.

eatonphil 5y ago
> I'm happy to see this, as honestly I don't really understand why you would compile TS without type checking. I don't think I'm remotely alone in doing this but I'll share from my perspective. I have two modes while writing TypeScript stuff: 1) prototyping something where I don't want to spend time writing types (or maybe I write some types but I don't care if they are correct or line up yet) but I still want to have JavaScript generated so I can test the thing out manually and 2) getting types correct as part of long-term maintenance and edge-case removal. There are fast TypeScript -> JavaScript generators today like swc and esbuild. tsc is not fast. So I can speed up "1)" from above if I use swc and esbuild and only use tsc for "2)" from above. As people write faster implementations of tsc-the-type-checker though I'll be happy to use that instead and just not use tsc anymore.
- m-hilgendorf 5y ago
  > 1) prototyping something where I don't want to spend time writing types It doesn't compile to JS, but this is one of the canonical use cases for Stanza 's (1) "optional" type system (1) http://lbstanza.org/optional_typing.html http://lbstanza.org/optional_typing.html
- tshaddox 5y ago
  Out of curiosity, how often do you have a project in the first mode that gets large enough that the speed of compiling TS to JS is significant?
  - eatonphil 5y ago
    There is a huge difference even in small and mid-size projects. https://datastation.multiprocess.io/blog/2021-11-13-benchmarking-esbuild-swc-typescript-babel.html https://datastation.multiprocess.io/blog/2021-11-13-benchmar...
  - nicoburns 5y ago
    If you're writing UI code then even a few seconds is slow. You don't want to be waiting 5 seconds every time you move something a couple of pixels to the left.
- forty 5y ago
  That makes sense, I would generally use JS directly for mode 1), but I understand the use case.
- mixedCase 5y ago
  Have you tried doing Type-Driven Development? IME, most of the time constraints are determined through types first with the code being done later tends to replace the game of "what kind of complex and non-obvious types fit my code the most" with a "oh, the implementation is obvious" reaction. It also makes it easier to determine what behavior is worth testing and which ones are already covered by your types, if you tend to write tests last. If instead you're used to Test-Driven Development you can do types->tests->code.
  - eatonphil 5y ago
    I've spent a lot of time programming in Standard ML and yeah that's a prerequisite there. But no I don't prefer it to what I do in TypeScript.
apazzolini 5y ago
We use SWC with ts-node to strip types in dev for faster iteration loops and run the full tsc typechecker in CI (and optionally as a separate process on your local machine).
postalrat 5y ago
You compile ts without type checking when the options to do so are 10x+ faster than with type checking.
- pjmlp 5y ago
  I really don't get this, when I save my TS files they are already compiled into JS on the fly, ready to fed into a <script />.
  - recursive 5y ago
    By what? What if they were updated from an upstream repo, and never saved from an editor? What if you change branches without an upstream repo?
    - pjmlp 5y ago
      By VS and VSCode, the IDE/editor that belongs to the same company that does TypesScript. If I really need to call tsc explicitly, the few seconds that I have to wait, versus the minutes on some other languages, won't make me lose sleep over it.
mattkrick 5y ago
Personally, I let vscode do typechecking on open files & have a pre-commit git hook to typecheck changed files. When it comes to starting up a development server & building the client, there's a huge cost to repeatedly typechecking the same 1000+ files. By cutting out typechecking & only compiling, I can reduce the webpack client build from 40 seconds to 3 seconds (using sucrase).
- forty 5y ago
  I'm answering to you but this is really for most siblings: I maintain a fairly large TS code base (nodejs backend). I think a full rebuild is 1:30 on my relatively recent laptop (i7-8650U, lots of RAM). But in practice I always use tsc -w and compile is mostly instant after editing a file (I do have to wait whenever I launch the command after I boot, but after that, it's fast enough). tsc now support incremental compilation too, though I haven't played with it too much as I'm happy with watch mode.
  - rezonant 5y ago
    Incremental compilation is _okay_ but it seems to perform full rebuilds if the previous build is _too_ old which is weird- meaning that more builds than expected take the entire time to complete. Nonetheless I use it for all of my builds. On the other hand this may just be because of the spiraling nature of type inference, where TS needs to re-examine a large amount of the program from a change that appears simple. Personally I only have a small number of projects that take more than 10-20 seconds to compile, but those ones are painful. I should probably do the same with -w for those.
- morelisp 5y ago
  What if not everyone else on your team is paying attention to their editor's warnings? What if the LS's async response doesn't come fast enough? What if, god forbid, you have to change something on a different computer? If it's not in your CI/CD scripts, it's not actually getting checked. > I can reduce the webpack client build... to 3 seconds This is about 10x longer than any interaction with a computer should be.
bastawhiz 5y ago
If it takes one minute to type check a codebase and ten seconds to compile it to JavaScript, then whenever I simply need to produce a JavaScript file without type checking I'll save myself the fifty seconds. If my editor's TS language server does the checking and my bundler is just compiling for my dev environment, I don't want both tools duplicating each other's work.
forty 5y ago
Answering to myself, a colleague suggestion (from tsc doc actually): use tsc --noEmit (for type checking) and a fast compiler. tsc --noEmit duration is 2/3 of a normal build on our fairly large code base, so that can definitely speeds things up (for CI for example)

mariogintili 5y ago

k__ 5y ago

tsc relies in shared state and is slow, so they port it to a language that allows shared state. Strange reasoning.

munificent 5y ago
The reasoning is only strange if you assume that the cause of slowness is the shared state. CPUs have no problems with mutability. JS is slow because of dynamic typing, dispatch, flexible object layout, and an object model that requires lots and lots of heap allocation and pointer chasing.
Yoric 5y ago
(er, some text in the parent seems to have vanished) Sure, if you absolutely wish it to, in an `unsafe` block, you can define two writeable pointers to the same piece of data. I don't think I've ever seen anybody write such code in Rust (except perhaps when linking to a C library that required it?), nor have I ever felt the need to do so. Now, it is true that Rust is designed to discourage writing cyclic data structures. I can imagine that trying to port (rather than redesign) an algorithm that deeply requires cyclic data structures, that will make your life miserable.
- wizzwizz4 5y ago
  You actually can't define two `&mut` references to the same data in Rust. It's invalid Rust code; if the compiler notices, it'll refuse to compile, and if it doesn't notice, your code will randomly break.
  - Yoric 5y ago
    You can define two `*mut`, though, in unsafe code.
    - anderskaseorg 5y ago
      A better option is to use Cell or RefCell; they allow completely safe shared mutability in single-threaded code. https://doc.rust-lang.org/std/cell/ https://doc.rust-lang.org/std/cell/

adamnemecek 5y ago

The way of dealing with cycles is generational arenas. Also where is the codebase? I would like to see the uses of unsafe.

jupp0r 5y ago

> tsc depends on shared mutability and has a cyclical mutable reference. Rust is designed to prevent this behavior. Having two references to the same data (shared mutability) is undefined behavior in Rust. This problem has numerous solutions in Rust - (A)Rc, RefCell etc. I don't understand how this caused the author of the article to stop considering Rust.

petmon 5y ago
Non-GC languages are bad in general at handling cyclical references (mutable or not), and Rust is especially ill suited for it because of the borrow checker. For example should an arbitrary reference in tsc become an rc::Rc or an rc::Weak? You have to tease out the implicit ownership model and one may not even exist. tsc was designed expecting GC and it makes perfect sense to port it to another fast GC language.
- throw10920 5y ago
  I'm not a Rust fanboy (in fact, I rather despise the RESF, especially given Rust's mediocre tooling), but: I think that "You have to tease out the implicit ownership model and one may not even exist" suggests a problem with the program itself. I can't think of any reason why a program must have an ill-formed ownership model (unlike, say, the fact that many interesting programs must have an embedded dynamic type system (tagged unions etc) to do their thing). Now. The Rust language and existing tooling doesn't give you any help in figuring out the ownership model, but I can easily imagine a language and tools that would help with that. This isn't meant to refute anything you've said (you said "[existing] non-GC languages are bad in general" not that non-GC languages cannot be good) - just add clarity.
  - adamnemecek 5y ago
    > I rather despise the RESF, especially given Rust's mediocre tooling Which tooling are you talking about?
    - throw10920 5y ago

pjmlp 5y ago

The thing with all those projects is playing catch-up with upstream, and the possible variations from what is actually the official semantics depending on the TypeScript version.

throwaway894345 5y ago
Personally I wouldn't fret about keeping pace with upstream. Eventually the syntax will stabilize if it hasn't already. I'd be more worried about fidelity--does this produce the same result as the reference implementation? TFA says he's running against the "conformance" test suite, so that should keep things pretty tight and when bugs are found they'll be fixed in time and presumably the test suite will be updated accordingly.
- wwwigham 5y ago
  The `conformance` suite is the "set of tests people added when a feature was first released", usually without the decade of following regression tests in the `compiler` suite. In TS itself, the `conformance` and `compiler` suite are considered the part of the same unit most of the time. (The compiler suite, however, is not helpfully subdivided by feature, since it oft deals with cross-cutting concerns!)
- munificent 5y ago
  > Eventually the syntax will stabilize if it hasn't already. Every language in wide use today is still evolving and, if anything, it seems that the pace of language change is generally increasing.
  - throwaway894345 5y ago
    Some amount of evolution is fine, and I think your “pace of language change is increasing” may be true for some languages and time scales, but most languages don’t change more in their second five years than their first.
cellularmitosis 5y ago

londons_explore 5y ago

At first I thought this was for type checking Go code... Ya know, all that interface{} stuff...

andreypopp 5y ago

Well... How many paid work hours spent on tsc? Then does TypeScript have a formal language specification? I'm pessimistic it's possible to rewrite tsc with full compat (bug-to-bug, otherwise it doesn't really make much sense) without MS support... but who knows... I'd like fast compiles too, so good luck!

paavohtl 5y ago
A formal specification would be basically useless, if nothing guarantees that the reference implementation actually follows it. Sharing & contributing to the same test suite is a much more practical approach.
- quickthrower2 5y ago
  As a smoke you can swap it in for a lot of the npm ecosystem and run the tests of those projects on the compiled code.
- andreypopp 5y ago
  Specification isn't being followed is of course useless. But yeah, tests are definitely useless and in the absence of the spec is the only sane way to approach such rewrite.

hardwaregeek 5y ago

This reminds me of when people try to make a new Ruby implementation. They get the basics down and think "this isn't so bad! I can do this!". Fast forward a few months and they're stuck on some nasty little edge case around eval and method_missing that they need to run Rails. Or some weird aspect of C extensions. TypeScript's compiler is an unholy, unsound mess, because JavaScript is an unholy, unsound mess. Sure the basics of type checking structs and variables is somewhat easy. But what about conditional types? What about weird generic constraints? What about indexing types? There's some really confusing stuff going on underneath the hood. It's also a mess that *does not have a spec*. Even if you somehow manage to make a type checker that implements most of the features, there's no way to ensure it has 1 to 1 parity with TypeScript. Plus that parity can be broken at any point. The only way I could see this happening is if the author somehow convinced Anders Hejlsberg or a significant portion of the TypeScript team to work on this new implementation. I'm not sure that's gonna happen.

danenania 5y ago
If they focus on the core features and make a significant enough improvement, perhaps the core TS team could be convinced to pull it in as a native extension and make calls to it for the operations it covers? If something like 62x improvement is really achievable, it's hard to imagine the team not being interested at all. I would personally be very happy to use a significantly slimmed down subset of TypeScript if it provided a 62x performance improvement. Slow type checking is currently the main drawback of TypeScript development in large projects.
- travisd 5y ago
  To this point, many teams are already using subsets of TypeScript to improve compilation times. Often times it’s things like always declaring return types on functions to avoid the compiler having to infer it (especially between module boundaries). Calling this a “subset of typescript” might be strong wording, but it does suggest there is some desire in that general area.
  - fault1 5y ago

danenania 5y ago

If anyone happens to know: what are tsc's bottlenecks? I presume that if the bottleneck was just number of loop iterations, V8 would already be about as fast as C or any other language? Is it simply that accessing/mutating javascript data structures like objects, arrays, set, etc. is slow? Are those not fully optimizable by V8 for some reason? Or is it something else like the GC being slow?

deleted 5y ago
[deleted]
cpcallen 5y ago
Yes, I'm curious about this too. Given that gopherjs[1] can compile Go to JavaScript and in some cases thereby achieve better performance than the native Go compiler[2], it's not obvious to me that a JavaScript program ported mechanically to Go is likely to be any faster than the original. It seems to me that the effort might be better applied to optimising the performance of tsc itself. [1] https://github.com/gopherjs/gopherjs https://github.com/gopherjs/gopherjs [2] https://medium.com/gopherjs/surprises-in-gopherjs-performance-4a0a49b04ecd https://medium.com/gopherjs/surprises-in-gopherjs-performanc...
newlisp 5y ago
This is a problem that can take advantage of good parallelism, Go will give you that and at the same time be more memory efficient.

konart 5y ago

>shared mutability Sure but this goes against "Don't communicate by sharing memory", no?

scottlamb 5y ago
> Sure but this goes against "Don't communicate by sharing memory", no? Yes, but that's advice rather than something Go requires. It's reasonable to say it's outweighed by the desire to avoid significantly restructuring the existing logic. The author is explicitly choosing to match the upstream program's structure to limit the project scope, and whether that structure is good or not is almost besides the point. btw: especially in the Rust community, shared mutability doesn't always mean shared between threads/tasks. It's also a concern in single-threaded/non-concurrent code due to aliasing. This case does actually seem to be involving multithreading given that the author mentioned "using 8 threads".
- newlisp 5y ago
  Curious, can you elaborate on what are the issues of mutability in single-threaded code?
  - scottlamb 5y ago
    It comes down to how a language construct allocates freedom to the programmer vs the optimizing compiler. * Rust references are strict. &mut references guarantee that nothing else can access the backing memory while the &mut is valid. & references guarantee that nothing (including &mut by the rule above, or unsafe code with raw pointers by programmer's pinkie swear) can change the backing memory while the &mut is valid. This allows the compiler to perform more optimizations behind the scene, like assuming a value loaded into a register is still up-to-date, even if there's been a function call between. * Rust raw pointers make no guarantees (but require the programmer to type "unsafe" to use). * C pointers are IMHO weirder. iirc, two pointers of the same type may alias (unless you use the rare "restrict" keyword), but two pointers of different type can't, unless one is "char*". * AFAIK, Go pointers are at least as permissive as C's. They might not make any aliasing guarantees at all. * Other languages vary...

Zababa 5y ago

I see a few people wanting to rewrite tsc in something else than Typescript because tsc itself is too slow. I don't remember seeing any "proof" or analysis that Typescript is the limiting factor in tsc. The basis often used is the performance of esbuild compared to other bundlers. From the esbuild FAQ, "Why is esbuild fast?": > Several reasons: > It's written in Go and compiles to native code. > Parallelism is used heavily. > Everything in esbuild is written from scratch. > Memory is used efficienty. Using another language will solve 1 and will maybe solve 2. From this FAQ too: > For example, many bundlers use the official TypeScript compiler as a parser. But it was built to serve the goals of the TypeScript compiler team and they do not have performance as a top priority. Their code makes pretty heavy use of megamorphic object shapes and unnecessary dynamic property accesses (both well-known JavaScript speed bumps). tsc itself seems like it could be optimized. By how much, I have no idea. But it sounds like it would be easier than writing something from scratch. esbuild, while a great tool, is limited in scope. Typechecking Typescript on the other hand is a moving target, and a fast one at that. All in all, I'm not convinced by this approach compared to "regular performance work" on tsc itself. I think people talk about that because it's easier to start your project than to change tsc. In that case, the problem sound political rather than technical. There's the x62 speed from the article, but there's no info on what features are supported. I feel like this is something where the devil is in the details. I'm happy being proven wrong on that subject, I fear that we're going increase the number of tools in the frontend ecosystem for no good reasons, and make every a bit harder again.

n0w 5y ago
I had the same thought. A direct port of tsc to a "faster" language doesn't guarantee a faster type checker. Sure it might be a bit faster, but my guess is that the large performance wins come from a different design that takes performance into consideration from the start. Maybe the thought process is that a 1:1 rewrite will handle compatibility and feature completeness? I suppose you could optimise from there?
x-shadowban 5y ago
How can you avoid megamorphizing an ast hierarchy?

marcus_holmes 5y ago

Given that ESBuild [0] is a way better replacement for Webpack already (and similarly written in Go) I can see how we could pretty soon have an entire toolchain written in Go for dealing with compiling JS. Hopefully the whole toolchain will be embeddable, too. Being able to do configuration-in-code and produce an executable that does the whole build step in milliseconds would be cool. [0] https://esbuild.github.io/ https://esbuild.github.io/

deleted 5y ago
[deleted]
newlisp 5y ago
The author of esbuild also started the implementation of esbuild in rust, found it painful too and switched to build it in go.
- pjmlp 5y ago
  For me the ideal use case for Rust is really for places where automatic memory management is a hard sell, even if technically possible, kernels, drivers, hypervisors, firmware, GPGPU, ... For everything else I rather enjoy the productivity of automatic memory management.
  - ameliaquining 5y ago
    My general inclination is to go straight to Rust if I already tried a different reasonably performant language (like Node.js) and it wasn't fast enough—because only Rust (and C and C++ and a few more obscure languages) guarantees that basically any performance optimization that I might want to do will at least be possible, even if it might be inelegant or unsafe. I don't want to run the risk of hitting another performance wall that I can't climb over. That being said, porting this kind of codebase to Rust would indeed be really rough—definitely possible, but by no means pleasant. I like to think the platonic ideal Rust, which we're hopefully getting closer and closer to over time, is capable of elegantly handling cyclic data structures and the like even while encouraging a single-ownership model for the 95% of cases where that's the right design. I don't think the Rust we have today is there yet.

slig 5y ago

Cannot wait for their webpack replacement.

scottlamb 5y ago

The "why not port to Rust" part boils down to these paragraphs: > tsc uses a lot of shared mutability and many parts depend on garbage collection. Even though I’m an advocate and believer in Rust, it doesn’t feel like the right tool for the job here. Using Rust required me to use unsafe too much for this specific project. > tsc depends on shared mutability and has a cyclical mutable reference. Rust is designed to prevent this behavior. Having two references to the same data (shared mutability) is undefined behavior in Rust. I'd be curious to hear more. In particular: * Two references to the same data: were cells too awkward? * Actual garbage collection: my first instinct would be to use an arena for each compilation, destroying the whole thing at once rather than doing reference counting or garbage collection. I gather that's not cool in the age of language servers and incremental compilation, though. Does tsc support those?

yazaddaruvala 5y ago
All of that is valid for a rewrite, but not efficient if you’re porting the current implementation. As I understand it, in this case, Rust would be able to have a line to line port from JavaScript. I do still think your overall point holds. Why not use a library in the Rust ecosystem to help with this problem e.g. some GC<T> type.
- scottlamb 5y ago
  > All of that is valid for a rewrite, but not efficient if you’re porting the current implementation. Help me see it? I can easily see how getting rid of the cyclical structure crosses from "line-to-line port" into "rewrite" territory. I'm not saying you're wrong, but I'm having more trouble seeing how "wrap some types in cells and pass along a lifetime and arena parameter" crosses that threshold. It might not be feasible at all, though, if tsc compilations are long-lived for language server + incremental compilation support. > Why not use a library in the Rust ecosystem to help with this problem e.g. some GC<T> type. iirc I've seen some experimental GC library mentioned. I think something like that is at least off the beaten path enough that I can see how the author might decide it's better to just use Go instead.

antihero 5y ago

It will be interesting to see if they create a language server so we can use it in VSCode.

BadOakOx 5y ago

Just from a week ago on HN: I'm making a Typescript type-checker in Rust: https://news.ycombinator.com/item?id=30033119 https://news.ycombinator.com/item?id=30033119

epolanski 5y ago
Next week it will be Zig then
- udbhavs 5y ago
  https://bun.sh/ https://bun.sh/

nevir 5y ago

Super skeptical that porting this to a new language is going to have the kind of performance gains they expect ... assuming they want to replicate all of tsc's functionality

eyelidlessness 5y ago
It’s already proven (esbuild, SWC, Bun) to drastically improve performance of TypeScript’s other responsibility: compilation (tsc doesn’t offer an option for this, but ts-node does, for comparison). And while the type checker is certainly more complex, I have a hard time imagining it won’t show similar improvements in a native language.
- resoluteteeth 5y ago
  Compiling typescript is basically just stripping the type definitions/annotations and other typescript-specific parts of the syntax out. That's a huge difference from actually type checking it.
  - eyelidlessness 5y ago
    And yet, it’s drastically slower in JavaScript than in Go, Rust and Zig. I realize different algorithms will optimize differently between JS and native code, but I sincerely doubt tsc is anywhere near optimal speed.
    - lucian1900 5y ago
      That's mostly because tsc's parser is used by the JS implementations. Writing a parser for the purpose would achieve performance closer to the other implementations.
- strager 5y ago
  esbuild, SWC, and Bun are not ports of existing programs.

davepeck 5y ago

(In a different direction, Microsoft's pyright type checker for Python -- IMO the fastest and most correct type checker in the Python ecosystem -- is implemented in TypeScript!)

zem 5y ago
as someone who works on a competing typechecker, i'm definitely very impressed by pyright, especially since it has a single author! we have three people working on pytype and it definitely feels like too little; the way the pyright author manages such a great development velocity is mindblowing, and means they got the initial architecture very right.

wrs 5y ago

62X seems like a lot. It makes me wonder if someone has put serious effort into profiling and optimizing the JavaScript version. If not, that could be a better use of time than a port. I'd guess the TypeScript team may not be interested in maintaining a Go version, so this port will will always be behind the current TypeScript definition.

jkelleyrtp 5y ago

TSC doesn't need to "stick around", right? Just a run-once and the program is over? In those cases, https://github.com/fitzgen/bumpalo https://github.com/fitzgen/bumpalo works amazingly as an arena. You can pretty much forget about reference counting and have direct references everywhere in your graph. The disadvantage is that it's hard to modify your tree without leaving memory around. We use it extensively in http://github.com/dioxusLabs/dioxus http://github.com/dioxusLabs/dioxus and don't need to worry about Rc anywhere in the graph/diffing code.

anonymousCar 5y ago
I think --watch mode means it's running while developing
- connor4312 5y ago
  Additionally, the language server used to power editors is long lived. TypeScript isn't just the compiler.

sam0x17 5y ago

Oof trust me from experience you do _not_ want to write a compiler or parser or anything like that in Go. The string support is so limited -- for example you have to implement basic things like reverse string from scratch instead of them (and much more) being in the standard library. Much better to use something like Crystal or Rust.

ngrilly 5y ago
The only string handling a compiler needs is usually during tokenization. After that the compiler only manipulates tokens, AST, SSA, IR, etc.
- sam0x17 5y ago
  And then you face Go's usual awkward issues with having a lack of generics which becomes important when, say, traversing a tree.
  - pjmlp 5y ago
    Thankfully that will end next month.
    - arwineap 5y ago
      Truthfully I'm not looking forward to it.
  - constexpr 5y ago
    I'm the author of esbuild. I hadn't written Go before I started esbuild and I thought I would miss generics more, but I didn't. There is literally only one situation in the whole esbuild code base where I've found myself wishing for generics: sorting an array. The workaround is easy and involves implementing an interface with three methods, each of which is typically a single line: https://pkg.go.dev/sort#Interface https://pkg.go.dev/sort#Interface. And I only needed a few of these interfaces for a few different types. That wasn't nearly enough of a reason to not use Go, especially considering the other advantages.

smasher164 5y ago

I wonder how it would fare if instead of rewriting from scratch, they applied automated transformation from the Typescript source code to Go. It wouldn't have to capture all the semantics of TS/JS, but be opinionated enough to transform the Typescript source. Another possibility is to use something like GraalVM native-image to compile Typescript's type-checker into a binary. This might offer a speedup over its execution on V8.

tantalor 5y ago

This is a "rewrite" not a "port".

epolanski 5y ago

I'm not so positive about these projects. They are good willed but TypeScript doesn't have an official spec, it's an implementation-defined language. It's not C, it's not JavaScript, it's whatever each release wants it to be. This makes it very difficult to follow every version and evolution, you're consistently chasing whatever has been merged on release.

peterangular 5y ago
And it's the exact reason I'm really reluctant to fully buy into Typescript... I always feel like I'm wrong about the "not an official spec" part of things but you are 100% correct in my understanding. Also on "it's whatever each release wants it to be" - that is hell on earth if you're looking to target consistent software processes and it only gets worse the more you scale (both solution and team). I get the value of Typescript - hell I've used it extensively myself... it's just that I can't stop feeling like this is all an anti-pattern: trying to make a loosely-typed language a typed language (if you can call it that?). =/
- lprd 5y ago
  I share the same concerns. It’s also a little irking that if I want to enjoy good tooling for typescript, I now have to install another language on my system since the language that typescript sits on is a hot mess. I don’t mean to cast shame on JavaScript, but I’d be remiss to not say that modern js development is a bit insane. The only reason we all go along with it is because we have to.
  - eropple 5y ago
    You don't have to install another language to use tools written in Golang (or Rust) unless you have to build binaries yourself.
- deleted 5y ago
  [deleted]

silasb 5y ago

I'm surprised to see no one comment on Zig. It seems like the bun.sh developers are making some bold changes with such a low-level language.

kbd 5y ago
I'm super bullish on Zig, but it's a bit early to bank on for production things as the language is still changing/being developed. To give another example of where this came up: they were discussing using Zig for the Ruby JIT, but despite really liking Zig the team decided to go with Rust because of its maturity: https://bugs.ruby-lang.org/issues/18481#note-21 https://bugs.ruby-lang.org/issues/18481#note-21
newlisp 5y ago
I'm more curious about if the author research Ocaml as an alternative.

sjhuang26 5y ago

One thing that was not addressed in the post is the option of using a GC library in Rust. I'm not a Rust expert, but to me, it seems like this makes it quite feasible to quickly port a compiler. They could wrap most complex data structures in GC and optimize them as time goes by. That being said, I'd assume that the library's GC would be slower than Go, so perhaps that's a deal-breaker.

politician 5y ago

I'm sorrowful that such a significant percentage of this post was a preemptive apology for not using Rust.

newlisp 5y ago
Well this a blow for the "Rewrite it in Rust" movement.

constexpr 5y ago

I'm the author of esbuild. I think this is a very smart, practical choice and is one of the approaches that has the best chance of succeeding (the other being a semi-automated porting tool instead of a purely manual port). I'm very excited to see someone take this approach. This is how I would do it if I were to do it. Web developers around the world are hoping this works out :) The hard part about this project isn't doing the port but keeping up with the constant changes from the TypeScript team, especially if the TypeScript team isn't invested in helping you out. Staying as close as possible to the source language like this will be a big win there. One of the big benefits of a native language like Go aside from lack of a JIT is the ability to easily add shared-memory parallelism. I haven't studied the TypeScript compiler internals so I'm not sure how easy it is to parallelize their type checker. However, I assume that it would be relatively straightforward to parallelize the parsing of JavaScript files, so I'd hope for at least be a big speedup there.

joshxyz 5y ago
love esbuild, ditched webpack and rollpack since then. been a fan of golang too since then.
vips7L 5y ago
> the ability to easily add shared-memory parallelism I’m honestly not sure if there is any popular language where this isn’t true?
- constexpr 5y ago
  The problem is that you can't use shared-memory parallelism in TypeScript. If you could, then the most straightforward way of speeding up the TypeScript compiler would be to parallelize the compiler, not to port it.
  - vips7L 5y ago
    I just don’t see how Go is more uniquely suited to do this than any other language with the same feature and admittedly I may have read too much into your comment.

sapphyrus 5y ago

Maybe tsc already does this, but why not just make extensive use of caching to get the compile times for "incremental builds" under control? Why would it have to re-check all 10,000 files in my project just because I modified a single file? That seems way more manageable (and less error-prone) than rewriting the entire thing.

kaba0 5y ago

Slightly off topic, but do I not get something about Go? It comes up waay to often nowadays here, perhaps even more than LISPs - last time Rust was the hyped one, but it seemed to have decreased a bit. But at least rust is an interesting language with some novel features, adding new value to the low level PL domain. I fail to see what is unique about Go, other than the perhaps least known non-blocking concurrency (based on the recent “invisible” article on this). I find the simplicity of it (with the often claimed reason being basically “code monkeys not hindering each other’s work”, quite offending) really off-puttingz

gonzo41 5y ago
Go is about getting things done. I think it's interesting to HN because it is enough engineering to solve a problem, and no more. I think the biggest feature of golang is the community process that considers change slowly, in contrast to the JS ecosystem, that's a distinct feature.
Mawr 5y ago
> I fail to see what is unique about Go That's because you're looking at it purely from only one point of view - the number of available features in the language itself. What's unique about Go is not so much about the language itself as the approach of its authors to creating it. Go hasn't meaningfully changed in the 10 years of its existence. That's not an accident - the bar to add anything is extremely high. A feature needs to be punching way above its cost to even be considered. If you look around the programming language space, you'll see the exact opposite attitudes, features get added into languages willy-nilly. Not much point in having different languages when they just trade features until they become indistinguishable from each other. There are only three other languages that I know of that at least sort-of follow a similar philosophy to Go: 1. C 2. Zig 3. Elixir You aren't going to be writing general-purpose code in C, so that's out. Zig is about a decade away from being a contender. Elixir is a valid choice, albeit much more niche and unfamiliar to the average programmer than Go is. So if you want a language that'll be there for you in the long haul, that will not invalidate what you've learned about it every 3 years or so, you're left with Go.
- 5y ago

overmachine 5y ago

deleted 5y ago

[deleted]

xixixao 5y ago

> The more code you have, the longer it takes to compile. On medium-to-large-sized TypeScript projects, this compilation is extremely slow. Or you could, you know, use the JS typechecker already built with large scale performance in mind (and in native language) - Flow. I know Flow hasn’t had the bandwith to invest in supporting open source, and Typescript pretty much “won”, and there is now a large amount of library definitions without Flow support, but: If your codebase is large enough to suffer from TS’s architecture, you might have the resources to deal with the hurdles of adopting Flow. Flow still has some better designs such as nullables (?T). That said, Flow is now being heavily optimized for large, strictly typed codebase, which means there is now no typechecker geared towards checking JS with minimal type annotations. I think that would still be useful for the prototyping stage of a project.

imcotton 5y ago
Last time I heard news about Flow was facebook/jest replacing it with TypeScript.

pea 5y ago

Aside from preferences, is there a reason why C# or F# wouldn't be a more natural choice? I'd imagine porting TS to C# would be easier (especially for this specific use-case) and that the performance would be good.

npretto 5y ago

So this is a "This week vercel is sponsoring" episode...

derpthinker 5y ago

But why isn't Microsoft doing it? Leaving all the hard work to others?

lftp 5y ago

Is this something that could be automated with something like https://aws.github.io/jsii/ https://aws.github.io/jsii/?

mdwagner 5y ago

I'll throw this out there (even though I know the response I'll get), why not use Crystal-lang instead of Go-lang? Besides the obvious differences in funding, what does Go have that Crystal doesn't?