16 ms·

Hi folks, Daniel Rosenwasser from the TypeScript team here. We're obviously very excited to announce this! RyanCavanaugh (our dev lead) and I are around to answ

Hi folks, Daniel Rosenwasser from the TypeScript team here. We're obviously very excited to announce this! RyanCavanaugh (our dev lead) and I are around to answer any quick questions you might have. You can also tune in to the Discord AMA mentioned in the blog this upcoming Thursday.

loevborg 2y ago
Congrat on the announcement, this is a great achievement!
dimitropoulos 2y ago
thank you, to both of you, for so many years of groundbreaking work. you've both been on the project for, what, 11 years now? such legends.
jherdman 2y ago
I'd be curious to hear about the politics and behinds the scenes of this project. How did you get buy-in? What were some of the sticking points in getting this project off of the ground? When you mention that many other languages were used to spike the new compiler, were there interesting learnings?
eknkc 2y ago
I feel like you'll need to provide a wasm binary for browser environments and maybe as a fallback in node itself. Last time I checked, Go really struggles to perform when targeting wasm. This might be the only reason I'd like to see it in Rust but I'm still glad you went with Go. Are there any insights on the platform decision?
- jchw 2y ago
  Honestly, the choice seems fine to me: the vast majority of users are not compiling huge TypeScript projects in the browser. If you're using Vite/ESBuild, you're already using a Go-based JS toolchain, and last I checked Vite was pretty darn popular. I don't suspect there will be a huge burden for things like playground; given the general performance uplift that the Go tsc implementation already gets, it may in fact be faster even after paying the Wasm tax. (And even if it isn't, it should be more than fine for playground anyways.)
  - merb 2y ago
    I‘m pretty sure that a lot of vite users with hot reload will run tsc inside the browser (tanstack, react-router)
    - jchw 2y ago
      I am not a Vite expert, however, when running Vite in dev mode, I can see two things: - There is an esbuild process running in the background. - If I look at the JavaScript returned to the browser, it is transpiled without any types present. So even though the URLs in Vite dev mode look like they're pointing to "raw" TypeScript files, they're actually transpiled JavaScript, just not bundled. I could be incorrect, of course, but it sure seems to me like Vite is using ESBuild on the Node.JS side and not tsc on the web browser side.
pjmlp 2y ago
Why not AOT compiled C#, given the team's historical background?
- tgv 2y ago
  Not involved, but there's a faq in their repo, and this answers your question, perhaps, a bit: https://github.com/microsoft/typescript-go/discussions/411 https://github.com/microsoft/typescript-go/discussions/411
  - pjmlp 2y ago
    Thanks, but it really doesn't clarify why a team with roots on the .NET ecosystem decided C#/Native AOT isn't fit for purpose.
    - deleted 2y ago
      [deleted]
    - vessenes 2y ago
      Pure speculation, but C# is not nearly the first class citizen that go binaries are when you look at all possible deployment targets. The “new” Microsoft likely has some built-in bias against “embrace and extend” architectural and business decisions for developers. Overall this doesn’t seem like a hard choice to me. Cue rust devotees in 3, 2, ..
      - deleted 2y ago
        [deleted]
      - zozbot234 2y ago
        > Cue rust devotees in 3, 2, .. If you are a rust devotee, you can use https://github.com/FractalFir/rustc_codegen_clr https://github.com/FractalFir/rustc_codegen_clr to compile your rust code to the same .NET runtime as C#. The project is still in the works but support is said to be about 95% complete.
    - 2y ago
nozzlegear 2y ago
> You can also tune in to the Discord AMA mentioned in the blog this upcoming Thursday. Will the questions and answers be posted anywhere outside of Discord after it's concluded?
- AshleyGrant 2y ago
  Daniel, please make this a priority. Post the Q&A transcript to GitHub, at least.
jauntywundrkind 2y ago
What is the forward paths available for efforts like the TS Playground under Typescript 7 (native)? One of the nice advantages of js is that it can run so many places. Will TypeScript still be able to enjoy that legacy going forward, or is native only what we should expect in 7+?
- DanRosenwasser 2y ago
  We anticipate that we will eventually get a playground working on the new native codebase. We know we'll likely compile down to WebAssembly, but a lot of how it gets integrated will depend on what the API looks like. We're currently giving a lot of thought to that, but we have good ideas. https://github.com/microsoft/typescript-go/discussions/455 https://github.com/microsoft/typescript-go/discussions/455
  - Tadpole9181 2y ago
    Will this be a prerequisite of the 7.0 release?
phpnode 2y ago
This is very exciting! I'm curious if this move eventually unlocks features that have been deemed too expensive/slow so far, e.g. typing `ReactElement` more accurately, typing `TemplateStringsArray` etc
umvi 2y ago
Since the new tsc is written in go, will I be able to pull it into my go web server as a middleware to dynamically transpile ts?
- DanRosenwasser 2y ago
  We'll be working on an API that ideally can be used through any language - that would be our preferred means of consuming the new codebase.
jakub_g 2y ago
Hi Daniel! What's your stance on support for yarn pnp?
- DanRosenwasser 2y ago
  pnp is still very cool, and it would be great if we can find a better API story that works well with pnp!
no_wizard 2y ago
Like others I'm curious about the choice of technology here. I see you went with Go, which is great! I know Go is fast! But its also a more 'primitive' language (for lack of a better way of putting it) with no frills. Why not something like Rust? Most of the JS ecosystem that is moving toward faster tools seem to be going straight to Rust (Rolldown, rspack (the webpack successor) SWC, OXC, Lightning CSS / Parcel etc) and one of the reasons given is it has really great language constructs for parsers and traversing ASTs (I think largely due to the existence of `match` but i'm not entirely sure) Was any thought given to this? And if so what was the deciding factors for Go vs something like Rust or another language entirely?
- DanRosenwasser 2y ago
  We did anticipate this question, and we have actually written up an FAQ entry on our GitHub Discussions. I'll post the response below. https://github.com/microsoft/typescript-go/discussions/411 https://github.com/microsoft/typescript-go/discussions/411. ____ Language choice is always a hot topic! We extensively evaluated many language options, both recently and in prior investigations. We also considered hybrid approaches where certain components could be written in a native language, while keeping core typechecking algorithms in JavaScript. We wrote multiple prototypes experimenting with different data representations in different languages, and did deep investigations into the approaches used by existing native TypeScript parsers like swc, oxc, and esbuild. To be clear, many languages would be suitable in a ground-up rewrite situation. Go did the best when considering multiple criteria that are particular to this situation, and it's worth explaining a few of them. By far the most important aspect is that we need to keep the new codebase as compatible as possible, both in terms of semantics and in terms of code structure. We expect to maintain both codebases for quite some time going forward. Languages that allow for a structurally similar codebase offer a significant boon for anyone making code changes because we can easily port changes between the two codebases. In contrast, languages that require fundamental rethinking of memory management, mutation, data structuring, polymorphism, laziness, etc., might be a better fit for a ground-up rewrite, but we're undertaking this more as a port that maintains the existing behavior and critical optimizations we've built into the language. Idiomatic Go strongly resembles the existing coding patterns of the TypeScript codebase, which makes this porting effort much more tractable. Go also offers excellent control of memory layout and allocation (both on an object and field level) without requiring that the entire codebase continually concern itself with memory management. While this implies a garbage collector, the downsides of a GC aren't particularly salient in our codebase. We don't have any strong latency constraints that would suffer from GC pauses/slowdowns. Batch compilations can effectively forego garbage collection entirely, since the process terminates at the end. In non-batch scenarios, most of our up-front allocations (ASTs, etc.) live for the entire life of the program, and we have strong domain information about when "logical" times to run the GC will be. Go's model therefore nets us a very big win in reducing codebase complexity, while paying very little actual runtime cost for garbage collection. We also have an unusually large amount of graph processing, specifically traversing trees in both upward and downward walks involving polymorphic nodes. Go does an excellent job of making this ergonomic, especially in the context of needing to resemble the JavaScript version of the code. Acknowledging some weak spots, Go's in-proc JS interop story is not as good as some of its alternatives. We have upcoming plans to mitigate this, and are committed to offering a performant and ergonomic JS API. We've been constrained in certain possible optimizations due to the current API model where consumers can access (or worse, modify) practically anything, and want to ensure that the new codebase keeps the door open for more freedom to change internal representations without having to worry about breaking all API users. Moving to a more intentional API design that also takes interop into account will let us move the ecosystem forward while still delivering these huge performance wins.
noodletheworld 2y ago
Go is quite difficult to embed in other applications due to the runtime. What do you see as the future for use cases where the typescript compiler is embedded in other projects? (Eg. Deno, Jupyter kernels, etc.) There’s some talk of an inter process api, but vague hand waving here about technical details. What’s the vision? In TS7 will you be able to embed the compiler? Or is that not supported?
- DanRosenwasser 2y ago
  We are sure there will be a way to embed via something like WebAssembly, but the goal is to start from the IPC layer (similar to LSP), and then explore how possible it will be to integrate at a tighter level.
- _benton 2y ago
  Golang is actually pretty easy to embed into JS/TS via wasm. See esbuild.
  - curtisblaine 2y ago
    Esbuild is distributed as a series of native executables that are selectively installed by looking at arch and platform. Although you can build esbuild in wasm (and that's what you use when you run it in the browser), what you actually run from .bin in the CLI is a native executable, not wasm.
- nine_k 2y ago
  Why embed it if you can run a process alongside yours and use efficient IPC? I suppose the compiler code should not be in some tight loop where an IPC boundary would be a noticeable slowdown. Compilation occurs relatively rarely, compared to running the compiled code, in things like Node / Deno / Bun / Jupyter. LSPs use this model with a pretty wasteful XML IPC, and they don't seem to feel slow.
  - wokwokwok 2y ago
    Because running a parallel process is often difficult. In most cases, the question becomes: So, how exactly is my app/whatever supposed to spin up a parallel process in the OS and then talk to it over IPC? How do you shut it down when the 'host' process dies? Not vaguely. Not hand wave "just launch it". How exactly do you do it? How do you do it in environments where that capability (spawning arbitrary processes) is limited? eg. mobile. How do you package it so that you distribute it in parallel? Will it conflict with other applications that do the same thing? When you look at, for example, a jupyter kernel, it is already a host process launched and managed by jupyter-lab or whatever, which talks via network chatter. So now each kernel process has to manage another process, which it talks to via IPC? ... Certainly, there are no obvious performance reasons to avoid IPC, but I think there are use cases where having the compiler embedded makes more sense.
Etheryte 2y ago
This might be an oddly specific question, but do you think performance improvements like this might eventually lead to features like partial type argument inference in generics? If I recall correctly off the top of my head, performance was one of the main reasons it was never implemented.
AshleysBrain 2y ago
Well-optimized JavaScript can get to within about 1.5x the performance of C++ - something we have experience with having developed a full game engine in JavaScript [1]. Why is the TypeScript team moving to an entirely different technology instead of working on optimizing the existing TS/JS codebase? [1] https://www.construct.net/en https://www.construct.net/en
- jchw 2y ago
  Please note that compilers and game engines have extremely different needs and performance characteristics—and also that statements like "about 1.5x the performance of C++" are virtually meaningless out-of-context. I feel we've long passed this type of performance discussion by and could do with more nuanced and specific discussions.
- internetter 2y ago
  Sometimes, the time required to optimize is greater than the time required to rewrite.
- grandempire 2y ago
  What kind of C++ and what kind of JS? - C++ with thousands of tiny objects and virtual function calls? - JavaScript where data is stored in large Int32Array and does operations on it like a VM? If you know anything about how JavaScript works, you know there is a lot of costly and challenging resource management.
- do_not_redeem 2y ago
  Well-optimized JavaScript can, if you jump through hoops like avoiding object creation and storing your data in `Uint8Array`s. But idiomatic, maintainable JS simply can't (except in microbenchmarks where allocations and memory layout aren't yet concerns). In a game engine, you probably aren't recreating every game object from frame to frame. But in a compiler, you're creating new objects for every file you parse. That's a huge amount of work for the GC.
  - deleted 2y ago
    [deleted]
jasonthorsness 2y ago
I write a lot of Go and a decent amount of TypeScript. Was there anything you found during this project that you found particularly helpful/nice in Go, vs. TypeScript? Or was there anything about Go that increased the difficulty or required a change of approach?
sbjs 2y ago
Your patience with Michael Saboff is incredible.
titzer 2y ago
I'm curious about the choice of Go to develop the new toolchain. Was the support for parallelism/concurrency a factor in the decision?
spankalee 2y ago
Hey Daniel. I write a lot of tools that depend on the TypeScript compiler API, and they run in a lot of a lot of JS environments including Node and the browser. The current CJS codebase is even a little tricky to load into standard JS module supporting environments like browsers, so I've been _really_ looking forward to what Jake and others have said will be an upcoming standard modules based version. Is that still happening, and how will the native compiler be distributed for us tools authors? I presume WASM? Will the compiler API be compatible? Transforms, the AST, LanguageService, Program, SourceFile, Checker, etc.? I'm quite concerned that the migration path for tools could be extremely difficult. [edit] To add to this as I think about it: I maintain libraries that build on top of the TS API, and are then in turn used by other libraries that still access the TS APIs. Things like framework static analysis, then used by various linters, compilers, etc. Some linters are integrated with eslint via typescript-eslint. So the dependency chain is somewhat deep and wide. Is the path forward going to be that just the TS compiler has a JS interop layer and the rest stays the same, or are all TS ecosystem tools going to have to port to Go to run well?
- lytedev 2y ago
  Reading the article, it looks like they are writing go, so will probably be distributing go binaries.
  - maxloh 2y ago
    Maybe they'll also be distributed in WASM too, which is easier to be integrated with JavaScript codebases.
    - nine_k 2y ago
      Would running WASM be any faster than running JS in V8?
      - kevingadd 2y ago
        Interop with a WASM-compiled Go binary from JS will be slower but the WASM binary itself might be a lot faster than a JS implementation, if that makes sense. So it depends on how chatty your interop is. The main place you get bogged down is typically exchanging strings across the boundary between WASM and JS. Exchanging buffers (file data, etc) can also be a source of slowdown.
h1fra 2y ago
Amazing news, but I'm wondering what will happen to Monaco editor and all the SaaS that use typescript in the browser?
- dataviz1000 2y ago
  Not sure if it does but the video linked in the post might answer your question? I think he is compiling vscode which includes Monaco editor which is where they are getting 10x faster stat. (I might be wrong here.) [0] [0] https://youtu.be/pNlq-EVld70?feature=shared&t=112 https://youtu.be/pNlq-EVld70?feature=shared&t=112
  - h1fra 2y ago
    Yeah I saw that but will they maintain a browser compatible version is another question
    - dataviz1000 2y ago
      Ah, inception compiling. The issue isn't compiling the Monaco editor, but rather will the Monaco editor compile TypeScript 7 in the browser? That is a good question.
felixrieseberg 2y ago
Daniel, congrats! I'm _so_ excited about everything y'all have achieved in the last few years.
pbreit 2y ago
Amazing!! I did not see timing. When might we see in VS Code? Edge?
polskibus 2y ago
Have you considered a closer to metal language to implement the compiler in like c or rust ? Have you evaluated further perf improvements ?
- davedx 2y ago
  I don't think c or rust are really 'closer to the metal' than golang (what they're using)
  - cube00 2y ago
    Considering Go is the only language with a garbage collector out of the three languages you mentioned, I'm not sure how you reach the conclusion they're all as close to the metal. C and Rust both have predictable memory behaviour, Go does not.
    - nasretdinov 2y ago
      Go isn't that bad in terms of memory predictability to be honest. It generally has roughly 100% overhead in terms of memory usage compared to no GC. This can be reduced by using GOGC env variable, at the cost of worse performance if not careful.
    - gwbas1c 2y ago
      When I read the article it was very clear, due to the compiler's in-memory graphs, that they needed a GC. (IE, as opposed to reference counting, where if you have cyclic loops, you need to manually go in and "break" the loop so memory gets reclaimed.)
      - tuveson 2y ago
        > When I read the article it was very clear, due to the compiler's in-memory graphs, that they needed a GC. It's actually pretty easy to do something like this with C, just using something like an arena allocator, or honestly, leaking memory. I actually wrote a little allocator yesterday that just dumps memory into a linkedlist, it's not very complicated: http://github.com/danieltuveson/dsalloc/ http://github.com/danieltuveson/dsalloc/ You allocate wherever you want, and when you're done with the big messy memory graph, you throw it all out at once. There are obviously a lot of other reasons to choose go over C, though (easier to learn, nicer tooling, memory safety, etc).
ezekg 2y ago
When can we just replace the JS runtime with TS and skip the compiler altogether? Start fresh, if you will.
hedora 2y ago
Go is an extremely strange choice, given the ecosystem you're targeting. I've got quite a bit of experience in it, TS, Rust and C++. I'd pick any of those for productivity and (in the case of C++ and Rust, thread-safety) over Go, simply because Go's type system is so impoverished. From a performance perspective, I'd expect C++ and Rust to be much easier targets too, since I've seen quite a few industrial Go services be rewritten in C++/Rust after they fail to meet runtime performance / operability targets. Wasn't there a recent study from Google that came to the same conclusion? (They see improved productivity for Go with junior programmers that don't understand static typing, but then they can never actually stabilize the resulting codebase.)
- blueredmodern 2y ago
  [dead]
ackfoobar 2y ago
Will we still have compiler plugins? What will this mean for projects like ts-patch?
gwbas1c 2y ago
Thanks for answering questions. One thing I'm curious about: What about updating the original Typescript-based compiler to target WASM and/or native code, without needing to run in a Javascript VM? Was that considered? What would (at a high level) the obstacles be to achieving similar performance to Golang? Edit: Clarified to show that I indicate updating the original compiler.
- rafram 2y ago
  It's unlikely that you would get much performance benefit from AOT compiling a TypeScript codebase. (At least not with a ton of manual optimization of the native code, and if you're going to do that, why not just rewrite in a native-first language?) JavaScript, like other dynamic languages, runs well with a JIT because the runtime can optimize for hotspots and common patterns (e.g. this method's first argument is generally an object with this shape, so write a fast path for that case). In theory you could write an AOT compiler for TypeScript that made some of those inferences at compile time based on type definitions, but (a) nobody's done that (b) it still wouldn't be as fast as native, or much faster than JIT (c) it would be limited - any optimizations would die as soon as you used an inherently dynamic method like JSON.parse()
  - gwbas1c 2y ago
    So basically, TypeScript as a language doesn't allow compiling to as as efficient machine code as Golang? (Edit) And I assume it's not practical to alter the language in a way that this kind of information can be added. (Such as adding a typed version of JSON.parse()).
devit 2y ago
[flagged]
- umvi 2y ago
  > inexpressive type system Simplicity is a feature, not a bug. Overly expressive languages become nightmares to work with and reason about (see: C++ templates) Go's compilation times are also extremely fast compared to Rust, which is a non-negligible cost when iterating on large projects.
- deleted 2y ago
  [deleted]
ksec 2y ago
Is 10x a starting point or could we expect even more improvements in the future?
textlapse 2y ago
I am curious why dotnet was not considered - it should run everywhere Go does with added NativeAoT too, so I am especially curious given the folks involved ;) (FWIW, It must have been a very well thought out rationale.) Edit: watched the revenant clip from the GH discussion- makes sense. Maybe push NativeAoT to be as good? I am (positively) surprised Hejlsberg has not used this opportunity to push C#: a rarity in the software world where people never let go of their darlings. :)
- neals 2y ago
  It was considered and tested, just not used in the end.
- jamescrowley 2y ago
  Discussion and video link here for anyone else interested: https://github.com/microsoft/typescript-go/discussions/411#discussioncomment-12463258 https://github.com/microsoft/typescript-go/discussions/411#d... And lightly edited transcript here: https://github.com/microsoft/typescript-go/discussions/411#discussioncomment-12464695 https://github.com/microsoft/typescript-go/discussions/411#d...
imbnwa 2y ago
Will the refactor possibly be an occasion for ironing out a spec
rayiner 2y ago
Why Go?
culi 2y ago
> While we’re not yet feature-complete This is a big concern to me. Could you expand on what work is left to do for the native implementation of gsc? In particular, can you make an argument why that last bit of work won't reduce these 10x figures we're seeing? I'm worried the marketing got ahead of the engineering
- sashank_1509 2y ago
  It’s fine, if it’s 2x faster after being feature complete, I don’t really mind. It still is a free speedup to all existing code-bases. Developers don’t need to anything than install the latest version of TypeScript I presume
  - culi 2y ago
    With an added dependency on golang. Might have some consequences for certain people's build process
conartist6 2y ago
Hi Daniel! Really interesting news, and uniquely dismaying to me as someone who is fighting tooth and claw to keep JS language tooling in the JS ecosystem. My question has to do with Ryan's statement: > We also considered hybrid approaches where certain components could be written in a native language, while keeping core typechecking algorithms in JavaScript I've experimented deeply in this area (maybe 15k hours invested in BABLR so far) and what I've found is that it's richly rewarding. Javascript is fast enough for what is needed, and its ability to cache on immutable data can make it lightning fast not through doing more work faster, but by making it possible to do less work. In other words, change the complexity class not the constant factor. Is this a direction you investigated? What made you decide to try to move sideways instead of forwards?
- moogly 2y ago
  > as someone who is fighting tooth and claw to keep JS language tooling in the JS ecosystem Have you considered the man-years and energy you're making everyone waste? Just as an example, I wonder what the carbon footprint of ESLint has been over the years... Now, it pales in comparison to Python, but still...
  - conartist6 2y ago
    I'm no more thrilled than you at the cost of running ESLint, but using a high-level language doesn't need to mean being wasteful of resources. TS currently wastes tons of resources (most especially peoples' time) by not being able to share its data and infrastructure with other tools and ecosystems, but while there would be much bigger wins from tackling the systemic problem, you wouldn't be able to say something as glib as "TS is 10x faster". Only the work that can be distilled to a metric is done now, because that's how to get a promotion when you work for a company like Microsoft
    - Timon3 2y ago
      If I could choose between Typescript speeding up 10x or all the surrounding tooling speeding up 20x, I'd take Typescript in a heartbeat. Slow type checking is the biggest pain point in my daily dev cycle. Thank you Typescript team for chasing those promotions!
aylmao 2y ago
This is awesome. Thanks to you and all the TypeScript team for the work they put on this project! Also, nice to see you here, engaging with the community. Porting to Go was the right decision, but part of me would've liked to see a different approach to solve the performance issue. Here I'm not thinking about the practicality, but simply about how cool it would've been if performance had instead been improved via: - porting to OCaml. I contributed to Flow once upon a time, and a version of TypeScript in OCaml would've been huge in unifying the efforts here. - porting to Rust. Having "official" TypeScript crates in rust would be huge for the Rust javascript-tooling ecosystem. - a new runtime (or compiler!). I'm thinking here an optional, stricter version of TypeScript that forbids all the dynamic behaviours that make JavaScript hard to optimize. I'm also imagining an interpreter or compiler that can then use this stricter TypeScript to run faster or produce an efficient native binary, skipping JavaScript altogether and using types for optimization. This last option would've been especially exciting since it is my opinion that Flow was hindered by the lack of dogfooding, at least when I was somewhat involved with the project. I hope this doesn't happen in the TypeScript project. None of these are questions, just wanted to share these fanciful perspectives. I do agree Go sounds like the right choice, and and in any case I'm excited about the improvement in performance and memory usage. It really is the biggest gripe I have with TypeScript right now!
- muglug 2y ago
  Not Daniel, but I've ported a typechecker from PHP to Rust (with some functional changes) and also tried working with the official Hack OCaml-based typechecker (a precursor to Flow). Rust and OCaml are _maybe_ prettier to look at, but for the average TypeScript developer Go is a much more understandable target IMO. Lifetimes and ownership are not trivial topics to grasp, and they add overhead (as discussed here: https://github.com/microsoft/typescript-go/discussions/411 https://github.com/microsoft/typescript-go/discussions/411) that not all contributors might grasp immediately.
  - deleted 2y ago
    [deleted]
timmg 2y ago
Does the mean the future toolset will be less reliant on NPM (etc)? Maybe I'm unique, but I prefer to not have the compiler tied up with a paryicular dependency-management/library system.

loevborg 2y ago

Congrat on the announcement, this is a great achievement!

dimitropoulos 2y ago

thank you, to both of you, for so many years of groundbreaking work. you've both been on the project for, what, 11 years now? such legends.

jherdman 2y ago

I'd be curious to hear about the politics and behinds the scenes of this project. How did you get buy-in? What were some of the sticking points in getting this project off of the ground? When you mention that many other languages were used to spike the new compiler, were there interesting learnings?

eknkc 2y ago

I feel like you'll need to provide a wasm binary for browser environments and maybe as a fallback in node itself. Last time I checked, Go really struggles to perform when targeting wasm. This might be the only reason I'd like to see it in Rust but I'm still glad you went with Go. Are there any insights on the platform decision?

jchw 2y ago
Honestly, the choice seems fine to me: the vast majority of users are not compiling huge TypeScript projects in the browser. If you're using Vite/ESBuild, you're already using a Go-based JS toolchain, and last I checked Vite was pretty darn popular. I don't suspect there will be a huge burden for things like playground; given the general performance uplift that the Go tsc implementation already gets, it may in fact be faster even after paying the Wasm tax. (And even if it isn't, it should be more than fine for playground anyways.)
- merb 2y ago
  I‘m pretty sure that a lot of vite users with hot reload will run tsc inside the browser (tanstack, react-router)
  - jchw 2y ago
    I am not a Vite expert, however, when running Vite in dev mode, I can see two things: - There is an esbuild process running in the background. - If I look at the JavaScript returned to the browser, it is transpiled without any types present. So even though the URLs in Vite dev mode look like they're pointing to "raw" TypeScript files, they're actually transpiled JavaScript, just not bundled. I could be incorrect, of course, but it sure seems to me like Vite is using ESBuild on the Node.JS side and not tsc on the web browser side.

pjmlp 2y ago

Why not AOT compiled C#, given the team's historical background?

tgv 2y ago
Not involved, but there's a faq in their repo, and this answers your question, perhaps, a bit: https://github.com/microsoft/typescript-go/discussions/411 https://github.com/microsoft/typescript-go/discussions/411
- pjmlp 2y ago
  Thanks, but it really doesn't clarify why a team with roots on the .NET ecosystem decided C#/Native AOT isn't fit for purpose.
  - deleted 2y ago
    [deleted]
  - vessenes 2y ago
    Pure speculation, but C# is not nearly the first class citizen that go binaries are when you look at all possible deployment targets. The “new” Microsoft likely has some built-in bias against “embrace and extend” architectural and business decisions for developers. Overall this doesn’t seem like a hard choice to me. Cue rust devotees in 3, 2, ..
    - deleted 2y ago
      [deleted]
    - zozbot234 2y ago
      > Cue rust devotees in 3, 2, .. If you are a rust devotee, you can use https://github.com/FractalFir/rustc_codegen_clr https://github.com/FractalFir/rustc_codegen_clr to compile your rust code to the same .NET runtime as C#. The project is still in the works but support is said to be about 95% complete.
  - 2y ago

nozzlegear 2y ago

> You can also tune in to the Discord AMA mentioned in the blog this upcoming Thursday. Will the questions and answers be posted anywhere outside of Discord after it's concluded?

AshleyGrant 2y ago
Daniel, please make this a priority. Post the Q&A transcript to GitHub, at least.

jauntywundrkind 2y ago

What is the forward paths available for efforts like the TS Playground under Typescript 7 (native)? One of the nice advantages of js is that it can run so many places. Will TypeScript still be able to enjoy that legacy going forward, or is native only what we should expect in 7+?

DanRosenwasser 2y ago
We anticipate that we will eventually get a playground working on the new native codebase. We know we'll likely compile down to WebAssembly, but a lot of how it gets integrated will depend on what the API looks like. We're currently giving a lot of thought to that, but we have good ideas. https://github.com/microsoft/typescript-go/discussions/455 https://github.com/microsoft/typescript-go/discussions/455
- Tadpole9181 2y ago
  Will this be a prerequisite of the 7.0 release?

phpnode 2y ago

This is very exciting! I'm curious if this move eventually unlocks features that have been deemed too expensive/slow so far, e.g. typing `ReactElement` more accurately, typing `TemplateStringsArray` etc

umvi 2y ago

Since the new tsc is written in go, will I be able to pull it into my go web server as a middleware to dynamically transpile ts?

DanRosenwasser 2y ago
We'll be working on an API that ideally can be used through any language - that would be our preferred means of consuming the new codebase.

jakub_g 2y ago

Hi Daniel! What's your stance on support for yarn pnp?

DanRosenwasser 2y ago
pnp is still very cool, and it would be great if we can find a better API story that works well with pnp!

no_wizard 2y ago

Like others I'm curious about the choice of technology here. I see you went with Go, which is great! I know Go is fast! But its also a more 'primitive' language (for lack of a better way of putting it) with no frills. Why not something like Rust? Most of the JS ecosystem that is moving toward faster tools seem to be going straight to Rust (Rolldown, rspack (the webpack successor) SWC, OXC, Lightning CSS / Parcel etc) and one of the reasons given is it has really great language constructs for parsers and traversing ASTs (I think largely due to the existence of `match` but i'm not entirely sure) Was any thought given to this? And if so what was the deciding factors for Go vs something like Rust or another language entirely?

DanRosenwasser 2y ago
We did anticipate this question, and we have actually written up an FAQ entry on our GitHub Discussions. I'll post the response below. https://github.com/microsoft/typescript-go/discussions/411 https://github.com/microsoft/typescript-go/discussions/411. ____ Language choice is always a hot topic! We extensively evaluated many language options, both recently and in prior investigations. We also considered hybrid approaches where certain components could be written in a native language, while keeping core typechecking algorithms in JavaScript. We wrote multiple prototypes experimenting with different data representations in different languages, and did deep investigations into the approaches used by existing native TypeScript parsers like swc, oxc, and esbuild. To be clear, many languages would be suitable in a ground-up rewrite situation. Go did the best when considering multiple criteria that are particular to this situation, and it's worth explaining a few of them. By far the most important aspect is that we need to keep the new codebase as compatible as possible, both in terms of semantics and in terms of code structure. We expect to maintain both codebases for quite some time going forward. Languages that allow for a structurally similar codebase offer a significant boon for anyone making code changes because we can easily port changes between the two codebases. In contrast, languages that require fundamental rethinking of memory management, mutation, data structuring, polymorphism, laziness, etc., might be a better fit for a ground-up rewrite, but we're undertaking this more as a port that maintains the existing behavior and critical optimizations we've built into the language. Idiomatic Go strongly resembles the existing coding patterns of the TypeScript codebase, which makes this porting effort much more tractable. Go also offers excellent control of memory layout and allocation (both on an object and field level) without requiring that the entire codebase continually concern itself with memory management. While this implies a garbage collector, the downsides of a GC aren't particularly salient in our codebase. We don't have any strong latency constraints that would suffer from GC pauses/slowdowns. Batch compilations can effectively forego garbage collection entirely, since the process terminates at the end. In non-batch scenarios, most of our up-front allocations (ASTs, etc.) live for the entire life of the program, and we have strong domain information about when "logical" times to run the GC will be. Go's model therefore nets us a very big win in reducing codebase complexity, while paying very little actual runtime cost for garbage collection. We also have an unusually large amount of graph processing, specifically traversing trees in both upward and downward walks involving polymorphic nodes. Go does an excellent job of making this ergonomic, especially in the context of needing to resemble the JavaScript version of the code. Acknowledging some weak spots, Go's in-proc JS interop story is not as good as some of its alternatives. We have upcoming plans to mitigate this, and are committed to offering a performant and ergonomic JS API. We've been constrained in certain possible optimizations due to the current API model where consumers can access (or worse, modify) practically anything, and want to ensure that the new codebase keeps the door open for more freedom to change internal representations without having to worry about breaking all API users. Moving to a more intentional API design that also takes interop into account will let us move the ecosystem forward while still delivering these huge performance wins.

noodletheworld 2y ago

Go is quite difficult to embed in other applications due to the runtime. What do you see as the future for use cases where the typescript compiler is embedded in other projects? (Eg. Deno, Jupyter kernels, etc.) There’s some talk of an inter process api, but vague hand waving here about technical details. What’s the vision? In TS7 will you be able to embed the compiler? Or is that not supported?

DanRosenwasser 2y ago
We are sure there will be a way to embed via something like WebAssembly, but the goal is to start from the IPC layer (similar to LSP), and then explore how possible it will be to integrate at a tighter level.
_benton 2y ago
Golang is actually pretty easy to embed into JS/TS via wasm. See esbuild.
- curtisblaine 2y ago
  Esbuild is distributed as a series of native executables that are selectively installed by looking at arch and platform. Although you can build esbuild in wasm (and that's what you use when you run it in the browser), what you actually run from .bin in the CLI is a native executable, not wasm.
nine_k 2y ago
Why embed it if you can run a process alongside yours and use efficient IPC? I suppose the compiler code should not be in some tight loop where an IPC boundary would be a noticeable slowdown. Compilation occurs relatively rarely, compared to running the compiled code, in things like Node / Deno / Bun / Jupyter. LSPs use this model with a pretty wasteful XML IPC, and they don't seem to feel slow.
- wokwokwok 2y ago
  Because running a parallel process is often difficult. In most cases, the question becomes: So, how exactly is my app/whatever supposed to spin up a parallel process in the OS and then talk to it over IPC? How do you shut it down when the 'host' process dies? Not vaguely. Not hand wave "just launch it". How exactly do you do it? How do you do it in environments where that capability (spawning arbitrary processes) is limited? eg. mobile. How do you package it so that you distribute it in parallel? Will it conflict with other applications that do the same thing? When you look at, for example, a jupyter kernel, it is already a host process launched and managed by jupyter-lab or whatever, which talks via network chatter. So now each kernel process has to manage another process, which it talks to via IPC? ... Certainly, there are no obvious performance reasons to avoid IPC, but I think there are use cases where having the compiler embedded makes more sense.

Etheryte 2y ago

This might be an oddly specific question, but do you think performance improvements like this might eventually lead to features like partial type argument inference in generics? If I recall correctly off the top of my head, performance was one of the main reasons it was never implemented.

AshleysBrain 2y ago

Well-optimized JavaScript can get to within about 1.5x the performance of C++ - something we have experience with having developed a full game engine in JavaScript [1]. Why is the TypeScript team moving to an entirely different technology instead of working on optimizing the existing TS/JS codebase? [1] https://www.construct.net/en https://www.construct.net/en

jchw 2y ago
Please note that compilers and game engines have extremely different needs and performance characteristics—and also that statements like "about 1.5x the performance of C++" are virtually meaningless out-of-context. I feel we've long passed this type of performance discussion by and could do with more nuanced and specific discussions.
internetter 2y ago
Sometimes, the time required to optimize is greater than the time required to rewrite.
grandempire 2y ago
What kind of C++ and what kind of JS? - C++ with thousands of tiny objects and virtual function calls? - JavaScript where data is stored in large Int32Array and does operations on it like a VM? If you know anything about how JavaScript works, you know there is a lot of costly and challenging resource management.
do_not_redeem 2y ago
Well-optimized JavaScript can, if you jump through hoops like avoiding object creation and storing your data in `Uint8Array`s. But idiomatic, maintainable JS simply can't (except in microbenchmarks where allocations and memory layout aren't yet concerns). In a game engine, you probably aren't recreating every game object from frame to frame. But in a compiler, you're creating new objects for every file you parse. That's a huge amount of work for the GC.
- deleted 2y ago
  [deleted]

jasonthorsness 2y ago

I write a lot of Go and a decent amount of TypeScript. Was there anything you found during this project that you found particularly helpful/nice in Go, vs. TypeScript? Or was there anything about Go that increased the difficulty or required a change of approach?

sbjs 2y ago

Your patience with Michael Saboff is incredible.

titzer 2y ago

I'm curious about the choice of Go to develop the new toolchain. Was the support for parallelism/concurrency a factor in the decision?

spankalee 2y ago

Hey Daniel. I write a lot of tools that depend on the TypeScript compiler API, and they run in a lot of a lot of JS environments including Node and the browser. The current CJS codebase is even a little tricky to load into standard JS module supporting environments like browsers, so I've been _really_ looking forward to what Jake and others have said will be an upcoming standard modules based version. Is that still happening, and how will the native compiler be distributed for us tools authors? I presume WASM? Will the compiler API be compatible? Transforms, the AST, LanguageService, Program, SourceFile, Checker, etc.? I'm quite concerned that the migration path for tools could be extremely difficult. [edit] To add to this as I think about it: I maintain libraries that build on top of the TS API, and are then in turn used by other libraries that still access the TS APIs. Things like framework static analysis, then used by various linters, compilers, etc. Some linters are integrated with eslint via typescript-eslint. So the dependency chain is somewhat deep and wide. Is the path forward going to be that just the TS compiler has a JS interop layer and the rest stays the same, or are all TS ecosystem tools going to have to port to Go to run well?

lytedev 2y ago
Reading the article, it looks like they are writing go, so will probably be distributing go binaries.
- maxloh 2y ago
  Maybe they'll also be distributed in WASM too, which is easier to be integrated with JavaScript codebases.
  - nine_k 2y ago
    Would running WASM be any faster than running JS in V8?
    - kevingadd 2y ago
      Interop with a WASM-compiled Go binary from JS will be slower but the WASM binary itself might be a lot faster than a JS implementation, if that makes sense. So it depends on how chatty your interop is. The main place you get bogged down is typically exchanging strings across the boundary between WASM and JS. Exchanging buffers (file data, etc) can also be a source of slowdown.

h1fra 2y ago

Amazing news, but I'm wondering what will happen to Monaco editor and all the SaaS that use typescript in the browser?

dataviz1000 2y ago
Not sure if it does but the video linked in the post might answer your question? I think he is compiling vscode which includes Monaco editor which is where they are getting 10x faster stat. (I might be wrong here.) [0] [0] https://youtu.be/pNlq-EVld70?feature=shared&t=112 https://youtu.be/pNlq-EVld70?feature=shared&t=112
- h1fra 2y ago
  Yeah I saw that but will they maintain a browser compatible version is another question
  - dataviz1000 2y ago
    Ah, inception compiling. The issue isn't compiling the Monaco editor, but rather will the Monaco editor compile TypeScript 7 in the browser? That is a good question.

felixrieseberg 2y ago

Daniel, congrats! I'm _so_ excited about everything y'all have achieved in the last few years.

pbreit 2y ago

Amazing!! I did not see timing. When might we see in VS Code? Edge?

polskibus 2y ago

Have you considered a closer to metal language to implement the compiler in like c or rust ? Have you evaluated further perf improvements ?

davedx 2y ago
I don't think c or rust are really 'closer to the metal' than golang (what they're using)
- cube00 2y ago
  Considering Go is the only language with a garbage collector out of the three languages you mentioned, I'm not sure how you reach the conclusion they're all as close to the metal. C and Rust both have predictable memory behaviour, Go does not.
  - nasretdinov 2y ago
    Go isn't that bad in terms of memory predictability to be honest. It generally has roughly 100% overhead in terms of memory usage compared to no GC. This can be reduced by using GOGC env variable, at the cost of worse performance if not careful.
  - gwbas1c 2y ago
    When I read the article it was very clear, due to the compiler's in-memory graphs, that they needed a GC. (IE, as opposed to reference counting, where if you have cyclic loops, you need to manually go in and "break" the loop so memory gets reclaimed.)
    - tuveson 2y ago
      > When I read the article it was very clear, due to the compiler's in-memory graphs, that they needed a GC. It's actually pretty easy to do something like this with C, just using something like an arena allocator, or honestly, leaking memory. I actually wrote a little allocator yesterday that just dumps memory into a linkedlist, it's not very complicated: http://github.com/danieltuveson/dsalloc/ http://github.com/danieltuveson/dsalloc/ You allocate wherever you want, and when you're done with the big messy memory graph, you throw it all out at once. There are obviously a lot of other reasons to choose go over C, though (easier to learn, nicer tooling, memory safety, etc).

ezekg 2y ago

When can we just replace the JS runtime with TS and skip the compiler altogether? Start fresh, if you will.

hedora 2y ago

Go is an extremely strange choice, given the ecosystem you're targeting. I've got quite a bit of experience in it, TS, Rust and C++. I'd pick any of those for productivity and (in the case of C++ and Rust, thread-safety) over Go, simply because Go's type system is so impoverished. From a performance perspective, I'd expect C++ and Rust to be much easier targets too, since I've seen quite a few industrial Go services be rewritten in C++/Rust after they fail to meet runtime performance / operability targets. Wasn't there a recent study from Google that came to the same conclusion? (They see improved productivity for Go with junior programmers that don't understand static typing, but then they can never actually stabilize the resulting codebase.)

blueredmodern 2y ago
[dead]

ackfoobar 2y ago

Will we still have compiler plugins? What will this mean for projects like ts-patch?

gwbas1c 2y ago

Thanks for answering questions. One thing I'm curious about: What about updating the original Typescript-based compiler to target WASM and/or native code, without needing to run in a Javascript VM? Was that considered? What would (at a high level) the obstacles be to achieving similar performance to Golang? Edit: Clarified to show that I indicate updating the original compiler.

rafram 2y ago
It's unlikely that you would get much performance benefit from AOT compiling a TypeScript codebase. (At least not with a ton of manual optimization of the native code, and if you're going to do that, why not just rewrite in a native-first language?) JavaScript, like other dynamic languages, runs well with a JIT because the runtime can optimize for hotspots and common patterns (e.g. this method's first argument is generally an object with this shape, so write a fast path for that case). In theory you could write an AOT compiler for TypeScript that made some of those inferences at compile time based on type definitions, but (a) nobody's done that (b) it still wouldn't be as fast as native, or much faster than JIT (c) it would be limited - any optimizations would die as soon as you used an inherently dynamic method like JSON.parse()
- gwbas1c 2y ago
  So basically, TypeScript as a language doesn't allow compiling to as as efficient machine code as Golang? (Edit) And I assume it's not practical to alter the language in a way that this kind of information can be added. (Such as adding a typed version of JSON.parse()).

devit 2y ago

[flagged]

umvi 2y ago
> inexpressive type system Simplicity is a feature, not a bug. Overly expressive languages become nightmares to work with and reason about (see: C++ templates) Go's compilation times are also extremely fast compared to Rust, which is a non-negligible cost when iterating on large projects.
deleted 2y ago
[deleted]

ksec 2y ago

Is 10x a starting point or could we expect even more improvements in the future?

textlapse 2y ago

I am curious why dotnet was not considered - it should run everywhere Go does with added NativeAoT too, so I am especially curious given the folks involved ;) (FWIW, It must have been a very well thought out rationale.) Edit: watched the revenant clip from the GH discussion- makes sense. Maybe push NativeAoT to be as good? I am (positively) surprised Hejlsberg has not used this opportunity to push C#: a rarity in the software world where people never let go of their darlings. :)

neals 2y ago
It was considered and tested, just not used in the end.
jamescrowley 2y ago
Discussion and video link here for anyone else interested: https://github.com/microsoft/typescript-go/discussions/411#discussioncomment-12463258 https://github.com/microsoft/typescript-go/discussions/411#d... And lightly edited transcript here: https://github.com/microsoft/typescript-go/discussions/411#discussioncomment-12464695 https://github.com/microsoft/typescript-go/discussions/411#d...

imbnwa 2y ago

Will the refactor possibly be an occasion for ironing out a spec

rayiner 2y ago

Why Go?

culi 2y ago

> While we’re not yet feature-complete This is a big concern to me. Could you expand on what work is left to do for the native implementation of gsc? In particular, can you make an argument why that last bit of work won't reduce these 10x figures we're seeing? I'm worried the marketing got ahead of the engineering

sashank_1509 2y ago
It’s fine, if it’s 2x faster after being feature complete, I don’t really mind. It still is a free speedup to all existing code-bases. Developers don’t need to anything than install the latest version of TypeScript I presume
- culi 2y ago
  With an added dependency on golang. Might have some consequences for certain people's build process

conartist6 2y ago

Hi Daniel! Really interesting news, and uniquely dismaying to me as someone who is fighting tooth and claw to keep JS language tooling in the JS ecosystem. My question has to do with Ryan's statement: > We also considered hybrid approaches where certain components could be written in a native language, while keeping core typechecking algorithms in JavaScript I've experimented deeply in this area (maybe 15k hours invested in BABLR so far) and what I've found is that it's richly rewarding. Javascript is fast enough for what is needed, and its ability to cache on immutable data can make it lightning fast not through doing more work faster, but by making it possible to do less work. In other words, change the complexity class not the constant factor. Is this a direction you investigated? What made you decide to try to move sideways instead of forwards?

moogly 2y ago
> as someone who is fighting tooth and claw to keep JS language tooling in the JS ecosystem Have you considered the man-years and energy you're making everyone waste? Just as an example, I wonder what the carbon footprint of ESLint has been over the years... Now, it pales in comparison to Python, but still...
- conartist6 2y ago
  I'm no more thrilled than you at the cost of running ESLint, but using a high-level language doesn't need to mean being wasteful of resources. TS currently wastes tons of resources (most especially peoples' time) by not being able to share its data and infrastructure with other tools and ecosystems, but while there would be much bigger wins from tackling the systemic problem, you wouldn't be able to say something as glib as "TS is 10x faster". Only the work that can be distilled to a metric is done now, because that's how to get a promotion when you work for a company like Microsoft
  - Timon3 2y ago
    If I could choose between Typescript speeding up 10x or all the surrounding tooling speeding up 20x, I'd take Typescript in a heartbeat. Slow type checking is the biggest pain point in my daily dev cycle. Thank you Typescript team for chasing those promotions!

aylmao 2y ago

This is awesome. Thanks to you and all the TypeScript team for the work they put on this project! Also, nice to see you here, engaging with the community. Porting to Go was the right decision, but part of me would've liked to see a different approach to solve the performance issue. Here I'm not thinking about the practicality, but simply about how cool it would've been if performance had instead been improved via: - porting to OCaml. I contributed to Flow once upon a time, and a version of TypeScript in OCaml would've been huge in unifying the efforts here. - porting to Rust. Having "official" TypeScript crates in rust would be huge for the Rust javascript-tooling ecosystem. - a new runtime (or compiler!). I'm thinking here an optional, stricter version of TypeScript that forbids all the dynamic behaviours that make JavaScript hard to optimize. I'm also imagining an interpreter or compiler that can then use this stricter TypeScript to run faster or produce an efficient native binary, skipping JavaScript altogether and using types for optimization. This last option would've been especially exciting since it is my opinion that Flow was hindered by the lack of dogfooding, at least when I was somewhat involved with the project. I hope this doesn't happen in the TypeScript project. None of these are questions, just wanted to share these fanciful perspectives. I do agree Go sounds like the right choice, and and in any case I'm excited about the improvement in performance and memory usage. It really is the biggest gripe I have with TypeScript right now!

muglug 2y ago
Not Daniel, but I've ported a typechecker from PHP to Rust (with some functional changes) and also tried working with the official Hack OCaml-based typechecker (a precursor to Flow). Rust and OCaml are _maybe_ prettier to look at, but for the average TypeScript developer Go is a much more understandable target IMO. Lifetimes and ownership are not trivial topics to grasp, and they add overhead (as discussed here: https://github.com/microsoft/typescript-go/discussions/411 https://github.com/microsoft/typescript-go/discussions/411) that not all contributors might grasp immediately.
- deleted 2y ago
  [deleted]

timmg 2y ago

Does the mean the future toolset will be less reliant on NPM (etc)? Maybe I'm unique, but I prefer to not have the compiler tied up with a paryicular dependency-management/library system.