15 ms·

Rewriting TypeScript in Rust?

237 points by mariuz 4y ago

Tade0 4y ago
> But there's a problem. Rewriting a library like TypeScript is extremely challenging. First, "absence of [a] specification" for TypeScript's behaviour. Donny's had to infer TypeScript's behaviour mostly test cases alone. IIRC the TypeScirpt leadership is of the position that the implementation is the specification, which is unfortunate, because it means that the bugs in the implementation are necessarily part of the specification as well.
- Kuinox 4y ago
  Specification are not immune to bugs.
  - rk06 4y ago
    But you can use them as evidence that tsc is not working as it "should". Then go ahead and fix tsc Or spec or both as needed. Without the specs, there is no way to tell if tac is behaving as developers intended it
    - gfxgirl 4y ago
      It's not doing what I expect. Oh? Change the spec It's not doing what I expect. Oh? Change the code How is that different from now?
      - htkibar 4y ago
        Allows competing implementations and saves TS from MSFT
      - rk06 4y ago
        Right now: 1.It's not doing what I expect. 2.Is it intentional behavior or a bug? 3. Let's ask on github/reddit etc With Special: 1. It is not doing what I expect 2. Is this a spec issue or code issue? 3. Let's check spec. 4a. "Working as specified". Shit. Hope spec improves in future versions 4b. It is not working as spec says. Let's file a bug on tsc. And that is the difference.
        steveklabnik 4y ago
        The real rub is in step 4: it is not always easy to determine what the spec means. And then you're basically back to "let's ask on github/reddit etc."
        yencabulator 4y ago
        Just because the comprehensiveness of something is on a spectrum is not a reason to just not bother and have none at all. Also, when done well a spec is a great way to learn & understand the language too. http://golang.org/ref/spec http://golang.org/ref/spec is so much more usable than Typescript's docs.
        steveklabnik 4y ago
        I agree that specifications are useful. I also think they're often presented as a panacea. It is important to acknowledge that they're a tool, with pros and cons, like any tool. The existence of a specification does not immediately mean that any and all ambiguities are resolved.
        tshaddox 4y ago
        Still doesn’t seem like much a difference. If you run into 4a you still file an issue somewhere.
      - randomdata 4y ago
        If an implementation isn't doing what you expect, is it just an issue with that single implementation? A problem with all implementations? A problem with what you expect? If another implementation does do what you expect is it correct or just lucky? A spec resolves such discrepancies. Indeed, the spec may be where the problem is, but once the spec is corrected then implementations know what they must do. Absent of a spec, who knows?
    - kristiandupont 4y ago
      And such a spec can be 1) a reference implementation, 2) written in natural language, or 3) written as formal semantics #1 is the same as the current situation. With #2, you might get a bit closer but natural language has many ambiguities and idiosyncrasies that can render it useless in various situations. You can try to define formal semantics, but that is not only hard to create, it's also hard to translate into an implementation, adding significant overhead in both parts of the process. And there can still be bugs, and there can still be edge cases with undefined behavior.
    - Kuinox 4y ago
      Code, called implementation here, is a specification written in a certain grammar that is executed. You want 'evidence' tsc is not working as it should, by writting a duplicated less specifying specification that would sometimes conflict with the typescript one. It make me remember the people that wanted to specify everything and generate implementation from it: that called a programming language.
      - rk06 4y ago
        Code is meant for machine. Specification is meant for compiler author. Writing one of them is easier than otht
        randomdata 4y ago
        They are not mutually exclusive. Code identifies a class of language. Specification identifies something that is conveyed through language. Specification can be provided in code. In fact, while poorly named, what we call testing is actually specification. A test suite specifies for software authors what a program is intended to do and how it is intended to be used. Of course, as a nice side bonus, because the specification is delivered in code the documentation can be validated for truthfulness against an implementation by machine.
  - remexre 4y ago
    But at least formal methods tools can be applied directly to the specification, to show properties like "well, if you carve out this subset of TypeScript, you have something you can safely eval()" or "you can never have type errors with this usage of this construct in this way."
    - zokier 4y ago
      > But at least formal methods tools can be applied directly to the specification What makes you think that? For example for C language it has been significant effort to translate the specification to something analyzable by tools
    - brundolf 4y ago
      > "well, if you carve out this subset of TypeScript, you have something you can safely eval()" or "you can never have type errors with this usage of this construct in this way." This kind of certainty is very at odds with TypeScript's design philosophy and the way it (necessarily) works. Because it's overlaid on JavaScript, and JavaScript is wildly dynamic (Proxies, prototype modification, etc), almost nothing is known with absolute certainty. To deal with that, TypeScript makes a lot of "optimistic" assumptions, but holds them lightly (i.e. its assumptions may cause type checks to pass when they shouldn't, but it's not going to lean on them for anything that could really blow up if they're wrong) IMO it strikes a very good balance given the constraints - not criticizing the designers at all - but TypeScript has to be treated differently than other statically typed languages might be, because nearly everything it knows/says is just a "probably"
  - namuol 4y ago
    Neither are programs, tests, or type definitions, but these are generally still valuable things to have.
- ape4 4y ago
  Maybe the first step should be writing down a spec of TypeScript.
  - progx 4y ago
    Boring! That is the problem, nobody want to do such boring things.
    - yencabulator 4y ago
      Nobody except Microsoft is able to write a spec for Typescript. Microsoft has already declared that the implementation is it, any spec a random person writes will just be ignored.
    - ansraliant 4y ago
    - epolanski 4y ago
      It's not about boredom, it's simply an implementation derived language as are Haskell or PHP or Python. Having a spec is really valuable only if you intend to target different environments, that's a non-goal for TS it simply transpiles, rather than compiles, into another programming language. It's also bound to follow each new JS feature and change.
      - randomdata 4y ago
        The value of a spec is having something that resolves how to deal with quirks discovered in an implementation. Absent of a spec, any implementation quirks become a part of the language. Which is okay as long as you only ever have one implementation, but when you want to, say, reimplement Typescript in Rust then you have to duplicate all of the quirks that came before, even in cases where nobody has yet realized that the quirk exists. Failure to do so means that your implementation won't be compatible with the canonical implementation. At the other end of the spectrum, the Go project maintains two independent compiler implementations to ensure that the spec is implementable and to see that a single implementation, with all of its quirks, doesn't end up defining the language. That puts a lot of confidence in the spec to guide additional implementations, but, of course, is a lot of added work. Tradeoffs, as always.
- miohtama 4y ago
  For the sake of being fair for other programming languages, what are the languages with proper specification? C, C++, JavaScript, others?
  - randomdata 4y ago
    Go is staunch about its specification. To the point that the Go team maintains two independent compiler implementations to ensure that "features"/bugs in one implementation doesn't end up defining the standard and to ensure that a compiler can be implemented from the spec.
    - tristan957 4y ago
      Is the other compiler you are referring to gcc-go?
    - recov 4y ago
      There's actually 3 now from google afaik - gc - gccgo - gollvm [0] [0] https://go.googlesource.com/gollvm/ https://go.googlesource.com/gollvm/
  - munificent 4y ago
    Dart has a complete specification. It tends to lag somewhat behind the most recent features (which are documented in separate language proposals), but we keep it as current as we can. Maintaining a language specification is a lot more work than people realize, and a job that few have the skills to do well.
  - xchkr1337 4y ago
    In my opinion the presence of undefined behavior in the C/C++ specification makes it no better than the "specification by implementation" approach
    - spac 4y ago
      without a specification you can never tell with certainty if there's a bug
- the_gipsy 4y ago
  That's one way to keep tight control.
noahtallen 4y ago
This reminds me a bit of Rome, which aims to re-implement most parts of a JS tool chain without requiring dozens of different, misaligned dependencies. They also happen to be writing it in Rust, though they haven’t finished the bundler portion yet. I wonder if there’s an opportunity for collaboration here! https://rome.tools/blog/2021/09/21/rome-will-be-rewritten-in-rust/ https://rome.tools/blog/2021/09/21/rome-will-be-rewritten-in...
clintonb 4y ago
If successful, this project would save an immense amount of person and compute hours.
mlajtos 4y ago
Could new AI tools help with such endeavor?
tiberriver256 4y ago
> "Honestly, I'm still not sure if open-source is sustainable without companies like Vercel." Honestly, I'm still not sure if people could breathe without things like air.
- intelVISA 4y ago
mgomez 4y ago
At the beginning of this year, kdy1 (Donny) said he'd be switching to Go after initially trying things out in Rust [1][2], I guess he decided to go through with Rust after all? Or is this something slightly different? [1] https://kdy1.dev/posts/2022/1/tsc-go https://kdy1.dev/posts/2022/1/tsc-go [2] https://news.ycombinator.com/item?id=30074414 https://news.ycombinator.com/item?id=30074414
- munbun 4y ago
  https://kdy1.dev/posts/2022/10/open-sourcing-stc#ix27ll-switch-back-to-the-rust-version https://kdy1.dev/posts/2022/10/open-sourcing-stc#ix27ll-swit...
- clarkdave 4y ago
  From the interview (it's way down the page): > The most challenging thing is the absence of the specification. I had to infer everything just from test cases. > The second thing is the velocity of tsc. It's way too fast, and I was unsure if I could follow it up. So I decided to use semi-automated translation and selected Golang. > But it was too boring, and more importantly, my programming ability has improved enough to follow tsc only by myself.
- rob74 4y ago
  Looks like he moved back to Rust sometime between January and October, at which point he also changed the name to stc: https://kdy1.dev/posts/2022/10/open-sourcing-stc https://kdy1.dev/posts/2022/10/open-sourcing-stc
- CPUTranslator 4y ago
  From the main post and other comments it seems like it was for personal reasons, rather than pragmatic ones. They said they didn’t know how fast the Go implementation would be, but felt better about the original Rust one (I’m assuming before this really became a problem). I think the reasons they switched are pretty weak, but justified at the same time. Kdy1 looks like more of a Rust person (their GitHub has more active Rust repos than Go), so this should’ve been the choice from the beginning. Going with the comfortable choice over the “pragmatic” one is almost always the best option if you’re the only contributor (or plan to be for a while).
muglug 4y ago
If you're just porting the code to a new language, the process can be significantly faster than writing it from scratch. But if you're also re-interpreting the code, which seems to be Donny's approach, you can get bogged down with the discrepancies. The more the two diverge, the harder it is to debug discrepancies and keep track of updates in the upstream code. I'm also porting a static analysis tool from an interpreted language to Rust, but it's a fairly straightforward port and I've been able to stick to schedule pretty well.
- tuyiown 4y ago
  There is also the scenario that tsc is really not that good, and the discrepancies are not a problem enough to avoid potential stc benefits. This can eventually phase out tsc in typescript altogether, the core typescript team making stc the official implementation. This is especially credible with rust having good support with wasm, keeping the js runtime compatibility of tsc.
  - cardanome 4y ago
    Would that be a wise career choice for the people currently working on tsc? Sounds more like they should purposely break compatibility form time to time to prevent Donny from catching up.
    - rk06 4y ago
      Should people really make their professional decisions based on future career choice? I think not. Besides, such malicious action can easily be counterproductive. People can just stop using tsc and switch completely to Rust version. Then, it would effectively make any development on tsc useless.
lizen_one 4y ago
Rust is a "new" language and many packages from other languages get reimplemented in it. This is similar to Julia. Unfortunately, I had the experience that many Julia packages are not of high quality, not maintained, or do not run any more on the newest version. How is this in Rust?
- vardump 4y ago
  I don't know how it's in general, but the few Rust packages I checked seem to be of very high quality.
- kibwen 4y ago
  Like every package repository (or human endeavor in general) it follows Sturgeon's Law: 80% of everything is crap. That said, there's 100k crates on crates.io, and many of them are fantastic (well-supported, actively developed, documented, etc.). For a new user, understanding which are high-quality is a daunting task, and is expedited by just asking an experienced person for specific recommendations.
  - civopsec 4y ago
    Didn’t expect to see such harsh criticism of the Rust ecosystem by this user.
    - steveklabnik 4y ago
      Contrary to the memes, a lot of long running Rust users are happy to tell you about where Rust is deficient.
      - civopsec 4y ago
        Not what I meant.
        cstrahan 4y ago
        What did you mean?
bluelightning2k 4y ago
Wow. What a huge undertaking!
csande17 4y ago
Very exciting! I can only imagine how many doctoral research projects' worth of work this represents.
nnurmanov 4y ago
Shouldn’t be an ML model that rewrites from one language to another? ChatGPT comes to mind
protortyp 4y ago
I hope he and others succeed! I've been using chatgpt to translate Python/TS into Rust/C++ for some smaller projects of mine in the past week. And with some tweaking it works beautifully. Sometimes it makes up crates that do not exist though.
- zozbot234 4y ago
  > Sometimes it makes up crates that do not exist though. You're missing the point, these are ideas for further work.
thefaux 4y ago
This is incredibly impressive but I also simultaneously find it kind of depressing. In my not so humble opinion, typescript is already way too complicated and getting worse thanks to its velocity. A better approach would be to kill typescript's velocity by identifying a stable subset of the language that provides the vast majority of its benefits and optimize the heck out of a parser for that smaller language. This approach would be both easier and more sustainable since it would cap the language complexity making it easier to maintain and/or write alternative implementations.
- serverlessmania 4y ago
  At this point, TS is becoming C# V2
- Zaskoda 4y ago
  I find this comment validating. Most of the languages I've used start to click and feel good after a while - but it feels like this just won't happen for me with typescript. I'm clearly missing some fundamentals and I'm not even sure what it is I need to go study.
  - evandwight 4y ago
    What doesn't click? I've found it pretty intuitive, though I've explicitly avoided making complicated types.
    - randomdata 4y ago
      The type erasure. You get this nicely expressive type system that can encode all sorts of useful metadata, and then you have to recreate it all again by hand in 'Javascript' to access it at runtime. Say I have `type Foo = 'bar' | 'baz'` and want to validate that an input string is within the set of Foo. Why can't I just enumerate the type to determine the valid values at runtime?
      - scotty79 4y ago
thenoblesunfish 4y ago
If there was ever a HN headline I'd suspect of being machine generated, this would be it :D
galaxyLogic 4y ago
From the article some reasons not to use TypeScript: "... in VSCode, the red lines and warnings take a long time to update on large projects. ... absence of [a] specification" for TypeScript's behaviour. Donny's had to infer TypeScript's behaviour mostly test cases alone. The pure size of TypeScript is daunting. It's had ten years to iterate, grow and add features. " --- Therefore for practical reasons, I stick with plain JavaScript (on Node.js etc.) and use a small assertions library https://www.npmjs.com/package/ok6 Assertions are the ultimate "types", they can assert any property you can express in your programming language. The key is to be able to make them succinct so they don't obscure your main code. Yes they don't give you compile-time type-checking, which would be useful indeed. But as hinted at the excerpts above, in practice compile-time type-checking can slow you down. In most case all I need is a way to prove to myself that "if this function runs a set of tests-cases, then the assertions in it are truthful. And, if anybody calls this function with wrong types of arguments my assertions will tell me there is a problem. Often enough of a good thing is enough. You don't need the most sophisticated language and type-system to create robust programs quickly.
- ajkjk 4y ago
  I dunno, I love types as documentations of how things are supposed to be structured. But I also agree that assertions are great because they can express any condition in the language. I wonder sometimes if it's possible to have a type system that is "any possible assertion, as a type".
  - zem 4y ago
    in theory no, because you run into the halting problem. but even in practical terms it would impede your ability to do static type analysis long before that point.
    - ajkjk 4y ago
      I don't really care about static type analysis, in a sense: If I write an assertion `x is a number`, then fine, statically analyze that, that works today. If I write a hard-to-compute assertion like `x is Prime` (or `x is a counterexample to the Riemann Hypothesis`), then I don't care if the compiler can statically figure it out. It can simply require all callers to test that before calling this function, they assert `x is Prime` directly -- no analysis required. This way my function can be sure it is only called with prime numbers, but nobody has to write any logic in a typechecker that can figure out whether that's true for arbitrary inputs.
marmada 4y ago
I was investigating doing a typescript compiler speedup. I think the time might have finally arrived where we can use ML to generate a faster compiler in a language like Zig. Probably not for all portions of the compiler. But I think with some reinforcement learning, it ( ... might ...) be possible to automatically generate a robust Typescript parser in Zig.
plumeria 4y ago
Off-topic: is erroneous redundant in the sentence "reducing erroneous errors"?
- francasso 4y ago
  I don't think so, it should mean "reducing things that are currently reported as errors but shuouldn't"
  - plumeria 4y ago
    Ah yes, now it makes sense. Thank you.
ducaale 4y ago
Slightly related, there is TypeRunner[1] which aims to rewrite a subset of TypeScript in C++ [1] https://github.com/marcj/TypeRunner https://github.com/marcj/TypeRunner
zellyn 4y ago
The obvious solution is for Microsoft to hire him and support the Rust version as a first-class typescript compiler, no?
- namuol 4y ago
  Maybe as an experiment. If it means sacrificing velocity, I don’t think it would ultimately be “blessed” by MS. Perhaps a better way MS could contribute is to define a formal specification for the type checker, which would make efforts like this a lot easier. But these aren’t mutually exclusive.
  - xiphias2 4y ago
    If the compiler team already spends a significant time optimizing the compiler, the difference in velocity may be not that much. Achieving 100% compatibility on public code bases and switching is a great strategy at this point.
- epolanski 4y ago
  Two compilers would be mostly a source of issues and reduce TS velocity. All of that for what, slightly faster compilations? TS server is already fast enough for me to develop giant ts codebases in codespaces on 2 vCPUS with 4 GB of ram.
brundolf 4y ago
I've poked around in the TypeScript codebase just a little bit, and it's clear they've squeezed as much performance out of JS as they can (though really, you can tell that just by using the thing; it's impressively fast even though it's still "slow"). It's funny to see stuff like bit-flags in a JavaScript codebase I am a little surprised nobody at Microsoft has started working on an official native compiler (or maybe they have, and we just don't know about it yet). It does feel like they've hit a ceiling
- epolanski 4y ago
  One of the core goals of TS is that it has to run in browsers, a native compiler is just not a good goal and it is not in the roadmap. I think WASM is still not a good fit for a typechecking server.
  - brundolf 4y ago
    I wasn't aware of that as a core goal. Do you have a reference where it's stated? I also don't see why WASM wouldn't be a good fit for that
    - epolanski 4y ago
      The core metric for TypeScript's success was to be able to put an IDE in a browser able to run and compile TypeScript itself, which is what the TypeScript team did with the Monaco (and Visual Studio Code) editor in 2011.
      - brundolf 4y ago
        Monaco is just the GUI, it doesn't require language support (and in fact, the LSP that was pioneered alongside it means you don't even have to run the language support on the same machine, much less in the same codebase) Again- any references for this claim? Genuinely curious if this is part of TypeScript's known history, or if it's just speculation
  - pjmlp 4y ago
tayo42 4y ago
Why is typescript not able to do things in parallel? Why is that definitely the issue?
liminal 4y ago
Unrelated, but my Typescript wish is an (optional) way to express its types at runtime in a reflective way. Most of the time you don't want this, but sometimes... sometimes you wish you could use `keyof` or somesuch at runtime.
liminal 4y ago
I'm assuming they've looked into this, but what about parallelizing the existing code with web workers?
- steve_adams_86 4y ago
  The problem isn't runtime implementation details, but the static analysis and compilation steps which devs rely on constantly. The tooling has had major performance issues at times (though in my experience it has improved consistently) and the experience of writing TS could be dramatically improved by having a language like Rust or Go powering it at the core. If you meant parallel processes within the tooling, I suspect web workers wouldn't be the solution (they're part of the Web API). Node does expose a worker thread API, which they might already use in the typescript repo (I'm not familiar with any of their code).
v0idzer0 4y ago
Donny is doing god’s work. May the force be with him
pjmlp 4y ago
Just as information that seems to be missing from other comments, there is already a Microsoft compiler for TypeScript, which compiles to native code via C++, however it is only a subset and is targeted to IoT https://makecode.com/language https://makecode.com/language

237 points by mariuz 4y ago

> But there's a problem. Rewriting a library like TypeScript is extremely challenging. First, "absence of [a] specification" for TypeScript's behaviour. Donny's had to infer TypeScript's behaviour mostly test cases alone. IIRC the TypeScirpt leadership is of the position that the implementation is the specification, which is unfortunate, because it means that the bugs in the implementation are necessarily part of the specification as well.

Kuinox 4y ago
Specification are not immune to bugs.
- rk06 4y ago
  But you can use them as evidence that tsc is not working as it "should". Then go ahead and fix tsc Or spec or both as needed. Without the specs, there is no way to tell if tac is behaving as developers intended it
  - gfxgirl 4y ago
    It's not doing what I expect. Oh? Change the spec It's not doing what I expect. Oh? Change the code How is that different from now?
    - htkibar 4y ago
      Allows competing implementations and saves TS from MSFT
    - rk06 4y ago
      Right now: 1.It's not doing what I expect. 2.Is it intentional behavior or a bug? 3. Let's ask on github/reddit etc With Special: 1. It is not doing what I expect 2. Is this a spec issue or code issue? 3. Let's check spec. 4a. "Working as specified". Shit. Hope spec improves in future versions 4b. It is not working as spec says. Let's file a bug on tsc. And that is the difference.
      - steveklabnik 4y ago
        The real rub is in step 4: it is not always easy to determine what the spec means. And then you're basically back to "let's ask on github/reddit etc."
        yencabulator 4y ago
        Just because the comprehensiveness of something is on a spectrum is not a reason to just not bother and have none at all. Also, when done well a spec is a great way to learn & understand the language too. http://golang.org/ref/spec http://golang.org/ref/spec is so much more usable than Typescript's docs.
        steveklabnik 4y ago
        I agree that specifications are useful. I also think they're often presented as a panacea. It is important to acknowledge that they're a tool, with pros and cons, like any tool. The existence of a specification does not immediately mean that any and all ambiguities are resolved.
      - tshaddox 4y ago
        Still doesn’t seem like much a difference. If you run into 4a you still file an issue somewhere.
    - randomdata 4y ago
      If an implementation isn't doing what you expect, is it just an issue with that single implementation? A problem with all implementations? A problem with what you expect? If another implementation does do what you expect is it correct or just lucky? A spec resolves such discrepancies. Indeed, the spec may be where the problem is, but once the spec is corrected then implementations know what they must do. Absent of a spec, who knows?
  - kristiandupont 4y ago
    And such a spec can be 1) a reference implementation, 2) written in natural language, or 3) written as formal semantics #1 is the same as the current situation. With #2, you might get a bit closer but natural language has many ambiguities and idiosyncrasies that can render it useless in various situations. You can try to define formal semantics, but that is not only hard to create, it's also hard to translate into an implementation, adding significant overhead in both parts of the process. And there can still be bugs, and there can still be edge cases with undefined behavior.
  - Kuinox 4y ago
    Code, called implementation here, is a specification written in a certain grammar that is executed. You want 'evidence' tsc is not working as it should, by writting a duplicated less specifying specification that would sometimes conflict with the typescript one. It make me remember the people that wanted to specify everything and generate implementation from it: that called a programming language.
    - rk06 4y ago
      Code is meant for machine. Specification is meant for compiler author. Writing one of them is easier than otht
      - randomdata 4y ago
        They are not mutually exclusive. Code identifies a class of language. Specification identifies something that is conveyed through language. Specification can be provided in code. In fact, while poorly named, what we call testing is actually specification. A test suite specifies for software authors what a program is intended to do and how it is intended to be used. Of course, as a nice side bonus, because the specification is delivered in code the documentation can be validated for truthfulness against an implementation by machine.
- remexre 4y ago
  But at least formal methods tools can be applied directly to the specification, to show properties like "well, if you carve out this subset of TypeScript, you have something you can safely eval()" or "you can never have type errors with this usage of this construct in this way."
  - zokier 4y ago
    > But at least formal methods tools can be applied directly to the specification What makes you think that? For example for C language it has been significant effort to translate the specification to something analyzable by tools
  - brundolf 4y ago
    > "well, if you carve out this subset of TypeScript, you have something you can safely eval()" or "you can never have type errors with this usage of this construct in this way." This kind of certainty is very at odds with TypeScript's design philosophy and the way it (necessarily) works. Because it's overlaid on JavaScript, and JavaScript is wildly dynamic (Proxies, prototype modification, etc), almost nothing is known with absolute certainty. To deal with that, TypeScript makes a lot of "optimistic" assumptions, but holds them lightly (i.e. its assumptions may cause type checks to pass when they shouldn't, but it's not going to lean on them for anything that could really blow up if they're wrong) IMO it strikes a very good balance given the constraints - not criticizing the designers at all - but TypeScript has to be treated differently than other statically typed languages might be, because nearly everything it knows/says is just a "probably"
- namuol 4y ago
  Neither are programs, tests, or type definitions, but these are generally still valuable things to have.
ape4 4y ago
Maybe the first step should be writing down a spec of TypeScript.
- progx 4y ago
  Boring! That is the problem, nobody want to do such boring things.
  - yencabulator 4y ago
    Nobody except Microsoft is able to write a spec for Typescript. Microsoft has already declared that the implementation is it, any spec a random person writes will just be ignored.
  - ansraliant 4y ago
  - epolanski 4y ago
    It's not about boredom, it's simply an implementation derived language as are Haskell or PHP or Python. Having a spec is really valuable only if you intend to target different environments, that's a non-goal for TS it simply transpiles, rather than compiles, into another programming language. It's also bound to follow each new JS feature and change.
    - randomdata 4y ago
      The value of a spec is having something that resolves how to deal with quirks discovered in an implementation. Absent of a spec, any implementation quirks become a part of the language. Which is okay as long as you only ever have one implementation, but when you want to, say, reimplement Typescript in Rust then you have to duplicate all of the quirks that came before, even in cases where nobody has yet realized that the quirk exists. Failure to do so means that your implementation won't be compatible with the canonical implementation. At the other end of the spectrum, the Go project maintains two independent compiler implementations to ensure that the spec is implementable and to see that a single implementation, with all of its quirks, doesn't end up defining the language. That puts a lot of confidence in the spec to guide additional implementations, but, of course, is a lot of added work. Tradeoffs, as always.
miohtama 4y ago
For the sake of being fair for other programming languages, what are the languages with proper specification? C, C++, JavaScript, others?
- randomdata 4y ago
  Go is staunch about its specification. To the point that the Go team maintains two independent compiler implementations to ensure that "features"/bugs in one implementation doesn't end up defining the standard and to ensure that a compiler can be implemented from the spec.
  - tristan957 4y ago
    Is the other compiler you are referring to gcc-go?
  - recov 4y ago
    There's actually 3 now from google afaik - gc - gccgo - gollvm [0] [0] https://go.googlesource.com/gollvm/ https://go.googlesource.com/gollvm/
- munificent 4y ago
  Dart has a complete specification. It tends to lag somewhat behind the most recent features (which are documented in separate language proposals), but we keep it as current as we can. Maintaining a language specification is a lot more work than people realize, and a job that few have the skills to do well.
- xchkr1337 4y ago
  In my opinion the presence of undefined behavior in the C/C++ specification makes it no better than the "specification by implementation" approach
  - spac 4y ago
    without a specification you can never tell with certainty if there's a bug
the_gipsy 4y ago
That's one way to keep tight control.

noahtallen 4y ago

This reminds me a bit of Rome, which aims to re-implement most parts of a JS tool chain without requiring dozens of different, misaligned dependencies. They also happen to be writing it in Rust, though they haven’t finished the bundler portion yet. I wonder if there’s an opportunity for collaboration here! https://rome.tools/blog/2021/09/21/rome-will-be-rewritten-in-rust/ https://rome.tools/blog/2021/09/21/rome-will-be-rewritten-in...

clintonb 4y ago

If successful, this project would save an immense amount of person and compute hours.

mlajtos 4y ago

Could new AI tools help with such endeavor?

tiberriver256 4y ago

> "Honestly, I'm still not sure if open-source is sustainable without companies like Vercel." Honestly, I'm still not sure if people could breathe without things like air.

intelVISA 4y ago

mgomez 4y ago

At the beginning of this year, kdy1 (Donny) said he'd be switching to Go after initially trying things out in Rust [1][2], I guess he decided to go through with Rust after all? Or is this something slightly different? [1] https://kdy1.dev/posts/2022/1/tsc-go https://kdy1.dev/posts/2022/1/tsc-go [2] https://news.ycombinator.com/item?id=30074414 https://news.ycombinator.com/item?id=30074414

munbun 4y ago
https://kdy1.dev/posts/2022/10/open-sourcing-stc#ix27ll-switch-back-to-the-rust-version https://kdy1.dev/posts/2022/10/open-sourcing-stc#ix27ll-swit...
clarkdave 4y ago
From the interview (it's way down the page): > The most challenging thing is the absence of the specification. I had to infer everything just from test cases. > The second thing is the velocity of tsc. It's way too fast, and I was unsure if I could follow it up. So I decided to use semi-automated translation and selected Golang. > But it was too boring, and more importantly, my programming ability has improved enough to follow tsc only by myself.
rob74 4y ago
Looks like he moved back to Rust sometime between January and October, at which point he also changed the name to stc: https://kdy1.dev/posts/2022/10/open-sourcing-stc https://kdy1.dev/posts/2022/10/open-sourcing-stc
CPUTranslator 4y ago
From the main post and other comments it seems like it was for personal reasons, rather than pragmatic ones. They said they didn’t know how fast the Go implementation would be, but felt better about the original Rust one (I’m assuming before this really became a problem). I think the reasons they switched are pretty weak, but justified at the same time. Kdy1 looks like more of a Rust person (their GitHub has more active Rust repos than Go), so this should’ve been the choice from the beginning. Going with the comfortable choice over the “pragmatic” one is almost always the best option if you’re the only contributor (or plan to be for a while).

muglug 4y ago

If you're just porting the code to a new language, the process can be significantly faster than writing it from scratch. But if you're also re-interpreting the code, which seems to be Donny's approach, you can get bogged down with the discrepancies. The more the two diverge, the harder it is to debug discrepancies and keep track of updates in the upstream code. I'm also porting a static analysis tool from an interpreted language to Rust, but it's a fairly straightforward port and I've been able to stick to schedule pretty well.

tuyiown 4y ago
There is also the scenario that tsc is really not that good, and the discrepancies are not a problem enough to avoid potential stc benefits. This can eventually phase out tsc in typescript altogether, the core typescript team making stc the official implementation. This is especially credible with rust having good support with wasm, keeping the js runtime compatibility of tsc.
- cardanome 4y ago
  Would that be a wise career choice for the people currently working on tsc? Sounds more like they should purposely break compatibility form time to time to prevent Donny from catching up.
  - rk06 4y ago
    Should people really make their professional decisions based on future career choice? I think not. Besides, such malicious action can easily be counterproductive. People can just stop using tsc and switch completely to Rust version. Then, it would effectively make any development on tsc useless.

lizen_one 4y ago

Rust is a "new" language and many packages from other languages get reimplemented in it. This is similar to Julia. Unfortunately, I had the experience that many Julia packages are not of high quality, not maintained, or do not run any more on the newest version. How is this in Rust?

vardump 4y ago
I don't know how it's in general, but the few Rust packages I checked seem to be of very high quality.
kibwen 4y ago
Like every package repository (or human endeavor in general) it follows Sturgeon's Law: 80% of everything is crap. That said, there's 100k crates on crates.io, and many of them are fantastic (well-supported, actively developed, documented, etc.). For a new user, understanding which are high-quality is a daunting task, and is expedited by just asking an experienced person for specific recommendations.
- civopsec 4y ago
  Didn’t expect to see such harsh criticism of the Rust ecosystem by this user.
  - steveklabnik 4y ago
    Contrary to the memes, a lot of long running Rust users are happy to tell you about where Rust is deficient.
    - civopsec 4y ago
      Not what I meant.
      - cstrahan 4y ago
        What did you mean?

bluelightning2k 4y ago

Wow. What a huge undertaking!

csande17 4y ago

Very exciting! I can only imagine how many doctoral research projects' worth of work this represents.

nnurmanov 4y ago

Shouldn’t be an ML model that rewrites from one language to another? ChatGPT comes to mind

protortyp 4y ago

I hope he and others succeed! I've been using chatgpt to translate Python/TS into Rust/C++ for some smaller projects of mine in the past week. And with some tweaking it works beautifully. Sometimes it makes up crates that do not exist though.

zozbot234 4y ago
> Sometimes it makes up crates that do not exist though. You're missing the point, these are ideas for further work.

thefaux 4y ago

This is incredibly impressive but I also simultaneously find it kind of depressing. In my not so humble opinion, typescript is already way too complicated and getting worse thanks to its velocity. A better approach would be to kill typescript's velocity by identifying a stable subset of the language that provides the vast majority of its benefits and optimize the heck out of a parser for that smaller language. This approach would be both easier and more sustainable since it would cap the language complexity making it easier to maintain and/or write alternative implementations.

serverlessmania 4y ago
At this point, TS is becoming C# V2
Zaskoda 4y ago
I find this comment validating. Most of the languages I've used start to click and feel good after a while - but it feels like this just won't happen for me with typescript. I'm clearly missing some fundamentals and I'm not even sure what it is I need to go study.
- evandwight 4y ago
  What doesn't click? I've found it pretty intuitive, though I've explicitly avoided making complicated types.
  - randomdata 4y ago
    The type erasure. You get this nicely expressive type system that can encode all sorts of useful metadata, and then you have to recreate it all again by hand in 'Javascript' to access it at runtime. Say I have `type Foo = 'bar' | 'baz'` and want to validate that an input string is within the set of Foo. Why can't I just enumerate the type to determine the valid values at runtime?
    - scotty79 4y ago

thenoblesunfish 4y ago

If there was ever a HN headline I'd suspect of being machine generated, this would be it :D

galaxyLogic 4y ago

From the article some reasons not to use TypeScript: "... in VSCode, the red lines and warnings take a long time to update on large projects. ... absence of [a] specification" for TypeScript's behaviour. Donny's had to infer TypeScript's behaviour mostly test cases alone. The pure size of TypeScript is daunting. It's had ten years to iterate, grow and add features. " --- Therefore for practical reasons, I stick with plain JavaScript (on Node.js etc.) and use a small assertions library https://www.npmjs.com/package/ok6 Assertions are the ultimate "types", they can assert any property you can express in your programming language. The key is to be able to make them succinct so they don't obscure your main code. Yes they don't give you compile-time type-checking, which would be useful indeed. But as hinted at the excerpts above, in practice compile-time type-checking can slow you down. In most case all I need is a way to prove to myself that "if this function runs a set of tests-cases, then the assertions in it are truthful. And, if anybody calls this function with wrong types of arguments my assertions will tell me there is a problem. Often enough of a good thing is enough. You don't need the most sophisticated language and type-system to create robust programs quickly.

ajkjk 4y ago
I dunno, I love types as documentations of how things are supposed to be structured. But I also agree that assertions are great because they can express any condition in the language. I wonder sometimes if it's possible to have a type system that is "any possible assertion, as a type".
- zem 4y ago
  in theory no, because you run into the halting problem. but even in practical terms it would impede your ability to do static type analysis long before that point.
  - ajkjk 4y ago
    I don't really care about static type analysis, in a sense: If I write an assertion `x is a number`, then fine, statically analyze that, that works today. If I write a hard-to-compute assertion like `x is Prime` (or `x is a counterexample to the Riemann Hypothesis`), then I don't care if the compiler can statically figure it out. It can simply require all callers to test that before calling this function, they assert `x is Prime` directly -- no analysis required. This way my function can be sure it is only called with prime numbers, but nobody has to write any logic in a typechecker that can figure out whether that's true for arbitrary inputs.

marmada 4y ago

I was investigating doing a typescript compiler speedup. I think the time might have finally arrived where we can use ML to generate a faster compiler in a language like Zig. Probably not for all portions of the compiler. But I think with some reinforcement learning, it ( ... might ...) be possible to automatically generate a robust Typescript parser in Zig.

plumeria 4y ago

Off-topic: is erroneous redundant in the sentence "reducing erroneous errors"?

francasso 4y ago
I don't think so, it should mean "reducing things that are currently reported as errors but shuouldn't"
- plumeria 4y ago
  Ah yes, now it makes sense. Thank you.

ducaale 4y ago

Slightly related, there is TypeRunner[1] which aims to rewrite a subset of TypeScript in C++ [1] https://github.com/marcj/TypeRunner https://github.com/marcj/TypeRunner

zellyn 4y ago

The obvious solution is for Microsoft to hire him and support the Rust version as a first-class typescript compiler, no?

namuol 4y ago
Maybe as an experiment. If it means sacrificing velocity, I don’t think it would ultimately be “blessed” by MS. Perhaps a better way MS could contribute is to define a formal specification for the type checker, which would make efforts like this a lot easier. But these aren’t mutually exclusive.
- xiphias2 4y ago
  If the compiler team already spends a significant time optimizing the compiler, the difference in velocity may be not that much. Achieving 100% compatibility on public code bases and switching is a great strategy at this point.
epolanski 4y ago
Two compilers would be mostly a source of issues and reduce TS velocity. All of that for what, slightly faster compilations? TS server is already fast enough for me to develop giant ts codebases in codespaces on 2 vCPUS with 4 GB of ram.

brundolf 4y ago

I've poked around in the TypeScript codebase just a little bit, and it's clear they've squeezed as much performance out of JS as they can (though really, you can tell that just by using the thing; it's impressively fast even though it's still "slow"). It's funny to see stuff like bit-flags in a JavaScript codebase I am a little surprised nobody at Microsoft has started working on an official native compiler (or maybe they have, and we just don't know about it yet). It does feel like they've hit a ceiling

epolanski 4y ago
One of the core goals of TS is that it has to run in browsers, a native compiler is just not a good goal and it is not in the roadmap. I think WASM is still not a good fit for a typechecking server.
- brundolf 4y ago
  I wasn't aware of that as a core goal. Do you have a reference where it's stated? I also don't see why WASM wouldn't be a good fit for that
  - epolanski 4y ago
    The core metric for TypeScript's success was to be able to put an IDE in a browser able to run and compile TypeScript itself, which is what the TypeScript team did with the Monaco (and Visual Studio Code) editor in 2011.
    - brundolf 4y ago
      Monaco is just the GUI, it doesn't require language support (and in fact, the LSP that was pioneered alongside it means you don't even have to run the language support on the same machine, much less in the same codebase) Again- any references for this claim? Genuinely curious if this is part of TypeScript's known history, or if it's just speculation
- pjmlp 4y ago

tayo42 4y ago

Why is typescript not able to do things in parallel? Why is that definitely the issue?

liminal 4y ago

Unrelated, but my Typescript wish is an (optional) way to express its types at runtime in a reflective way. Most of the time you don't want this, but sometimes... sometimes you wish you could use `keyof` or somesuch at runtime.

liminal 4y ago

I'm assuming they've looked into this, but what about parallelizing the existing code with web workers?

steve_adams_86 4y ago
The problem isn't runtime implementation details, but the static analysis and compilation steps which devs rely on constantly. The tooling has had major performance issues at times (though in my experience it has improved consistently) and the experience of writing TS could be dramatically improved by having a language like Rust or Go powering it at the core. If you meant parallel processes within the tooling, I suspect web workers wouldn't be the solution (they're part of the Web API). Node does expose a worker thread API, which they might already use in the typescript repo (I'm not familiar with any of their code).

v0idzer0 4y ago

Donny is doing god’s work. May the force be with him

pjmlp 4y ago

Just as information that seems to be missing from other comments, there is already a Microsoft compiler for TypeScript, which compiles to native code via C++, however it is only a subset and is targeted to IoT https://makecode.com/language https://makecode.com/language