7 ms·

Speeding up the JavaScript ecosystem – Isolated Declarations

45 points by realshadow 2y ago

viraptor 2y ago
I feel like the point is skipped in the post: if you ship TS sources, it means you need the TS compiler on the installed side, right? And specifically the right TS version for each installed package... which in some cases may mean conflicting requirements? (If you own code is in JS, that is)
- maxloh 2y ago
  The TypeScript compiler is, in most or even all cases, backward compatible. You just need to have the latest version installed.
  - viraptor 2y ago
    They may not be frequent, but they do happen https://www.typescriptlang.org/docs/handbook/release-notes/typescript-5-0.html#breaking-changes-and-deprecations https://www.typescriptlang.org/docs/handbook/release-notes/t... And given enough public modules, we will find some conflicts in the dependency graph. YOLO just use the latest version is rarely a full solution.
- mmoskal 2y ago
  You would be probably better off just running esbuild or similar - it will happily strip types from any version of TS.
- mhagemeister 2y ago
  Author here. True, I think I should've worded it better. I work for Deno and it being a JS runtime that is able to run TS natively makes this less of an issue. The problem that a dependency needs a specific TS version hasn't come up so far. But that might be because Deno users tend to update to the latest version pretty frequently as we haven't done any breaking changes so far.
rtsao 2y ago
I've heard of `isolatedDeclarations` in TS 5.5, but this post left me with more questions than answers because it conflates so many distinct things (TS 5.5, Deno, JSR). I'll try my best to break it down succinctly: 1) Historically, .d.ts generation is slow because it requires the full TypeScript type checker in order to correctly handle all cases (e.g. exported functions with inferred return types) 2) However, if types were to be fully explicit, then .d.ts generation could be performed without a full type checker (i.e. by a faster compiler via mere syntax transformation) 3) The isolatedDeclarations flag causes the TS compiler to trigger errors when any exports would require inference in order to generate .d.ts 4) Thus, the isolatedDeclarations flag can be used to guarantee compatibility with faster, simpler .d.ts generation using syntax-based transformation 5) A consequence of simpler .d.ts generation is that it can be trivially done in parallel, which can be helpful for large monorepos
- deleted 2y ago
  [deleted]
- rezonant 2y ago
  This is all fully correct, which is why isolatedDeclarations is a nice feature... for authors of large Typescript projects. The tradeoff is to be more explicit on your public API surface so that you can reduce build times. That's great. It really doesn't do the things the author thinks it does.
- mhagemeister 2y ago
  Author here. That's good feedback. Looks like that part of my article is a bit confusing and I've updated the phrasing a little to hopefully make it less so. You're spot on with all of your points. The isolated declarations feature forces your code to be written in a way that creating the .d.ts files is a mere exercise of stripping syntax and can be done easily without invoking the tsc compiler. For runtimes like Deno which support running TS natively (disclaimer: I work for Deno) you never had to care about creating .d.ts files when publishing your package to any of the Deno registries: previously /x, now JSR. In the background though we've always tried to feed the tsc compiler in Deno something like .d.ts files though as it's quicker for type checking purposes.
austin-cheney 2y ago
whilst reducing the time to create type definition files from minutes, sometimes even hours, down to less than a second. WTF. What kind of project would take long to compile from TS? Windows 11 (if it were TS)? I had a long time personal project that was several megs of code with several hundred types/interfaces. It took 13 seconds to compile. If your application takes hours to compile and is less than a petabyte of source code you have some catastrophic mistakes in your approach.
- vinnymac 2y ago
  I think they are referring to the manual creation of type definitions. As they mention it in the article.
  - rezonant 2y ago
    But then they assume the code is authored in Typescript... Who is authoring in Typescript and then simultaneously writing their d.ts manually?? It makes no sense.
    - austin-cheney 2y ago
      I write all my d.ts files manually. Maybe that’s the problem: people don’t know how to write TS, expect some software to do it for them, and then end up waiting hours on a build that should take 10 seconds.
      - rezonant 2y ago
        This confuses me. You either write the d.ts manually before build, which only makes sense if you wrote the original code in JavaScript (not Typescript) or you write the source in Typescript and enable the declaration option in your tsconfig and the d.ts pops out in seconds along with your final JS output. Unless you are using transpileOnly, generating declarations will not make a few second build become a few hours-- Typescript is already type checking your code.
seniorsassycat 2y ago
The blog doesn't explain how to take advantage of isolated modules without using JSR, or how JSR transforms published packages. How would a project configure node and typescript to use a npm module that has exported .ts files?
- streptomycin 2y ago
  You wouldn't, you would use JSR.
  - crooked-v 2y ago
    I despise this recurring trend in the JS-adjacent world of "cutting edge" people making their published packages intentionally incompatible with what everyone else is doing.
    - rezonant 2y ago
      Well, if they want no one to use their libraries, then fine. You would think that as a person publishing library code to a public registry, you would be incentivized to ensure your code is broadly usable. At least in general. I know this is in the context of JSR where this isn't an issue, but I suppose some people just want to just use a registry as a personal code repository. Id just hope they don't pollute the top level namespace in that case. Who am I kidding, sadly for NPM that ship sailed years ago :*-(
pzmarzly 2y ago
From what I understand, the author is persuading library authors to stop compiling TS->JS, and instead ship your TS files to the users over npm. I like the idea of it, but will it actually work in practice? What if some dependency only compiles with some version of tsc? What if it needs different tsconfig-s?
- rezonant 2y ago
  > but will it actually work in practice? No. It seems like they are assuming the user is using JSR, which handles generation of JS and d.ts for you. But then none of this would even be a problem. The NPM registry doesn't do any of that. You could absolutely publish only .ts files to NPM and then use a post install script to generate JS and d.ts files, but that is a pretty terrible idea. Even if the process is super fast, it's still an operation done potentially millions of times, versus an operation which can happen once at publish time. You could also assume the user will have their own mechanism for compiling TS or consuming TS directly, but this is even worse.
  - afavour 2y ago
    You can specify different exports based on environment in the package.json so you could ship TS files… alongside JS files, so it feels a little redundant.
    - rezonant 2y ago
      Yes, I recommend shipping TS source alongside compiled JS+d.ts even if you don't support a TS native environment, because you then have the potential of a source mapped debugging experience within that library (though currently doing that is a bit painful in some debuggers).
      - stevenpetryk 2y ago
        That can be nice, but do keep in mind that shipping more files leads to a larger tarball downloaded from npm.
ericyd 2y ago
This doesn't really seem to address my most common packaging pain points in the JS/TS ecosystem (which is the inability to easily use ESM in legacy commonjs applications) but maybe I'm just working in the wrong area? I think the author seriously exaggerates the difficulty of creating type declaration files. It's literally one boolean setting in your tsconfig. I wish they would have given more time to proving the alleged pain points with some examples rather than just telling us how great this new feature is. I personally don't see how this changes my life at all, and I'm a full time JS/TS dev.
- seniorsassycat 2y ago
  The article suggests that you ship typescript so the app would compile down to esm or cjs. Node 22 has an experimental flag that supports requiring esm as long as there's no top-level await
  - mhagemeister 2y ago
    Author here. I should've worded that better in the article. The takeaway should not be to publish only TS sources to npm. The whole npm ecosystem is based around the assumption that you ship .js files and doing anything else would break it. I've updated the wording to hopefully make it less confusing.
matthberg 2y ago
The site's down for me, here's an Internet Archive link: https://web.archive.org/web/20240706220437/https://marvinh.dev/blog/speeding-up-javascript-ecosystem-part-10/ https://web.archive.org/web/20240706220437/https://marvinh.d... I've really enjoyed this author's series on optimization techniques explored through non-trivial yet still small case studies, the other articles linked near the top are all worth checking out too!
- mhagemeister 2y ago
  Oh no, I just woke up and saw this comment. It seems to be up again. Apologies for the inconvenience. Looks like the HN hug of death is real! Happy to hear that you like the series!
rezonant 2y ago
Please do not start publishing raw Typescript into the NPM registry thinking that all the reasons we have not done that in the past do not continue to apply. I shouldn't have to say this, but at least 27 people upvoted this article, so here we are. isolatedDeclarations is most useful for large codebases and other situations where performing a full Typescript compilation is prohibitively expensive. This is not the vast majority of cases. It is the declarations analog of transpileOnly, which skips all type checks and simply strips Typescript's special syntax. The author seems to assume use of JSR, which is an alternative to the NPM registry which automatically compiles Typescript. Not being terribly familiar with it, it's possible this is more relevant to users of that registry, but it's not clear why this is an issue when the registry itself is handling transpiling and declaration generation. EDIT: To be clear, I think it's totally fine to publish your TS source in addition to your JS and d.ts outputs, just don't publish TS only packages, please.
- seniorsassycat 2y ago
  Id love it of npm added support for additional artifacts — separate tarballs for source, code, binaries, source maps, docs etc
  - rezonant 2y ago
    Fair, but if you don't fall into the pit of despair by consuming micro libraries (ie "is-true" or "leftpad"), the overhead of having source and source maps present on a developer environment or even a server deployment is not terrible. If you are bundling for web delivery, those files will be omitted automatically by nature of the bundling process. While it would be nice to have native support for package components, personally I think the best path is to ship all of it (save perhaps documentation, where there's currently no ergonomic way for a consumer to make use of it) in the main package. The above is assuming "docs" mean some kind of external documentation, you absolutely should not strip out your docblocks when you package your code. Ditto for minification: Just don't. If the end user needs to optimize for size, they will be minifying anyway, all packaging minification does is create painful debugging and code reading experiences for consumers.
- 2y ago
stevenpetryk 2y ago
I believe the intention behind isolatedModules is to make compilation faster by forcing you to define types that require no inference from other files (thus, being isolated). Not… whatever this is.
rty32 2y ago
...wait. Is the entire article based on the assumption that everyone has Typescript installed in every project? Otherwise, how would publish Typescript files work at all? > We only ever ship build artifacts, the compiled JS and the relevant .d.ts files in a package. Isn't this because anyone, regardless of whether they have use the project in JS or TS, can import the library without caring out how the library code is implemented (in JS or TS)? It is not perfect but has worked for almost everyone. How would the author's project layout help people who only ever write code in JavaScript and never installed the Typescript package? Does Typescript have to be a dev dependency of any project that has an upstream package written in Typescript? That seems a bad idea. I write almost all my JavaScript projects with Typescript now (it's a bad sentence but you know what I mean), but I don't think we should ever make the life harder for those who only write in JavaScript.
- mmoskal 2y ago
  I think most bundlers can handle TS nowadays so you don't actually need tsc.
  - rlt 2y ago
    Technically you still need tsc or an alternative TypeScript compiler like swc, but bundlers sometimes include them in their own dependencies so you don’t have to think about it.
29athrowaway 2y ago
JavaScript runtimes are full of optimizations that execute conditionally on specific scenarios. If you don't follow the rewarding path of optimization you then go to the deeply unhappy path of deoptimization. It's easy to get started with JavaScript, but to squeeze every bit of performance in JavaScript will quickly get you into a very complex topics that are not documented and change may change from version to version as they are often not "contracts". The only way to learn those obscure topics is by reading the source code for your JavaScript implementation, their bug tracker, etc. This means if you have not been reading C++ or executed a non-JS profiler to make your JS faster then you probably have not gone far enough. There are unexpected things that you would never have suspected if you had not read the implementation.
_jackwink 2y ago
This article is kind of a messy read, isolated declarations feels like it's being used as a mcguffin to advertise the jsr registry. Both may be neat things on their own, but I feel the connection is a bit misleading. Isolated declarations should allow parallelization and faster type checking with tooling that supports it. This shouldn't be changing your build/release process or what you export in your package.json. If you also happen to use the jsr registry to publish your package, it sounds like you can update your package.json to export TS files and they'll compile/inject the JS into your release artifacts on publish. Not sure if this feature requires isolated declarations though.
- mhagemeister 2y ago
  Author here. Thanks for sharing that feedback. I think I missed the mark by being so used to working with runtimes that natively run TS files, that I forgot that this is not the default for everyone. The article was mostly written from that perspective. For Node users the isolated declaration feature won't do much other than speed up the creation of the .d.ts files a little. For runtimes that can run TS natively like Deno (disclaimer: I work for Deno) the benefit is much more apparent as you always want the original TS sources, but can generate .d.ts files on the fly for faster type checking. EDIT: Formatting
tlarkworthy 2y ago
I don't know what an isolated declaration is. Do we write TS source differently?

45 points by realshadow 2y ago

I feel like the point is skipped in the post: if you ship TS sources, it means you need the TS compiler on the installed side, right? And specifically the right TS version for each installed package... which in some cases may mean conflicting requirements? (If you own code is in JS, that is)

maxloh 2y ago
The TypeScript compiler is, in most or even all cases, backward compatible. You just need to have the latest version installed.
- viraptor 2y ago
  They may not be frequent, but they do happen https://www.typescriptlang.org/docs/handbook/release-notes/typescript-5-0.html#breaking-changes-and-deprecations https://www.typescriptlang.org/docs/handbook/release-notes/t... And given enough public modules, we will find some conflicts in the dependency graph. YOLO just use the latest version is rarely a full solution.
mmoskal 2y ago
You would be probably better off just running esbuild or similar - it will happily strip types from any version of TS.
mhagemeister 2y ago
Author here. True, I think I should've worded it better. I work for Deno and it being a JS runtime that is able to run TS natively makes this less of an issue. The problem that a dependency needs a specific TS version hasn't come up so far. But that might be because Deno users tend to update to the latest version pretty frequently as we haven't done any breaking changes so far.

rtsao 2y ago

I've heard of `isolatedDeclarations` in TS 5.5, but this post left me with more questions than answers because it conflates so many distinct things (TS 5.5, Deno, JSR). I'll try my best to break it down succinctly: 1) Historically, .d.ts generation is slow because it requires the full TypeScript type checker in order to correctly handle all cases (e.g. exported functions with inferred return types) 2) However, if types were to be fully explicit, then .d.ts generation could be performed without a full type checker (i.e. by a faster compiler via mere syntax transformation) 3) The isolatedDeclarations flag causes the TS compiler to trigger errors when any exports would require inference in order to generate .d.ts 4) Thus, the isolatedDeclarations flag can be used to guarantee compatibility with faster, simpler .d.ts generation using syntax-based transformation 5) A consequence of simpler .d.ts generation is that it can be trivially done in parallel, which can be helpful for large monorepos

deleted 2y ago
[deleted]
rezonant 2y ago
This is all fully correct, which is why isolatedDeclarations is a nice feature... for authors of large Typescript projects. The tradeoff is to be more explicit on your public API surface so that you can reduce build times. That's great. It really doesn't do the things the author thinks it does.
mhagemeister 2y ago
Author here. That's good feedback. Looks like that part of my article is a bit confusing and I've updated the phrasing a little to hopefully make it less so. You're spot on with all of your points. The isolated declarations feature forces your code to be written in a way that creating the .d.ts files is a mere exercise of stripping syntax and can be done easily without invoking the tsc compiler. For runtimes like Deno which support running TS natively (disclaimer: I work for Deno) you never had to care about creating .d.ts files when publishing your package to any of the Deno registries: previously /x, now JSR. In the background though we've always tried to feed the tsc compiler in Deno something like .d.ts files though as it's quicker for type checking purposes.

austin-cheney 2y ago

whilst reducing the time to create type definition files from minutes, sometimes even hours, down to less than a second. WTF. What kind of project would take long to compile from TS? Windows 11 (if it were TS)? I had a long time personal project that was several megs of code with several hundred types/interfaces. It took 13 seconds to compile. If your application takes hours to compile and is less than a petabyte of source code you have some catastrophic mistakes in your approach.

vinnymac 2y ago
I think they are referring to the manual creation of type definitions. As they mention it in the article.
- rezonant 2y ago
  But then they assume the code is authored in Typescript... Who is authoring in Typescript and then simultaneously writing their d.ts manually?? It makes no sense.
  - austin-cheney 2y ago
    I write all my d.ts files manually. Maybe that’s the problem: people don’t know how to write TS, expect some software to do it for them, and then end up waiting hours on a build that should take 10 seconds.
    - rezonant 2y ago
      This confuses me. You either write the d.ts manually before build, which only makes sense if you wrote the original code in JavaScript (not Typescript) or you write the source in Typescript and enable the declaration option in your tsconfig and the d.ts pops out in seconds along with your final JS output. Unless you are using transpileOnly, generating declarations will not make a few second build become a few hours-- Typescript is already type checking your code.

seniorsassycat 2y ago

The blog doesn't explain how to take advantage of isolated modules without using JSR, or how JSR transforms published packages. How would a project configure node and typescript to use a npm module that has exported .ts files?

streptomycin 2y ago
You wouldn't, you would use JSR.
- crooked-v 2y ago
  I despise this recurring trend in the JS-adjacent world of "cutting edge" people making their published packages intentionally incompatible with what everyone else is doing.
  - rezonant 2y ago
    Well, if they want no one to use their libraries, then fine. You would think that as a person publishing library code to a public registry, you would be incentivized to ensure your code is broadly usable. At least in general. I know this is in the context of JSR where this isn't an issue, but I suppose some people just want to just use a registry as a personal code repository. Id just hope they don't pollute the top level namespace in that case. Who am I kidding, sadly for NPM that ship sailed years ago :*-(

pzmarzly 2y ago

From what I understand, the author is persuading library authors to stop compiling TS->JS, and instead ship your TS files to the users over npm. I like the idea of it, but will it actually work in practice? What if some dependency only compiles with some version of tsc? What if it needs different tsconfig-s?

rezonant 2y ago
> but will it actually work in practice? No. It seems like they are assuming the user is using JSR, which handles generation of JS and d.ts for you. But then none of this would even be a problem. The NPM registry doesn't do any of that. You could absolutely publish only .ts files to NPM and then use a post install script to generate JS and d.ts files, but that is a pretty terrible idea. Even if the process is super fast, it's still an operation done potentially millions of times, versus an operation which can happen once at publish time. You could also assume the user will have their own mechanism for compiling TS or consuming TS directly, but this is even worse.
- afavour 2y ago
  You can specify different exports based on environment in the package.json so you could ship TS files… alongside JS files, so it feels a little redundant.
  - rezonant 2y ago
    Yes, I recommend shipping TS source alongside compiled JS+d.ts even if you don't support a TS native environment, because you then have the potential of a source mapped debugging experience within that library (though currently doing that is a bit painful in some debuggers).
    - stevenpetryk 2y ago
      That can be nice, but do keep in mind that shipping more files leads to a larger tarball downloaded from npm.

ericyd 2y ago

This doesn't really seem to address my most common packaging pain points in the JS/TS ecosystem (which is the inability to easily use ESM in legacy commonjs applications) but maybe I'm just working in the wrong area? I think the author seriously exaggerates the difficulty of creating type declaration files. It's literally one boolean setting in your tsconfig. I wish they would have given more time to proving the alleged pain points with some examples rather than just telling us how great this new feature is. I personally don't see how this changes my life at all, and I'm a full time JS/TS dev.

seniorsassycat 2y ago
The article suggests that you ship typescript so the app would compile down to esm or cjs. Node 22 has an experimental flag that supports requiring esm as long as there's no top-level await
- mhagemeister 2y ago
  Author here. I should've worded that better in the article. The takeaway should not be to publish only TS sources to npm. The whole npm ecosystem is based around the assumption that you ship .js files and doing anything else would break it. I've updated the wording to hopefully make it less confusing.

matthberg 2y ago

The site's down for me, here's an Internet Archive link: https://web.archive.org/web/20240706220437/https://marvinh.dev/blog/speeding-up-javascript-ecosystem-part-10/ https://web.archive.org/web/20240706220437/https://marvinh.d... I've really enjoyed this author's series on optimization techniques explored through non-trivial yet still small case studies, the other articles linked near the top are all worth checking out too!

mhagemeister 2y ago
Oh no, I just woke up and saw this comment. It seems to be up again. Apologies for the inconvenience. Looks like the HN hug of death is real! Happy to hear that you like the series!

rezonant 2y ago

Please do not start publishing raw Typescript into the NPM registry thinking that all the reasons we have not done that in the past do not continue to apply. I shouldn't have to say this, but at least 27 people upvoted this article, so here we are. isolatedDeclarations is most useful for large codebases and other situations where performing a full Typescript compilation is prohibitively expensive. This is not the vast majority of cases. It is the declarations analog of transpileOnly, which skips all type checks and simply strips Typescript's special syntax. The author seems to assume use of JSR, which is an alternative to the NPM registry which automatically compiles Typescript. Not being terribly familiar with it, it's possible this is more relevant to users of that registry, but it's not clear why this is an issue when the registry itself is handling transpiling and declaration generation. EDIT: To be clear, I think it's totally fine to publish your TS source in addition to your JS and d.ts outputs, just don't publish TS only packages, please.

seniorsassycat 2y ago
Id love it of npm added support for additional artifacts — separate tarballs for source, code, binaries, source maps, docs etc
- rezonant 2y ago
  Fair, but if you don't fall into the pit of despair by consuming micro libraries (ie "is-true" or "leftpad"), the overhead of having source and source maps present on a developer environment or even a server deployment is not terrible. If you are bundling for web delivery, those files will be omitted automatically by nature of the bundling process. While it would be nice to have native support for package components, personally I think the best path is to ship all of it (save perhaps documentation, where there's currently no ergonomic way for a consumer to make use of it) in the main package. The above is assuming "docs" mean some kind of external documentation, you absolutely should not strip out your docblocks when you package your code. Ditto for minification: Just don't. If the end user needs to optimize for size, they will be minifying anyway, all packaging minification does is create painful debugging and code reading experiences for consumers.
2y ago

stevenpetryk 2y ago

I believe the intention behind isolatedModules is to make compilation faster by forcing you to define types that require no inference from other files (thus, being isolated). Not… whatever this is.

rty32 2y ago

...wait. Is the entire article based on the assumption that everyone has Typescript installed in every project? Otherwise, how would publish Typescript files work at all? > We only ever ship build artifacts, the compiled JS and the relevant .d.ts files in a package. Isn't this because anyone, regardless of whether they have use the project in JS or TS, can import the library without caring out how the library code is implemented (in JS or TS)? It is not perfect but has worked for almost everyone. How would the author's project layout help people who only ever write code in JavaScript and never installed the Typescript package? Does Typescript have to be a dev dependency of any project that has an upstream package written in Typescript? That seems a bad idea. I write almost all my JavaScript projects with Typescript now (it's a bad sentence but you know what I mean), but I don't think we should ever make the life harder for those who only write in JavaScript.

mmoskal 2y ago
I think most bundlers can handle TS nowadays so you don't actually need tsc.
- rlt 2y ago
  Technically you still need tsc or an alternative TypeScript compiler like swc, but bundlers sometimes include them in their own dependencies so you don’t have to think about it.

29athrowaway 2y ago

JavaScript runtimes are full of optimizations that execute conditionally on specific scenarios. If you don't follow the rewarding path of optimization you then go to the deeply unhappy path of deoptimization. It's easy to get started with JavaScript, but to squeeze every bit of performance in JavaScript will quickly get you into a very complex topics that are not documented and change may change from version to version as they are often not "contracts". The only way to learn those obscure topics is by reading the source code for your JavaScript implementation, their bug tracker, etc. This means if you have not been reading C++ or executed a non-JS profiler to make your JS faster then you probably have not gone far enough. There are unexpected things that you would never have suspected if you had not read the implementation.

_jackwink 2y ago

This article is kind of a messy read, isolated declarations feels like it's being used as a mcguffin to advertise the jsr registry. Both may be neat things on their own, but I feel the connection is a bit misleading. Isolated declarations should allow parallelization and faster type checking with tooling that supports it. This shouldn't be changing your build/release process or what you export in your package.json. If you also happen to use the jsr registry to publish your package, it sounds like you can update your package.json to export TS files and they'll compile/inject the JS into your release artifacts on publish. Not sure if this feature requires isolated declarations though.

mhagemeister 2y ago
Author here. Thanks for sharing that feedback. I think I missed the mark by being so used to working with runtimes that natively run TS files, that I forgot that this is not the default for everyone. The article was mostly written from that perspective. For Node users the isolated declaration feature won't do much other than speed up the creation of the .d.ts files a little. For runtimes that can run TS natively like Deno (disclaimer: I work for Deno) the benefit is much more apparent as you always want the original TS sources, but can generate .d.ts files on the fly for faster type checking. EDIT: Formatting

tlarkworthy 2y ago

I don't know what an isolated declaration is. Do we write TS source differently?