10 ms·

Rust Is the Future of JavaScript Infrastructure (2021)

142 points by winter_blue 4y ago

TheDesolate0 4y ago
[dead]
vlovich123 4y ago
Not sure why ESBuild is included since it's in Go and there's no working link to learn more about a hypothetical speedup over ESBuild. Having used both swc-node and Rome after having used ts-node and prettier/eslint. SWC is a lot more finicky than ts-node (which itself is quite finicky) with a lot less documentation when something goes wrong (& often just looking at your setup sideways can cause it to stop working in surprising ways). Certainly I enjoy using ESBuild and I doubt swc could fill that niche right now (too immature I think). Rome is "special" in that it has none of the depth of eslint's TypeScript linting support and has very strange opinionated defaults that make no sense with limited configuration because the whole philosophy of that project is "just follow our rules without any configuration". Rome is to its word quite fast but given that it does a fraction of the linting eslint can do, it's hard to say whether that speedup will win out. Special shout out to dprint. While it too has Rome's disease of limiting configurability, it does have a sensible amount of it and it is drastically faster than prettier. I will note though that one thing that still annoys me is that two identical ASTs will result in different output formats because the input formatting can affect the output (i.e. it tries to preserve input, particularly around line breaks). I understand the philosophy of it but I'm not sold if I'm trying to enforce a consistent style across the codebase.
- satvikpendem 4y ago
  The Rome team ran out of funding and had layoffs, if you didn't know. I'm also a Rome user and it's sad to see.
  - qudat 4y ago
    Do you have any resources online about this? I'm using Rome as well -- quite enjoying it -- but we are running into its limitations which this news does not instill confidence.
    - conaclos 4y ago
      While that's true, the project attracted new core contributors. The development slowed down, but the project is still in active development. A new release is planned for this month.
      - qudat 4y ago
        Oh that's good to read! I'm pretty excited about rometools and desperately want to get out of this super pluggable linting world with a dozen+ eslint related packages. Everyone wants to use airbnb's style guide anyway, just make that the default with some minor overrides and I'm happy.
    - satvikpendem 4y ago
      It's on their Discord. While there is going to be a new release, the development definitely has slowed down. People on the Discord at least are thinking about whether a fork makes sense, since the Rome trademark is with the corporation and not the OSS project, if I understand correctly.
      - satvikpendem 4y ago
        https://discord.gg/rome https://discord.gg/rome for those interested
- johnnypangs 4y ago
  I would argue that SWC is more mature than esbuild as SWC is on a version 1+ release (the author on their site says that is “late stage beta” and vite still doesn’t use it for production builds), has been around for 2 years longer than esbuild (going by github histories) and is being made by more than one (very capable) person.
  - vlovich123 4y ago
    ¯\_(ツ)_/¯. Just relating my experience. I've found the ESBuild plugin ecosystem to be quite mature and blazingly fast. Any additional speedup of SWC wouldn't be felt as the startup overhead of npm to do `npm run build` dominates whatever time ESBuild spends.
  - conaclos 4y ago
    ESbuild seems more popular than SWC. Moreover, ESbuild has really good documentation and a great CHANGELOG. It is always a pleasure to read a new entry in its CHANGELOG. IN the other side, the changelog of SWC is too close to the code. This makes hard to understand what is added in a new version from the user perspective. I also chose ESbuild over SWC because ESbuild has no dependencies. SWC has more than 130 (transitive) dependencies [0]. [0] https://npmgraph.js.org/?q=swc https://npmgraph.js.org/?q=swc
- conaclos 4y ago
  > Rome is "special" in that it has none of the depth of eslint's TypeScript linting support and has very strange opinionated defaults that make no sense with limited configuration because the whole philosophy of that project is "just follow our rules without any configuration". This was true at the start of the project. They added many configurations asked by dev such as semicolons and indentation style. Any lint rule can also be disabled.
satvikpendem 4y ago
It sure is. I've talked about this before but there is no reason to have tooling be in the same language as the end user writing it. Python for example has libraries in C and C++ (numpy, pandas, other scientific libraries) and the end user just uses Python to interface with them. Imagine if they were all written in Python instead, and how slow that'd be. The only reason JS and TS tooling are written in these languages is because V8 is decently fast enough, but at scale, we can definitely do a lot better.
- ItsABytecode 4y ago
  Wait, to install this python package I have to have a working c, rust, and fortran compiler?
  - staunton 4y ago
    No, you get pre-built binaries. Of course you can compile everything yourself if you want
    - pjmlp 4y ago
      Good luck sorting it out for random OS XYZ.
      - mehdix 4y ago
        We don't even have proper wheels for Alpine, let alone random xyz OS.
- giraffe_lady 4y ago
  Eh there's definitely some value in having the tooling be explorable and comprehensible to the line programmers. The value depends on the situation of course, like in the scientific python example probably low-close-to-zero. But I think that's an outlier and the value is usually higher than that. Not always necessarily enough to outweigh other concerns, and I don't think always using the end language for tooling is the correct choice either. But it's a sensible default that you should make sure you get real value out of deviating from.
ldehaan 4y ago
[dead]
Buttons840 4y ago
I know this is tangental, but will WASM ever get the ability to manipulate the DOM? And will JavaScript as we know it survive if it does? All these people building JavaScript tooling in other languages would probably be so happy if there was no need for tooling in the first place.
- tabtab 4y ago
  That would probably expose it to so many security concerns that it would face a similar fate as Java applets & Flash: causing more hacking/snooping than benefits for a typical browser user. And I can't get a consistent answer on what problem WASM is trying to solve.
  - ihatepython 4y ago
    Obfuscation? Job security?
  - NoahKAndrews 4y ago
    The problem with Flash and Java was that they were not integrated with the browser's sandbox. WASM is integrated with the sandbox. How would allowing DOM access be insecure? The DOM is designed to be completely controlled by the webpage. I don't see how adding an additional way for the webpage to do so adds any insecurity.
    - satvikpendem 4y ago
      Indeed, as if sites today already can't snoop with JS instead of WASM.
      - tabtab 4y ago
        Why offer TWO snoop vectors? And crypto miners would target WASM over JS, giving WASM a bad name.
    - pjmlp 4y ago
      Yet, one can still compile Heartbleed into a WASM module. It adds more ways to sneak unwanted code into the page or do cross execution that shouldn't happen in first place, while staying inside the sandbox.
sylware 4y ago
I do disagree. I am picturing assembly written assembly engines which are should become quite relevant if RISC-V is a success.
streptomycin 4y ago
ESBuild is probably more popular than all the other non-JS JS dev tools combined, and (as the blog post even mentions, despite the title and main thesis) it's not written in Rust.
- nine_k 4y ago
  I don't see a contradiction. Were ESBuild already written in Rust, Rust would be the present of JS infrastructure. The author hopes it merely to be the future.
  - kekkidy 4y ago
    [dead]
  - streptomycin 4y ago
    Yeah but that future is probably a long ways off, if it ever comes at all. Currently ESBuild has 5x as many npm installs as Rome and SWC combined, and the gap doesn't seem to be shrinking https://npmtrends.com/@swc/core-vs-esbuild-vs-rome https://npmtrends.com/@swc/core-vs-esbuild-vs-rome
    - Patrickmi 4y ago
      [dead]
    - acemarke 4y ago
      FWIW, that doesn't take into account that Next actually has its own wrapper around SWC, and also vendors it. I think it's probably the largest consumer of SWC: - https://github.com/vercel/next.js/tree/canary/packages/next-swc https://github.com/vercel/next.js/tree/canary/packages/next-... Those add up pretty fast: - https://npmtrends.com/@next/swc-linux-x64-gnu-vs-@next/swc-win32-x64-msvc-vs-@swc/core-vs-esbuild-vs-rome https://npmtrends.com/@next/swc-linux-x64-gnu-vs-@next/swc-w...
segphault 4y ago
I'm bullish on Rust, but there's a long way still to go. The overhead of passing values across the boundary between JavaScript and Rust is quite high. There are a lot of cases where you want to be able to provide a dynamic configuration to something on the Rust side, ideally from JavaScript, and that's still pretty costly from a performance perspective. One of my projects (https://markdoc.dev/ https://markdoc.dev/) is a Markdown dialect that supports custom tags and a React renderer. I recently experimented with implementing a parser for it in Rust in order to increase performance. My Rust-based parser is significantly faster than my existing JavaScript parser, but then I have to serialize the AST in order to move it from Rust to JavaScript. I'd like to implement the entire processor in Rust, but I need to let users define custom tags in JavaScript, and the overhead of going back and forth is far from ideal. I'm hopeful that the recently-ratified Wasm GC proposal—which introduces managed structs and arrays that don't cost anything to pass between the Wasm environment and JavaScript—will help a lot. But it's going to take awhile for Wasm GC features to land in LLVM and be properly supported in Rust.
- samsquire 4y ago
  Mozilla abandoned XPCOM partly due to the runtime cost of marshalling between C++ and JavaScript. https://yoric.github.io/post/why-did-mozilla-remove-xul-addons/ https://yoric.github.io/post/why-did-mozilla-remove-xul-addo... I wonder if the solution is transpiling. If Javascript was transpiled to Rust, there would be less data to copy. You would just have two API surfaces. But you wouldn't have runtime Javascript execution, just the capability of running Javascript code at native performances.
  - jraph 4y ago
    > If Javascript was transpiled to Rust Seems challenging. JavaScript code does not come with lifetime information that the Rust compiler expects to have to get rid of stuff that's not used anymore, while JS has a garbage collector to do this. There's also no borrowing information encoded in JavaScript code. Rust expects the developers to provide this information in their code so it knows who and where what can modify what.
  - 4y ago
chubot 4y ago
Yup, more generally, I'd say that statically typed languages that compile to native code (Rust, Go, C++, etc.) are necessary to achieve good performance when writing linters / formatters, type checkers, compilers, and interpreters (AST- and graph- based workloads). You could frame it as a "failure of JITs in the 2010's". JavaScript isn't a good language to write the TypeScript compiler or a linter, because v8 isn't fast enough. The semantics of JavaScript do not allow v8 to be fast enough. IIRC this was precisely why the Dart project was started more than 10 years ago by the original v8 authors. They were spending a lot of time looking into why real world web page performance was falling off various cliffs in v8. They realized they needed to change the LANGUAGE in order to be able to write fast programs. A major use case was developing programs like Google Docs and GMail in the browser, which had to compete with native programs written in C++. JITs are fast in common cases, but they not only have big costs in terms of memory (code storage) and startup time, but they're hard to ENGINEER with! Similar story with Python tooling -- the linters, formatters, and type checkers are quite slow due to being written in Python. mypyc gives a bit of speedup, but it still uses the Python runtime. Related story from yesterday: "Even the pylint codebase uses Ruff" (linter in Rust) https://news.ycombinator.com/item?id=35035618 https://news.ycombinator.com/item?id=35035618 --- Emery Berger has memorable framing of this -- "Perl, Python, PHP, and JS are the irrational exuberance" languages. https://www.sigarch.org/from-heavy-metal-to-irrational-exuberance/ https://www.sigarch.org/from-heavy-metal-to-irrational-exube... That is, he says that in the 90's and 2000's, we thought that clock speeds would continually increase, and JITs would get better, and so we could design language semantics without regard to performance -- language that almost REQUIRE slow implementations. (I think his take is about 50% true. The other 50% is that dynamic languages simply allowed people to produce popular and useful software at a greater rate, especially for the web, so we ended up with a lot of software written in dynamic languages! Doing web apps in Java vs. Ruby/Python/JS is a huge difference in productivity, and I'd say you often end up with a BETTER result, due to increased iteration / fast feedback.) --- This also tracks with my experience with https://www.oilshell.org https://www.oilshell.org, where we reverse-engineered the shell in an experimental fashion with Python, and then evolved that implementation into a statically typed language that generates C++ (using MyPy, ASDL, and algebraic data types). Oil Is Being Implemented "Middle Out" :https://www.oilshell.org/blog/2022/03/middle-out.html https://www.oilshell.org/blog/2022/03/middle-out.html (and many other blog posts) This core of the program is the elaborate and strongly typed "lossless syntax tree", which is basically what's used in linters and formatters. Even though I've been using both C++ and Python for >20 years, I was a little shocked how much worse Python is for AST- and graph-based workloads. I'm looking for references/measurements specifically on these types of workloads. I think a lot of papers about JITs are misleading with respect to them, or at least you have to read between the lines. I'd say that Python and JS are 10x as slow as native code for "business" and "web app" workloads, and I've never had a problem with them in those settings. Quite the contrary, I've actually sped up poorly working code in static languages with Python. If you're within 10x of the hardware's performance, you're doing VERY WELL compared to "typical software", which has layers of gunk and can be 100x to 1000x too slow. But bare Python and JS (no libraries) are closer to 100x too slow for ASTs and graphs. This is because of all the allocation and GC overhead -- in both time and space -- in addition to dynamic dispatch, etc. (The funny thing is that Oil is now the most statically-typed shell implementation, even though it's nominally written in Python :) It uses fine-grained static types, where as shells written in C use a homogeneous "WORD*" representation in C, and strings with control codes embedded in them for "structure" and "types". I should probably write a blog post about that ...) --- To shine some light on the other side, I'm still a bit skeptical of Rust specifically, because: - memory management is littered all over the codebase. - Borrow checking seems to work better for stateless/batch programs (it thinks about parameters and return values), but linters and type checkers for language servers are STATEFUL: https://news.ycombinator.com/item?id=34410187 https://news.ycombinator.com/item?id=34410187 - Many ASTs are actually graphs - pattern matching can't see through boxing apparently ? Also, the author of esbuild tried BOTH Rust and Go, and ended up with Go. IIRC it performed better because it didn't have deterministic destruction on the stack -- GC was more efficient? It seems like a language with both garbage collection and algebraic data types would be nicer, but neither Go or Rust fit that description! Rust makes a lot of sense for kernels and so forth, but for language processors -- especially stateful ones (which includes the Unix shell!) -- I think GC is still a big help. And we know already know how to make GC fast for that use case, i.e. you don't need a GC that scales to 1 TB of memory on 128 cores.
Annatar 4y ago
[dead]
lacker 4y ago
Rust is a very nice fit for language tooling. It has several features that are particularly useful for syntactic manipulation, like matching, enums, and Result<x> error handling. And a focus on efficiency is especially nice for language tooling, because no matter how fast your tools are, a codebase can get big enough to make them annoyingly slow.
- hulitu 4y ago
  > Rust is a very nice fit for language tooling If only they could fix the toolchain. If version 1.58.0 is so different from 1.58.1 that it cannot compile 1.66 something is terribly wrong. And getting crates from internet is a terrible idea from a security point of view.
  - ihatepython 4y ago
    Perhaps the Rust toolchain can be fixed by rewriting it in Javascript?
anon23432343 4y ago
and Java 8.0 /s ¯\_(ツ)_/¯
pjmlp 4y ago
Only because it is now hyped and cool to do so. Having an improved JIT, or compiled managed languages like Go, D, Java, C#, Dart is a much better option than a language for doing kernel stuff. Leave Rust for kernel and device drivers, that is where not having any kind of automatic memory management kind of matters.
- chubot 4y ago
  I think JITs for dynamic languages like JS and Python have ALREADY failed for this use case -- they're not fast enough https://news.ycombinator.com/item?id=35045520 https://news.ycombinator.com/item?id=35045520 I agree with using statically-typed, managed languages though
  - pjmlp 4y ago
    Python hasn't failed, because CPython doesn't have one, and PyPy has been largely ignored. So if anything it has been a failure in performance since early days. In regards to JS, there is still so much to implement, like JIT caches with PGO metadata across runs. So it isn't as if everything has been tried out. What happens, and is proven by Java and .NET, is that FOSS alone doesn't produce top level JIT and GC compilers. And since Google doesn't care about improving V8 for nodejs workloads, everyone else rewrites JavaScript into something else.
    - chubot 4y ago
      No, PyPy isn't fast enough for these workloads (again see my comment, I tried) v8 isn't fast enough either, and it's basically a Rube Goldberg machine maxed out in code complexity right now. It probably 2M lines of code, with multiple interpreters and compilers. They're lucky if they can improve metrics 5% -- things aren't getting 100% faster With JITs, both JS and Python runtimes are still ~10x too slow, as opposed to ~100x too slow without them. They also offer you approximately zero control over memory, and as a result use ~10x more memory. This can certainly matter even for a tool running along aside your editor on a desktop
      - 4y ago
worik 4y ago
[flagged]
klyrs 4y ago
Odd that there's no mention of Bun here. I agree with the general thrust that JavaScript infrastructure goes faster when written in a fast language, but as Bun and esbuild show, Rust isn't the only future. edit: there's a date tag on the title now... whoops.
- steveklabnik 4y ago
  Bun was created in July of 2022, and this blog post was written in November of 2021.
pcthrowaway 4y ago
(2021)
jpgvm 4y ago
Rust or some other good language should be the future of your software instead of just using it to prop up your JS/TS layer cake of spaghetti. I get that many feel like it's sunk cost etc but every day I use the TS stack I lose a little bit of love for programming. Not to mention the huge chunks of motivation that get eaten up by yet another bullshit "build" system issue, or sourcemaps not working after upgrading X or Y or tsserver getting confused or Babel macro plugin garbage, etc. Especially having worked with Rust, Kotlin, etc that you know.. work the way you would expect them to - hell, Java feels refreshing after Typescript. I miss just writing code with tools that work well. Especially on the server where there is really no reason to ever use Typescript and yet everyone seems to think it's a wonderful idea and I'm the crazy one for suggesting there are better options.
dang 4y ago
Discussed at the time: Rust Is the Future of JavaScript Infrastructure - https://news.ycombinator.com/item?id=29192088 https://news.ycombinator.com/item?id=29192088 - Nov 2021 (164 comments)
Patrickmi 4y ago
When it comes to JS ecosystem, developer common sense doesn’t work because it fast doesn’t mean it’s bound to succeed among its counterparts, the major thing such could help is their ecosystem they are written in
slmjkdbtl 4y ago
There's nothing inherent i Rust that makes it the perfect JS tooling language, it's more correct to say JS tooling will adopt more performant and safe languanges, and Rust is one of them.
qdequelen 4y ago
Make the web faster with Rust.

142 points by winter_blue 4y ago

[dead]

Not sure why ESBuild is included since it's in Go and there's no working link to learn more about a hypothetical speedup over ESBuild. Having used both swc-node and Rome after having used ts-node and prettier/eslint. SWC is a lot more finicky than ts-node (which itself is quite finicky) with a lot less documentation when something goes wrong (& often just looking at your setup sideways can cause it to stop working in surprising ways). Certainly I enjoy using ESBuild and I doubt swc could fill that niche right now (too immature I think). Rome is "special" in that it has none of the depth of eslint's TypeScript linting support and has very strange opinionated defaults that make no sense with limited configuration because the whole philosophy of that project is "just follow our rules without any configuration". Rome is to its word quite fast but given that it does a fraction of the linting eslint can do, it's hard to say whether that speedup will win out. Special shout out to dprint. While it too has Rome's disease of limiting configurability, it does have a sensible amount of it and it is drastically faster than prettier. I will note though that one thing that still annoys me is that two identical ASTs will result in different output formats because the input formatting can affect the output (i.e. it tries to preserve input, particularly around line breaks). I understand the philosophy of it but I'm not sold if I'm trying to enforce a consistent style across the codebase.

satvikpendem 4y ago
The Rome team ran out of funding and had layoffs, if you didn't know. I'm also a Rome user and it's sad to see.
- qudat 4y ago
  Do you have any resources online about this? I'm using Rome as well -- quite enjoying it -- but we are running into its limitations which this news does not instill confidence.
  - conaclos 4y ago
    While that's true, the project attracted new core contributors. The development slowed down, but the project is still in active development. A new release is planned for this month.
    - qudat 4y ago
      Oh that's good to read! I'm pretty excited about rometools and desperately want to get out of this super pluggable linting world with a dozen+ eslint related packages. Everyone wants to use airbnb's style guide anyway, just make that the default with some minor overrides and I'm happy.
  - satvikpendem 4y ago
    It's on their Discord. While there is going to be a new release, the development definitely has slowed down. People on the Discord at least are thinking about whether a fork makes sense, since the Rome trademark is with the corporation and not the OSS project, if I understand correctly.
    - satvikpendem 4y ago
      https://discord.gg/rome https://discord.gg/rome for those interested
johnnypangs 4y ago
I would argue that SWC is more mature than esbuild as SWC is on a version 1+ release (the author on their site says that is “late stage beta” and vite still doesn’t use it for production builds), has been around for 2 years longer than esbuild (going by github histories) and is being made by more than one (very capable) person.
- vlovich123 4y ago
  ¯\_(ツ)_/¯. Just relating my experience. I've found the ESBuild plugin ecosystem to be quite mature and blazingly fast. Any additional speedup of SWC wouldn't be felt as the startup overhead of npm to do `npm run build` dominates whatever time ESBuild spends.
- conaclos 4y ago
  ESbuild seems more popular than SWC. Moreover, ESbuild has really good documentation and a great CHANGELOG. It is always a pleasure to read a new entry in its CHANGELOG. IN the other side, the changelog of SWC is too close to the code. This makes hard to understand what is added in a new version from the user perspective. I also chose ESbuild over SWC because ESbuild has no dependencies. SWC has more than 130 (transitive) dependencies [0]. [0] https://npmgraph.js.org/?q=swc https://npmgraph.js.org/?q=swc
conaclos 4y ago
> Rome is "special" in that it has none of the depth of eslint's TypeScript linting support and has very strange opinionated defaults that make no sense with limited configuration because the whole philosophy of that project is "just follow our rules without any configuration". This was true at the start of the project. They added many configurations asked by dev such as semicolons and indentation style. Any lint rule can also be disabled.

satvikpendem 4y ago

It sure is. I've talked about this before but there is no reason to have tooling be in the same language as the end user writing it. Python for example has libraries in C and C++ (numpy, pandas, other scientific libraries) and the end user just uses Python to interface with them. Imagine if they were all written in Python instead, and how slow that'd be. The only reason JS and TS tooling are written in these languages is because V8 is decently fast enough, but at scale, we can definitely do a lot better.

ItsABytecode 4y ago
Wait, to install this python package I have to have a working c, rust, and fortran compiler?
- staunton 4y ago
  No, you get pre-built binaries. Of course you can compile everything yourself if you want
  - pjmlp 4y ago
    Good luck sorting it out for random OS XYZ.
    - mehdix 4y ago
      We don't even have proper wheels for Alpine, let alone random xyz OS.
giraffe_lady 4y ago
Eh there's definitely some value in having the tooling be explorable and comprehensible to the line programmers. The value depends on the situation of course, like in the scientific python example probably low-close-to-zero. But I think that's an outlier and the value is usually higher than that. Not always necessarily enough to outweigh other concerns, and I don't think always using the end language for tooling is the correct choice either. But it's a sensible default that you should make sure you get real value out of deviating from.

ldehaan 4y ago

[dead]

Buttons840 4y ago

I know this is tangental, but will WASM ever get the ability to manipulate the DOM? And will JavaScript as we know it survive if it does? All these people building JavaScript tooling in other languages would probably be so happy if there was no need for tooling in the first place.

tabtab 4y ago
That would probably expose it to so many security concerns that it would face a similar fate as Java applets & Flash: causing more hacking/snooping than benefits for a typical browser user. And I can't get a consistent answer on what problem WASM is trying to solve.
- ihatepython 4y ago
  Obfuscation? Job security?
- NoahKAndrews 4y ago
  The problem with Flash and Java was that they were not integrated with the browser's sandbox. WASM is integrated with the sandbox. How would allowing DOM access be insecure? The DOM is designed to be completely controlled by the webpage. I don't see how adding an additional way for the webpage to do so adds any insecurity.
  - satvikpendem 4y ago
    Indeed, as if sites today already can't snoop with JS instead of WASM.
    - tabtab 4y ago
      Why offer TWO snoop vectors? And crypto miners would target WASM over JS, giving WASM a bad name.
  - pjmlp 4y ago
    Yet, one can still compile Heartbleed into a WASM module. It adds more ways to sneak unwanted code into the page or do cross execution that shouldn't happen in first place, while staying inside the sandbox.

sylware 4y ago

I do disagree. I am picturing assembly written assembly engines which are should become quite relevant if RISC-V is a success.

streptomycin 4y ago

ESBuild is probably more popular than all the other non-JS JS dev tools combined, and (as the blog post even mentions, despite the title and main thesis) it's not written in Rust.

nine_k 4y ago
I don't see a contradiction. Were ESBuild already written in Rust, Rust would be the present of JS infrastructure. The author hopes it merely to be the future.
- kekkidy 4y ago
  [dead]
- streptomycin 4y ago
  Yeah but that future is probably a long ways off, if it ever comes at all. Currently ESBuild has 5x as many npm installs as Rome and SWC combined, and the gap doesn't seem to be shrinking https://npmtrends.com/@swc/core-vs-esbuild-vs-rome https://npmtrends.com/@swc/core-vs-esbuild-vs-rome
  - Patrickmi 4y ago
    [dead]
  - acemarke 4y ago
    FWIW, that doesn't take into account that Next actually has its own wrapper around SWC, and also vendors it. I think it's probably the largest consumer of SWC: - https://github.com/vercel/next.js/tree/canary/packages/next-swc https://github.com/vercel/next.js/tree/canary/packages/next-... Those add up pretty fast: - https://npmtrends.com/@next/swc-linux-x64-gnu-vs-@next/swc-win32-x64-msvc-vs-@swc/core-vs-esbuild-vs-rome https://npmtrends.com/@next/swc-linux-x64-gnu-vs-@next/swc-w...

segphault 4y ago

I'm bullish on Rust, but there's a long way still to go. The overhead of passing values across the boundary between JavaScript and Rust is quite high. There are a lot of cases where you want to be able to provide a dynamic configuration to something on the Rust side, ideally from JavaScript, and that's still pretty costly from a performance perspective. One of my projects (https://markdoc.dev/ https://markdoc.dev/) is a Markdown dialect that supports custom tags and a React renderer. I recently experimented with implementing a parser for it in Rust in order to increase performance. My Rust-based parser is significantly faster than my existing JavaScript parser, but then I have to serialize the AST in order to move it from Rust to JavaScript. I'd like to implement the entire processor in Rust, but I need to let users define custom tags in JavaScript, and the overhead of going back and forth is far from ideal. I'm hopeful that the recently-ratified Wasm GC proposal—which introduces managed structs and arrays that don't cost anything to pass between the Wasm environment and JavaScript—will help a lot. But it's going to take awhile for Wasm GC features to land in LLVM and be properly supported in Rust.

samsquire 4y ago
Mozilla abandoned XPCOM partly due to the runtime cost of marshalling between C++ and JavaScript. https://yoric.github.io/post/why-did-mozilla-remove-xul-addons/ https://yoric.github.io/post/why-did-mozilla-remove-xul-addo... I wonder if the solution is transpiling. If Javascript was transpiled to Rust, there would be less data to copy. You would just have two API surfaces. But you wouldn't have runtime Javascript execution, just the capability of running Javascript code at native performances.
- jraph 4y ago
  > If Javascript was transpiled to Rust Seems challenging. JavaScript code does not come with lifetime information that the Rust compiler expects to have to get rid of stuff that's not used anymore, while JS has a garbage collector to do this. There's also no borrowing information encoded in JavaScript code. Rust expects the developers to provide this information in their code so it knows who and where what can modify what.
- 4y ago

chubot 4y ago

Yup, more generally, I'd say that statically typed languages that compile to native code (Rust, Go, C++, etc.) are necessary to achieve good performance when writing linters / formatters, type checkers, compilers, and interpreters (AST- and graph- based workloads). You could frame it as a "failure of JITs in the 2010's". JavaScript isn't a good language to write the TypeScript compiler or a linter, because v8 isn't fast enough. The semantics of JavaScript do not allow v8 to be fast enough. IIRC this was precisely why the Dart project was started more than 10 years ago by the original v8 authors. They were spending a lot of time looking into why real world web page performance was falling off various cliffs in v8. They realized they needed to change the LANGUAGE in order to be able to write fast programs. A major use case was developing programs like Google Docs and GMail in the browser, which had to compete with native programs written in C++. JITs are fast in common cases, but they not only have big costs in terms of memory (code storage) and startup time, but they're hard to ENGINEER with! Similar story with Python tooling -- the linters, formatters, and type checkers are quite slow due to being written in Python. mypyc gives a bit of speedup, but it still uses the Python runtime. Related story from yesterday: "Even the pylint codebase uses Ruff" (linter in Rust) https://news.ycombinator.com/item?id=35035618 https://news.ycombinator.com/item?id=35035618 --- Emery Berger has memorable framing of this -- "Perl, Python, PHP, and JS are the irrational exuberance" languages. https://www.sigarch.org/from-heavy-metal-to-irrational-exuberance/ https://www.sigarch.org/from-heavy-metal-to-irrational-exube... That is, he says that in the 90's and 2000's, we thought that clock speeds would continually increase, and JITs would get better, and so we could design language semantics without regard to performance -- language that almost REQUIRE slow implementations. (I think his take is about 50% true. The other 50% is that dynamic languages simply allowed people to produce popular and useful software at a greater rate, especially for the web, so we ended up with a lot of software written in dynamic languages! Doing web apps in Java vs. Ruby/Python/JS is a huge difference in productivity, and I'd say you often end up with a BETTER result, due to increased iteration / fast feedback.) --- This also tracks with my experience with https://www.oilshell.org https://www.oilshell.org, where we reverse-engineered the shell in an experimental fashion with Python, and then evolved that implementation into a statically typed language that generates C++ (using MyPy, ASDL, and algebraic data types). Oil Is Being Implemented "Middle Out" :https://www.oilshell.org/blog/2022/03/middle-out.html https://www.oilshell.org/blog/2022/03/middle-out.html (and many other blog posts) This core of the program is the elaborate and strongly typed "lossless syntax tree", which is basically what's used in linters and formatters. Even though I've been using both C++ and Python for >20 years, I was a little shocked how much worse Python is for AST- and graph-based workloads. I'm looking for references/measurements specifically on these types of workloads. I think a lot of papers about JITs are misleading with respect to them, or at least you have to read between the lines. I'd say that Python and JS are 10x as slow as native code for "business" and "web app" workloads, and I've never had a problem with them in those settings. Quite the contrary, I've actually sped up poorly working code in static languages with Python. If you're within 10x of the hardware's performance, you're doing VERY WELL compared to "typical software", which has layers of gunk and can be 100x to 1000x too slow. But bare Python and JS (no libraries) are closer to 100x too slow for ASTs and graphs. This is because of all the allocation and GC overhead -- in both time and space -- in addition to dynamic dispatch, etc. (The funny thing is that Oil is now the most statically-typed shell implementation, even though it's nominally written in Python :) It uses fine-grained static types, where as shells written in C use a homogeneous "WORD*" representation in C, and strings with control codes embedded in them for "structure" and "types". I should probably write a blog post about that ...) --- To shine some light on the other side, I'm still a bit skeptical of Rust specifically, because: - memory management is littered all over the codebase. - Borrow checking seems to work better for stateless/batch programs (it thinks about parameters and return values), but linters and type checkers for language servers are STATEFUL: https://news.ycombinator.com/item?id=34410187 https://news.ycombinator.com/item?id=34410187 - Many ASTs are actually graphs - pattern matching can't see through boxing apparently ? Also, the author of esbuild tried BOTH Rust and Go, and ended up with Go. IIRC it performed better because it didn't have deterministic destruction on the stack -- GC was more efficient? It seems like a language with both garbage collection and algebraic data types would be nicer, but neither Go or Rust fit that description! Rust makes a lot of sense for kernels and so forth, but for language processors -- especially stateful ones (which includes the Unix shell!) -- I think GC is still a big help. And we know already know how to make GC fast for that use case, i.e. you don't need a GC that scales to 1 TB of memory on 128 cores.

Annatar 4y ago

[dead]

lacker 4y ago

Rust is a very nice fit for language tooling. It has several features that are particularly useful for syntactic manipulation, like matching, enums, and Result<x> error handling. And a focus on efficiency is especially nice for language tooling, because no matter how fast your tools are, a codebase can get big enough to make them annoyingly slow.

hulitu 4y ago
> Rust is a very nice fit for language tooling If only they could fix the toolchain. If version 1.58.0 is so different from 1.58.1 that it cannot compile 1.66 something is terribly wrong. And getting crates from internet is a terrible idea from a security point of view.
- ihatepython 4y ago
  Perhaps the Rust toolchain can be fixed by rewriting it in Javascript?

anon23432343 4y ago

and Java 8.0 /s ¯\_(ツ)_/¯

pjmlp 4y ago

Only because it is now hyped and cool to do so. Having an improved JIT, or compiled managed languages like Go, D, Java, C#, Dart is a much better option than a language for doing kernel stuff. Leave Rust for kernel and device drivers, that is where not having any kind of automatic memory management kind of matters.

chubot 4y ago
I think JITs for dynamic languages like JS and Python have ALREADY failed for this use case -- they're not fast enough https://news.ycombinator.com/item?id=35045520 https://news.ycombinator.com/item?id=35045520 I agree with using statically-typed, managed languages though
- pjmlp 4y ago
  Python hasn't failed, because CPython doesn't have one, and PyPy has been largely ignored. So if anything it has been a failure in performance since early days. In regards to JS, there is still so much to implement, like JIT caches with PGO metadata across runs. So it isn't as if everything has been tried out. What happens, and is proven by Java and .NET, is that FOSS alone doesn't produce top level JIT and GC compilers. And since Google doesn't care about improving V8 for nodejs workloads, everyone else rewrites JavaScript into something else.
  - chubot 4y ago
    No, PyPy isn't fast enough for these workloads (again see my comment, I tried) v8 isn't fast enough either, and it's basically a Rube Goldberg machine maxed out in code complexity right now. It probably 2M lines of code, with multiple interpreters and compilers. They're lucky if they can improve metrics 5% -- things aren't getting 100% faster With JITs, both JS and Python runtimes are still ~10x too slow, as opposed to ~100x too slow without them. They also offer you approximately zero control over memory, and as a result use ~10x more memory. This can certainly matter even for a tool running along aside your editor on a desktop
    - 4y ago

worik 4y ago

[flagged]

klyrs 4y ago

Odd that there's no mention of Bun here. I agree with the general thrust that JavaScript infrastructure goes faster when written in a fast language, but as Bun and esbuild show, Rust isn't the only future. edit: there's a date tag on the title now... whoops.

steveklabnik 4y ago
Bun was created in July of 2022, and this blog post was written in November of 2021.

pcthrowaway 4y ago

(2021)

jpgvm 4y ago

Rust or some other good language should be the future of your software instead of just using it to prop up your JS/TS layer cake of spaghetti. I get that many feel like it's sunk cost etc but every day I use the TS stack I lose a little bit of love for programming. Not to mention the huge chunks of motivation that get eaten up by yet another bullshit "build" system issue, or sourcemaps not working after upgrading X or Y or tsserver getting confused or Babel macro plugin garbage, etc. Especially having worked with Rust, Kotlin, etc that you know.. work the way you would expect them to - hell, Java feels refreshing after Typescript. I miss just writing code with tools that work well. Especially on the server where there is really no reason to ever use Typescript and yet everyone seems to think it's a wonderful idea and I'm the crazy one for suggesting there are better options.

dang 4y ago

Discussed at the time: Rust Is the Future of JavaScript Infrastructure - https://news.ycombinator.com/item?id=29192088 https://news.ycombinator.com/item?id=29192088 - Nov 2021 (164 comments)

Patrickmi 4y ago

When it comes to JS ecosystem, developer common sense doesn’t work because it fast doesn’t mean it’s bound to succeed among its counterparts, the major thing such could help is their ecosystem they are written in

slmjkdbtl 4y ago

There's nothing inherent i Rust that makes it the perfect JS tooling language, it's more correct to say JS tooling will adopt more performant and safe languanges, and Rust is one of them.

qdequelen 4y ago

Make the web faster with Rust.