8 ms·

It's the result of the data isolation above anything else attitude of Javascript. Or, in other words, it's the unavoidable result of insisting on using a langu

It's the result of the data isolation above anything else attitude of Javascript. Or, in other words, it's the unavoidable result of insisting on using a language created for the frontend to write everything else. You don't need to rewrite your code in Rust to get that saving. Any other language will do. (Personally, I'm surprised all the gains are so small. Looks like it's a very well optimized code path.)

adastra22 2y ago
There is no reason data isolation should cost you 100x memory usage.
- marcosdumay 2y ago
  There are plenty of reasons. They are just not intrinsic to the isolation, instead they come from complications rooted deeply on the underlying system. If you rebuild Linux from the ground up with isolation in mind, you will be able to do it more efficiently. People are indeed in the process of rewriting it, but it's far from complete (and moving back and forward, as not every Linux dev cares about it).
  - btilly 2y ago
    Unless you can be concrete and specific about some of those reasons, you're just replacing handwaving with more vigorous handwaving. What is it specifically about JavaScript's implementation of data isolation that, in your mind, helps cause the excessive memory usage?
    - marcosdumay 2y ago
      Just a day or two ago, there was an article here about problems implementing a kind of read-only memory constraint that Javacript benefited from in other OSes.
      - btilly 2y ago
        I must have missed that article. Can you find it? Unless you can come up with a specific reference, it seems unlikely that this would explain the large memory efficiency difference. By contrast it is simple and straightforward to understand why keeping temporary garbage until garbage collection could result in tying up a lot of memory while continually running code that allocates memory and lets it go out of scope. If you search, you'll find lots of references to this happening in a variety of languages.
- chipdart 2y ago
  > There is no reason data isolation should cost you 100x memory usage. It really depends on what you mean by "memory usage". The fundamental principle of any garbage collection system is that you allocate objects in the heap at will without freeing them until you really need to, and when that time comes you rely on garbage collection strategies to free and move objects. What this means is that processes end up allocating more data that the one being used, just because there is no need to free it. Consequently, with garbage collecting languages you configure processes with a specific memory budget. The larger the budget, the rarer these garbage collection strategies kick in. I run a service written with a garbage collected language. It barely uses more than 100MB of memory to handle a couple hundred requests per minute. The process takes over as much as 2GB of RAM before triggering generation 0 garbage collection events. These events trigger around 2 or 3 times per month. A simplistic critic would argue the service is wasting 10x the memory. That critic would be manifesting his ignorance, because there is absolutely nothing to gain by lowering the memory budget.
  - nicoburns 2y ago
    > That critic would be manifesting his ignorance, because there is absolutely nothing to gain by lowering the memory budget. Given that compute is often priced proportional to (maximum) memory usage, there is potentially a lot to be gained: dramatically cheaper hosting costs. Of course if your hosting costs are small to be begin with then this likely isn't worthwhile.
    - chipdart 2y ago
      > Given that compute is often priced proportional to (maximum) memory usage, (...) Let's look at numbers. Hetzner sells vCPUs with 4GB of RAM for less than 5$/month, and 8GB of RAM for less than $10/month. https://www.hetzner.com/cloud/ https://www.hetzner.com/cloud/ In my example, the cost of having garbage collection generation 0 events triggering twice a year would be an extra $5. If I wanted the frequency of these events to double, in theory I would save perhaps $2/month. If I ran a web-scale service with 10 times the nodes as-is, we're talking about a $50/month price tag difference. How much does a company charge for an engineer's hourly labor? How many years would it take to recover the cost of having an engineer tune a service's garbage collection strategy? People need to thing things through before discussing technical merits.
jvanderbot 2y ago
"Rust" really just means "Not javascript" as a recurring pattern in these articles.
- noirscape 2y ago
  It's also frankly kinda like comparing apples and oranges as a language. JavaScript (and many of the "bad performance" high level languages minus Rails; Rails is bad and should be avoided for projects as much as possible unless you have lots of legacy cruft) are also heavily designed around rapid iteration. Rust is however very much not capable of rapid iteration, the borrow checker will fight you heavily every step of the way to the point where it demands constant refactors. Basically the best place where Rust can work is one where all variables, all requirements and all edgecases are known ahead of time or cases where manual memory safety is a necessity vis-a-vis accepting a minor performance hike from things like the garbage collector. This works well in some spaces (notably; systems programming, embedded and Browser Engines and I wouldn't consider the latter a valid target), but webserver development is probably one of the furthest places where you are looking for Rust.
  - hathawsh 2y ago
    I have often thought that programmers can actually just choose to make Rust easy by using a cyclic garbage collector such as Samsara. [1] If cyclic GC in Rust works as well as I think it can, it should be the best option for the majority of high level projects that need fast development with a trade-off of slightly lower efficiency. I suspect we'll see a "hockey stick" adoption curve once everyone figures this out. [1] https://github.com/chc4/samsara https://github.com/chc4/samsara
    - 0cf8612b2e1e 2y ago
      I am still waiting for a scripting language to be bolted on top of Rust. Something that will silently Box all the values so the programmer does not have to think about the Rust specifics, but can still lean on all of the Rust machinery and libraries. If performance/correctness becomes a problem, the scripting layer could be replaced piecemeal with real Rust.
nh2 2y ago
It's important to be aware that often it isn't the programming language that has the biggest effect on memory usage, but simply settings of the memory allocator and OS behaviour. This also means that you cannot "simply measure memory usage" (e.g. using `time` or `htop`) without already having a relatively deep understanding of the underlying mechanisms. Most importantly: libc / malloc implementation: glibc by default has heavy memory fragmentation, especially in multi-threaded programs. It means it will not return `malloc()`ed memory back to the OS when the application `free()`s it, keeping it instead for the next allocation, because that's faster. Its default settings will e.g. favour 10x increased RESident memory usage for 2% speed gain. Some of this can be turned off in glibc using e.g. the env var `MALLOC_MMAP_THRESHOLD_=65536` -- for many applications I've looked at, this instantaneously reduced RES fro 7 GiB to 1 GiB. Some other issues cannot be addressed, because the corresponding glibc tunables are bugged [2]. For jemalloc `MALLOC_CONF=dirty_decay_ms:0,muzzy_decay_ms:0` helps to return memory to the OS immediately. Linux: Memory is generally allocated from the OS using `mmap()`, and returned using `munmap()`. But that can be a bit slow. So some applications and programming language runtimes use instead `madvise(MADV_FREE)`; this effectively returns the memory to the OS, but the OS does not actually do costly mapping table changes unless it's under memory pressure. As a result, one observes hugely increased memory usage in `time` or `htop`. [2] The above means that people are completely unware what actually eats their memory and what the actual resource usage is, easily "measuring wrong" by factor 10x. For example, I've seen people switch between Haskell and Go (both directions) because they thought the other one used less memory. It actually was just the glibc/Linux flags that made the actual difference. Nobody made the effort to really understand what's going on. Same thing for C++. You think without GC you have tight memory control, but in fact your memory is often not returned to the OS when the destructor is called, for the above reason. This also means that the numbers for Rust or JS may easily be wrong (in either direction, or both). So it's quite important to measure memory usage also with the tools above malloc(), otherwise you may just measure the wrong thing. [1]: https://sourceware.org/bugzilla/show_bug.cgi?id=14827 https://sourceware.org/bugzilla/show_bug.cgi?id=14827 [2]: https://downloads.haskell.org/ghc/latest/docs/users_guide/runtime_control.html#rts-flag---disable-delayed-os-memory-return https://downloads.haskell.org/ghc/latest/docs/users_guide/ru...
- Capricorn2481 2y ago
btilly 2y ago
Your claim makes zero sense to me. Particularly when I've personally seen similar behavior out of other languages, like Java. As I said in another comment, the most likely cause is that temporary garbage is not collected immediately in JavaScript, while garbage is collected immediately in Rust. See https://doc.rust-lang.org/nomicon/ownership.html https://doc.rust-lang.org/nomicon/ownership.html for the key idea behind how Rust manages this. If you truly believe that it is somehow due to data isolation, then I would appreciate a reference to where JavaScript's design causes it to behave differently.
chipdart 2y ago
> Or, in other words, it's the unavoidable result of insisting on using a language created for the frontend to write everything else. I don't think this is an educated take. The whole selling point of JavaScript in the backend has nothing to do with "frontend" things. The primary selling point is what makes Node.js take over half the world: it's async architecture. And by the way, benchmarks such as Tech Empower Web Framework still features JavaScript frameworks that outperform Rust frameworks. How do you explain that?
- nicce 2y ago
  > The primary selling point is what makes Node.js take over half the world: it's async architecture. It is the availability of the developers who know the language (JavaScript) (aka cheaper available workforce).
- runevault 2y ago
  Rust has had async for a while (though it can be painful, but I think request/response systems like APIs should not run into a lot of the major footguns). C# has excellent async for asp.net and has for a long time. I haven't touched Java in ages so cannot comment on the JVM ecosystem's async support. So there are other excellent options for async backends that don't have the drawbacks of javascript.
- consteval 2y ago
  I disagree, it's 100% to do with the frontend and pretty much only because of that. Node.js is popular because js is popular. You pretty much guarantee an infinite pool of developers for, like, ever. And you can even use those developers across the entire stack with greater velocity and much less onboarding. async is cool, but not that cool. CGI was doing basically that a long time ago, and it was even more automagical. > Tech Empower Web Framework still features JavaScript frameworks that outperform Rust frameworks. How do you explain that? The benchmarks are constructed in such a way that highlights the strengths of the particular JS JIT implementation. JS is good at a lot of things, so if you just do those things, it might appear that it has okay performance. People do the same thing with C# vs C++; this has been a problem forever. Sure, C# is about as fast or close if you have 16 gigs allocated to the GC and your app is using 100 megs. Now run at 95% memory usage with lots of churning and the order of magnitude differences come out. It's just a fundamental problem with GC langs.
smolder 2y ago
I rewrote the same web API in Javascript, Rust, C#, and Java as a "bench project" at work one time. The Rust version had smallest memory footprint by far as well as the best performance. So, no, "any other language" [than JS] is not all the same.
- manquer 2y ago
  They are not saying every language will have same level of improvement as Rust, they are saying you can most of the improvements is available in most languages. perhaps you get 1300MB to 20 MB with C# or Java or go, and 13MB with rust . Rust’s design is not the reason for bulk of the reduction is the point
  - acdha 2y ago
    Sure, but until people actually have real data that’s just supposition. If a Java rewrite went from 1300MB to, say, 500MB they’d have a valid point and optimizing for RAM consumption is severely contrary to mainstream Java culture.
- jeroenhd 2y ago
  C# and Java are closer but not really on the level of Rust when it comes to performance. A better comparison would be with C++ or a similarly low-level language. In my experience, languages like Ruby and Python are slower than languages like Javascript, which are slower than languages like C#/Java, which are slower than languages like C++/Rust, which are slower than languages like C and Fortran. Assembly isn't always the fastest approach these days, but well-placed assembly can blow C out of the water too. The ease of use and maintainability scale in reverse in my experience, though. I wouldn't want to maintain the equivalent of a quick and dirty RoR server reimplemented in C or assembly, especially after it's grown organically for a few years. Writing Rust can be very annoying when you can't take the normal programming shortcuts because of lifetimes or the borrow checker, in a way that JIT'ed languages allow. Everything is a scale and faster does not necessarily mean better if the code becomes unreadable.
  - 2y ago

adastra22 2y ago

There is no reason data isolation should cost you 100x memory usage.

marcosdumay 2y ago
There are plenty of reasons. They are just not intrinsic to the isolation, instead they come from complications rooted deeply on the underlying system. If you rebuild Linux from the ground up with isolation in mind, you will be able to do it more efficiently. People are indeed in the process of rewriting it, but it's far from complete (and moving back and forward, as not every Linux dev cares about it).
- btilly 2y ago
  Unless you can be concrete and specific about some of those reasons, you're just replacing handwaving with more vigorous handwaving. What is it specifically about JavaScript's implementation of data isolation that, in your mind, helps cause the excessive memory usage?
  - marcosdumay 2y ago
    Just a day or two ago, there was an article here about problems implementing a kind of read-only memory constraint that Javacript benefited from in other OSes.
    - btilly 2y ago
      I must have missed that article. Can you find it? Unless you can come up with a specific reference, it seems unlikely that this would explain the large memory efficiency difference. By contrast it is simple and straightforward to understand why keeping temporary garbage until garbage collection could result in tying up a lot of memory while continually running code that allocates memory and lets it go out of scope. If you search, you'll find lots of references to this happening in a variety of languages.
chipdart 2y ago
> There is no reason data isolation should cost you 100x memory usage. It really depends on what you mean by "memory usage". The fundamental principle of any garbage collection system is that you allocate objects in the heap at will without freeing them until you really need to, and when that time comes you rely on garbage collection strategies to free and move objects. What this means is that processes end up allocating more data that the one being used, just because there is no need to free it. Consequently, with garbage collecting languages you configure processes with a specific memory budget. The larger the budget, the rarer these garbage collection strategies kick in. I run a service written with a garbage collected language. It barely uses more than 100MB of memory to handle a couple hundred requests per minute. The process takes over as much as 2GB of RAM before triggering generation 0 garbage collection events. These events trigger around 2 or 3 times per month. A simplistic critic would argue the service is wasting 10x the memory. That critic would be manifesting his ignorance, because there is absolutely nothing to gain by lowering the memory budget.
- nicoburns 2y ago
  > That critic would be manifesting his ignorance, because there is absolutely nothing to gain by lowering the memory budget. Given that compute is often priced proportional to (maximum) memory usage, there is potentially a lot to be gained: dramatically cheaper hosting costs. Of course if your hosting costs are small to be begin with then this likely isn't worthwhile.
  - chipdart 2y ago
    > Given that compute is often priced proportional to (maximum) memory usage, (...) Let's look at numbers. Hetzner sells vCPUs with 4GB of RAM for less than 5$/month, and 8GB of RAM for less than $10/month. https://www.hetzner.com/cloud/ https://www.hetzner.com/cloud/ In my example, the cost of having garbage collection generation 0 events triggering twice a year would be an extra $5. If I wanted the frequency of these events to double, in theory I would save perhaps $2/month. If I ran a web-scale service with 10 times the nodes as-is, we're talking about a $50/month price tag difference. How much does a company charge for an engineer's hourly labor? How many years would it take to recover the cost of having an engineer tune a service's garbage collection strategy? People need to thing things through before discussing technical merits.

jvanderbot 2y ago

"Rust" really just means "Not javascript" as a recurring pattern in these articles.

noirscape 2y ago
It's also frankly kinda like comparing apples and oranges as a language. JavaScript (and many of the "bad performance" high level languages minus Rails; Rails is bad and should be avoided for projects as much as possible unless you have lots of legacy cruft) are also heavily designed around rapid iteration. Rust is however very much not capable of rapid iteration, the borrow checker will fight you heavily every step of the way to the point where it demands constant refactors. Basically the best place where Rust can work is one where all variables, all requirements and all edgecases are known ahead of time or cases where manual memory safety is a necessity vis-a-vis accepting a minor performance hike from things like the garbage collector. This works well in some spaces (notably; systems programming, embedded and Browser Engines and I wouldn't consider the latter a valid target), but webserver development is probably one of the furthest places where you are looking for Rust.
- hathawsh 2y ago
  I have often thought that programmers can actually just choose to make Rust easy by using a cyclic garbage collector such as Samsara. [1] If cyclic GC in Rust works as well as I think it can, it should be the best option for the majority of high level projects that need fast development with a trade-off of slightly lower efficiency. I suspect we'll see a "hockey stick" adoption curve once everyone figures this out. [1] https://github.com/chc4/samsara https://github.com/chc4/samsara
  - 0cf8612b2e1e 2y ago
    I am still waiting for a scripting language to be bolted on top of Rust. Something that will silently Box all the values so the programmer does not have to think about the Rust specifics, but can still lean on all of the Rust machinery and libraries. If performance/correctness becomes a problem, the scripting layer could be replaced piecemeal with real Rust.

nh2 2y ago

It's important to be aware that often it isn't the programming language that has the biggest effect on memory usage, but simply settings of the memory allocator and OS behaviour. This also means that you cannot "simply measure memory usage" (e.g. using `time` or `htop`) without already having a relatively deep understanding of the underlying mechanisms. Most importantly: libc / malloc implementation: glibc by default has heavy memory fragmentation, especially in multi-threaded programs. It means it will not return `malloc()`ed memory back to the OS when the application `free()`s it, keeping it instead for the next allocation, because that's faster. Its default settings will e.g. favour 10x increased RESident memory usage for 2% speed gain. Some of this can be turned off in glibc using e.g. the env var `MALLOC_MMAP_THRESHOLD_=65536` -- for many applications I've looked at, this instantaneously reduced RES fro 7 GiB to 1 GiB. Some other issues cannot be addressed, because the corresponding glibc tunables are bugged [2]. For jemalloc `MALLOC_CONF=dirty_decay_ms:0,muzzy_decay_ms:0` helps to return memory to the OS immediately. Linux: Memory is generally allocated from the OS using `mmap()`, and returned using `munmap()`. But that can be a bit slow. So some applications and programming language runtimes use instead `madvise(MADV_FREE)`; this effectively returns the memory to the OS, but the OS does not actually do costly mapping table changes unless it's under memory pressure. As a result, one observes hugely increased memory usage in `time` or `htop`. [2] The above means that people are completely unware what actually eats their memory and what the actual resource usage is, easily "measuring wrong" by factor 10x. For example, I've seen people switch between Haskell and Go (both directions) because they thought the other one used less memory. It actually was just the glibc/Linux flags that made the actual difference. Nobody made the effort to really understand what's going on. Same thing for C++. You think without GC you have tight memory control, but in fact your memory is often not returned to the OS when the destructor is called, for the above reason. This also means that the numbers for Rust or JS may easily be wrong (in either direction, or both). So it's quite important to measure memory usage also with the tools above malloc(), otherwise you may just measure the wrong thing. [1]: https://sourceware.org/bugzilla/show_bug.cgi?id=14827 https://sourceware.org/bugzilla/show_bug.cgi?id=14827 [2]: https://downloads.haskell.org/ghc/latest/docs/users_guide/runtime_control.html#rts-flag---disable-delayed-os-memory-return https://downloads.haskell.org/ghc/latest/docs/users_guide/ru...

Capricorn2481 2y ago

btilly 2y ago

Your claim makes zero sense to me. Particularly when I've personally seen similar behavior out of other languages, like Java. As I said in another comment, the most likely cause is that temporary garbage is not collected immediately in JavaScript, while garbage is collected immediately in Rust. See https://doc.rust-lang.org/nomicon/ownership.html https://doc.rust-lang.org/nomicon/ownership.html for the key idea behind how Rust manages this. If you truly believe that it is somehow due to data isolation, then I would appreciate a reference to where JavaScript's design causes it to behave differently.

chipdart 2y ago

> Or, in other words, it's the unavoidable result of insisting on using a language created for the frontend to write everything else. I don't think this is an educated take. The whole selling point of JavaScript in the backend has nothing to do with "frontend" things. The primary selling point is what makes Node.js take over half the world: it's async architecture. And by the way, benchmarks such as Tech Empower Web Framework still features JavaScript frameworks that outperform Rust frameworks. How do you explain that?

nicce 2y ago
> The primary selling point is what makes Node.js take over half the world: it's async architecture. It is the availability of the developers who know the language (JavaScript) (aka cheaper available workforce).
runevault 2y ago
Rust has had async for a while (though it can be painful, but I think request/response systems like APIs should not run into a lot of the major footguns). C# has excellent async for asp.net and has for a long time. I haven't touched Java in ages so cannot comment on the JVM ecosystem's async support. So there are other excellent options for async backends that don't have the drawbacks of javascript.
consteval 2y ago
I disagree, it's 100% to do with the frontend and pretty much only because of that. Node.js is popular because js is popular. You pretty much guarantee an infinite pool of developers for, like, ever. And you can even use those developers across the entire stack with greater velocity and much less onboarding. async is cool, but not that cool. CGI was doing basically that a long time ago, and it was even more automagical. > Tech Empower Web Framework still features JavaScript frameworks that outperform Rust frameworks. How do you explain that? The benchmarks are constructed in such a way that highlights the strengths of the particular JS JIT implementation. JS is good at a lot of things, so if you just do those things, it might appear that it has okay performance. People do the same thing with C# vs C++; this has been a problem forever. Sure, C# is about as fast or close if you have 16 gigs allocated to the GC and your app is using 100 megs. Now run at 95% memory usage with lots of churning and the order of magnitude differences come out. It's just a fundamental problem with GC langs.

smolder 2y ago

I rewrote the same web API in Javascript, Rust, C#, and Java as a "bench project" at work one time. The Rust version had smallest memory footprint by far as well as the best performance. So, no, "any other language" [than JS] is not all the same.

manquer 2y ago
They are not saying every language will have same level of improvement as Rust, they are saying you can most of the improvements is available in most languages. perhaps you get 1300MB to 20 MB with C# or Java or go, and 13MB with rust . Rust’s design is not the reason for bulk of the reduction is the point
- acdha 2y ago
  Sure, but until people actually have real data that’s just supposition. If a Java rewrite went from 1300MB to, say, 500MB they’d have a valid point and optimizing for RAM consumption is severely contrary to mainstream Java culture.
jeroenhd 2y ago
C# and Java are closer but not really on the level of Rust when it comes to performance. A better comparison would be with C++ or a similarly low-level language. In my experience, languages like Ruby and Python are slower than languages like Javascript, which are slower than languages like C#/Java, which are slower than languages like C++/Rust, which are slower than languages like C and Fortran. Assembly isn't always the fastest approach these days, but well-placed assembly can blow C out of the water too. The ease of use and maintainability scale in reverse in my experience, though. I wouldn't want to maintain the equivalent of a quick and dirty RoR server reimplemented in C or assembly, especially after it's grown organically for a few years. Writing Rust can be very annoying when you can't take the normal programming shortcuts because of lifetimes or the borrow checker, in a way that JIT'ed languages allow. Everything is a scale and faster does not necessarily mean better if the code becomes unreadable.
- 2y ago