10 ms·

In my opinion, the significant drop in memory footprint is truly underrated (13 MB vs 1300 MB). If everybody cared about optimizing for efficiency and performan

by xyst 2y ago

In my opinion, the significant drop in memory footprint is truly underrated (13 MB vs 1300 MB). If everybody cared about optimizing for efficiency and performance, the cost of computing wouldn’t be so burdensome. Even self-hosting on an rpi becomes viable.

leeoniya 2y ago
fwiw, Bun/webkit is much better in mem use if your code is written in a way that avoids creating new strings. it won't be a 100x improvement, but 5x is attainable.
- dangsux 2y ago
  [dead]
jchw 2y ago
It's a little more nuanced than that of course, a big reason why the memory usage is so high is because Node.JS needs more of it to take advantage of a large multicore machine for compute-intensive tasks. > Regarding the abnormally high memory usage, it's because I'm running Node.js in "cluster mode", which spawns 12 processes for each of the 12 CPU cores on my test machine, and each process is a standalone Node.js instance which is why it takes up 1300+ MB of memory even though we have a very simple server. JS is single-threaded so this is what we have to do if we want a Node.js server to make full use of a multi-core CPU. On a Raspberry Pi you would certainly not need so many workers even if you did care about peak throughput, I don't think any of them have >4 CPU threads. In practice I do run Node.JS and JVM-based servers on Raspberry Pi (although not Node.JS software that I personally have written.) The bigger challenge to a decentralized Internet where everyone self-hosts everything is, well, everything else. Being able to manage servers is awesome. Actually managing servers is less glorious, though: - Keeping up with the constant race of security patching. - Managing hardware. Which, sometimes, fails. - Setting up and testing backup solutions. Which can be expensive. - Observability and alerting; You probably want some monitoring so that the first time you find out your drives are dying isn't months after SMART would've warned you. Likewise, you probably don't want to find out you have been compromised after your ISP warns you about abuse months into helping carry out criminal operations. - Availability. If your home internet or power goes out, self-hosting makes it a bigger issue than it normally would be. I love the idea of a world where everyone runs their own systems at home, but this is by far the worst consequence. Imagine if all of your e-mails bounced while the power was out. Some of these problems are actually somewhat tractable to improve on but the Internet and computers in general marched on in a different more centralized direction. At this point I think being able to write self-hostable servers that are efficient and fast is actually not the major problem with self-hosting. I still think people should strive to make more efficient servers of course, because some of us are going to self-host anyways, and Raspberry Pis run longer on battery than large rack servers do. If Rust is the language people choose to do that, I'm perfectly content with that. However, it's worth noting that it doesn't have to be the only one. I'd be just as happy with efficient servers in Zig or Go. Or Node.JS/alternative JS-based runtimes, which can certainly do a fine job too, especially when the compute-intensive tasks are not inside of the event loop.
- bombela 2y ago
  > Imagine if all of your e-mails bounced while the power was out. Retry for a while until the destination becomes reachable again. That's how email was originally designed.
  - jasode 2y ago
    >Retry for a while until the destination becomes reachable again. That's how email was originally designed. Sure, the SMTP email protocol states guidelines for "retries" but senders don't waste resources retrying forever. E.g. max of 5 days: https://serverfault.com/questions/756086/whats-the-usual-re-sending-time-period-for-smtp-servers/756093#756093 https://serverfault.com/questions/756086/whats-the-usual-re-... So gp's point is that if your home email server is down for an extended power outage (maybe like a week from a bad hurricane) ... and you miss important emails (job interview appointments, bank fraud notifications, etc) ... then that's one of the risks of running an email server on the Raspberry Pi at home. Switching to a more energy-efficient language like Rust for server apps so it can run on RPi still doesn't alter the risk calculation above. In other words, many users would still prioritize email reliability of Gmail in the cloud over the self-hosted autonomy of a RPi at home.
    - School-Cotton 2y ago
      Another probably even bigger reason people don't self-host email specifically is that practically all email coming from a residential IP is spam from botnets, so email providers routinely block residential IPs.
    - jchw 2y ago
      Yeah, exactly this. The natural disaster in North Carolina is a great example of how I envision this going very badly. When you self-host at home, you just can't have the same kind of redundancy that data centers have. I don't think it's an obstacle that's absolutely insurmountable, but it feels like something where we would need to organize the entire Internet around solving problems like these. My personal preference would be to have devices act more independently. e.g. It's possible to sync your KeepassXC with SyncThing at which point any node is equal and thus only if you lose all of your devices simultaneously (e.g. including your mobile computer(s)) are you at risk of any serious trouble. (And it's easy to add new devices to back things up if you are especially worried about that.) I would like it if that sort of functionality could be generalized and integrated into software. For something like e-mail, the only way I can envision this working is if any of your devices could act as a destination in the event of a serious outage. I suspect this would be possible to accomplish to some degree today, but it is probably made a lot harder by two independent problems (IPv4 exhaustion/not having directly routable IPs on devices, mobile devices "roaming" through different IP addresses) which force you to rely on some centralized infrastructure anyways (e.g. something like Tailscale Funnels.) I for one welcome whoever wants to take on the challenge of making it possible to do reliable, durable self-hosting of all of my services without the pain. I would be an early adopter without question.
echoangle 2y ago
If every developer cared for optimizing efficiency and performance, development would become slower and more expensive though. People don’t write bad-performing code because it’s fun but because it’s easier. If hardware is cheap enough, it can be advantageous to quickly write slow code and get a big server instead of spending days optimizing it to save $100 on servers. When scaling up, the tradeoff has to be reconsidered of course.
- throwaway19972 2y ago
  Yea but we also write the same software over and over and over and over again. Perhaps slower, more methodical development might enable more software to be written fewer times. (Does not apply to commercially licensed software or services obviously, which is straight waste.)
  - Capricorn2481 2y ago
    > Perhaps slower, more methodical development might enable more software to be written fewer times I don't see why. People will just discover they rewrote something slower.
  - chaxor 2y ago
    This is a decent point, but in many cases writing software over again can be a great thing, even in replaceing some very well established software. The trick is getting everyone to switch over and ensure correct security and correctness for the newer software. A good example may be openssh. It is very well established, so many will use it - but it has had some issues over the years, and due to that, it is actually _very_ difficult now to know what the _correct_ way to configure it for the best, modern, performant, and _secure_ operation. There are hundreds of different options for it, almost all of them existing for 'legacy reasons' (in other words no one should ever use in any circumstance that requires any security). Then along comes things like mosh or dropbear, which seem like they _may_ improve security, but still basically do the same thing as openssh, so it is unclear if they have a the same security problems and simply don't get reported due to lower use, or if they aren't vulnerable. While simultaneously, things like quicssh-rs rewrite the idea but completely differently, such that it is likely far, far more secure (and importantly simpler!), but getting more eyes on it for security is still important. So effectively, having things like Linux move to Rust (but as the proper foundation rather than some new and untrusted entity) can be great when considering any 'rewrite' of software, not only for removing the cruft that we now know shouldn't be used due to having better solutions (enforce using only best and modern crypto or filesystems, and so on), but also to remodel the software to be more simple, cleaner, concise, and correct.
marcosdumay 2y ago
It's the result of the data isolation above anything else attitude of Javascript. Or, in other words, it's the unavoidable result of insisting on using a language created for the frontend to write everything else. You don't need to rewrite your code in Rust to get that saving. Any other language will do. (Personally, I'm surprised all the gains are so small. Looks like it's a very well optimized code path.)
- adastra22 2y ago
  There is no reason data isolation should cost you 100x memory usage.
  - marcosdumay 2y ago
    There are plenty of reasons. They are just not intrinsic to the isolation, instead they come from complications rooted deeply on the underlying system. If you rebuild Linux from the ground up with isolation in mind, you will be able to do it more efficiently. People are indeed in the process of rewriting it, but it's far from complete (and moving back and forward, as not every Linux dev cares about it).
    - btilly 2y ago
      Unless you can be concrete and specific about some of those reasons, you're just replacing handwaving with more vigorous handwaving. What is it specifically about JavaScript's implementation of data isolation that, in your mind, helps cause the excessive memory usage?
      - marcosdumay 2y ago
        Just a day or two ago, there was an article here about problems implementing a kind of read-only memory constraint that Javacript benefited from in other OSes.
        btilly 2y ago
beached_whale 2y ago
Im ok if it isnt popular. It will keep compute costs lower for those using it as the norm is excessive usage
btilly 2y ago
That's because you're churning temporary memory. JS can't free it until garbage collection runs. Rust is able to do a lifetime analysis, and knows it can free it immediately. The same will happen on any function where you're calling functions over and over again that create transient data which later gets discarded.
throwitaway1123 2y ago
There are flags you can set to tune memory usage (notably V8's --max-old-space-size for Node and the --smol flag for Bun). And of course in advanced scenarios you can avoid holding strong references to objects with weak maps, weak sets, and weak refs.
palata 2y ago
> If everybody cared about optimizing for efficiency and performance The problem is that most developers are not capable of optimizing for efficiency and performance. Having more powerful hardware has allowed us to make software frameworks/libraries that make programming a lot more accessible. At the same time lowering the quality of said software. Doesn't mean that all software is bad. Most software is bad, that's all.

by xyst 2y ago

leeoniya 2y ago

fwiw, Bun/webkit is much better in mem use if your code is written in a way that avoids creating new strings. it won't be a 100x improvement, but 5x is attainable.

dangsux 2y ago
[dead]

jchw 2y ago

It's a little more nuanced than that of course, a big reason why the memory usage is so high is because Node.JS needs more of it to take advantage of a large multicore machine for compute-intensive tasks. > Regarding the abnormally high memory usage, it's because I'm running Node.js in "cluster mode", which spawns 12 processes for each of the 12 CPU cores on my test machine, and each process is a standalone Node.js instance which is why it takes up 1300+ MB of memory even though we have a very simple server. JS is single-threaded so this is what we have to do if we want a Node.js server to make full use of a multi-core CPU. On a Raspberry Pi you would certainly not need so many workers even if you did care about peak throughput, I don't think any of them have >4 CPU threads. In practice I do run Node.JS and JVM-based servers on Raspberry Pi (although not Node.JS software that I personally have written.) The bigger challenge to a decentralized Internet where everyone self-hosts everything is, well, everything else. Being able to manage servers is awesome. Actually managing servers is less glorious, though: - Keeping up with the constant race of security patching. - Managing hardware. Which, sometimes, fails. - Setting up and testing backup solutions. Which can be expensive. - Observability and alerting; You probably want some monitoring so that the first time you find out your drives are dying isn't months after SMART would've warned you. Likewise, you probably don't want to find out you have been compromised after your ISP warns you about abuse months into helping carry out criminal operations. - Availability. If your home internet or power goes out, self-hosting makes it a bigger issue than it normally would be. I love the idea of a world where everyone runs their own systems at home, but this is by far the worst consequence. Imagine if all of your e-mails bounced while the power was out. Some of these problems are actually somewhat tractable to improve on but the Internet and computers in general marched on in a different more centralized direction. At this point I think being able to write self-hostable servers that are efficient and fast is actually not the major problem with self-hosting. I still think people should strive to make more efficient servers of course, because some of us are going to self-host anyways, and Raspberry Pis run longer on battery than large rack servers do. If Rust is the language people choose to do that, I'm perfectly content with that. However, it's worth noting that it doesn't have to be the only one. I'd be just as happy with efficient servers in Zig or Go. Or Node.JS/alternative JS-based runtimes, which can certainly do a fine job too, especially when the compute-intensive tasks are not inside of the event loop.

bombela 2y ago
> Imagine if all of your e-mails bounced while the power was out. Retry for a while until the destination becomes reachable again. That's how email was originally designed.
- jasode 2y ago
  >Retry for a while until the destination becomes reachable again. That's how email was originally designed. Sure, the SMTP email protocol states guidelines for "retries" but senders don't waste resources retrying forever. E.g. max of 5 days: https://serverfault.com/questions/756086/whats-the-usual-re-sending-time-period-for-smtp-servers/756093#756093 https://serverfault.com/questions/756086/whats-the-usual-re-... So gp's point is that if your home email server is down for an extended power outage (maybe like a week from a bad hurricane) ... and you miss important emails (job interview appointments, bank fraud notifications, etc) ... then that's one of the risks of running an email server on the Raspberry Pi at home. Switching to a more energy-efficient language like Rust for server apps so it can run on RPi still doesn't alter the risk calculation above. In other words, many users would still prioritize email reliability of Gmail in the cloud over the self-hosted autonomy of a RPi at home.
  - School-Cotton 2y ago
    Another probably even bigger reason people don't self-host email specifically is that practically all email coming from a residential IP is spam from botnets, so email providers routinely block residential IPs.
  - jchw 2y ago
    Yeah, exactly this. The natural disaster in North Carolina is a great example of how I envision this going very badly. When you self-host at home, you just can't have the same kind of redundancy that data centers have. I don't think it's an obstacle that's absolutely insurmountable, but it feels like something where we would need to organize the entire Internet around solving problems like these. My personal preference would be to have devices act more independently. e.g. It's possible to sync your KeepassXC with SyncThing at which point any node is equal and thus only if you lose all of your devices simultaneously (e.g. including your mobile computer(s)) are you at risk of any serious trouble. (And it's easy to add new devices to back things up if you are especially worried about that.) I would like it if that sort of functionality could be generalized and integrated into software. For something like e-mail, the only way I can envision this working is if any of your devices could act as a destination in the event of a serious outage. I suspect this would be possible to accomplish to some degree today, but it is probably made a lot harder by two independent problems (IPv4 exhaustion/not having directly routable IPs on devices, mobile devices "roaming" through different IP addresses) which force you to rely on some centralized infrastructure anyways (e.g. something like Tailscale Funnels.) I for one welcome whoever wants to take on the challenge of making it possible to do reliable, durable self-hosting of all of my services without the pain. I would be an early adopter without question.

echoangle 2y ago

If every developer cared for optimizing efficiency and performance, development would become slower and more expensive though. People don’t write bad-performing code because it’s fun but because it’s easier. If hardware is cheap enough, it can be advantageous to quickly write slow code and get a big server instead of spending days optimizing it to save $100 on servers. When scaling up, the tradeoff has to be reconsidered of course.

throwaway19972 2y ago
Yea but we also write the same software over and over and over and over again. Perhaps slower, more methodical development might enable more software to be written fewer times. (Does not apply to commercially licensed software or services obviously, which is straight waste.)
- Capricorn2481 2y ago
  > Perhaps slower, more methodical development might enable more software to be written fewer times I don't see why. People will just discover they rewrote something slower.
- chaxor 2y ago
  This is a decent point, but in many cases writing software over again can be a great thing, even in replaceing some very well established software. The trick is getting everyone to switch over and ensure correct security and correctness for the newer software. A good example may be openssh. It is very well established, so many will use it - but it has had some issues over the years, and due to that, it is actually _very_ difficult now to know what the _correct_ way to configure it for the best, modern, performant, and _secure_ operation. There are hundreds of different options for it, almost all of them existing for 'legacy reasons' (in other words no one should ever use in any circumstance that requires any security). Then along comes things like mosh or dropbear, which seem like they _may_ improve security, but still basically do the same thing as openssh, so it is unclear if they have a the same security problems and simply don't get reported due to lower use, or if they aren't vulnerable. While simultaneously, things like quicssh-rs rewrite the idea but completely differently, such that it is likely far, far more secure (and importantly simpler!), but getting more eyes on it for security is still important. So effectively, having things like Linux move to Rust (but as the proper foundation rather than some new and untrusted entity) can be great when considering any 'rewrite' of software, not only for removing the cruft that we now know shouldn't be used due to having better solutions (enforce using only best and modern crypto or filesystems, and so on), but also to remodel the software to be more simple, cleaner, concise, and correct.

marcosdumay 2y ago

It's the result of the data isolation above anything else attitude of Javascript. Or, in other words, it's the unavoidable result of insisting on using a language created for the frontend to write everything else. You don't need to rewrite your code in Rust to get that saving. Any other language will do. (Personally, I'm surprised all the gains are so small. Looks like it's a very well optimized code path.)

adastra22 2y ago
There is no reason data isolation should cost you 100x memory usage.
- marcosdumay 2y ago
  There are plenty of reasons. They are just not intrinsic to the isolation, instead they come from complications rooted deeply on the underlying system. If you rebuild Linux from the ground up with isolation in mind, you will be able to do it more efficiently. People are indeed in the process of rewriting it, but it's far from complete (and moving back and forward, as not every Linux dev cares about it).
  - btilly 2y ago
    Unless you can be concrete and specific about some of those reasons, you're just replacing handwaving with more vigorous handwaving. What is it specifically about JavaScript's implementation of data isolation that, in your mind, helps cause the excessive memory usage?
    - marcosdumay 2y ago
      Just a day or two ago, there was an article here about problems implementing a kind of read-only memory constraint that Javacript benefited from in other OSes.
      - btilly 2y ago

beached_whale 2y ago

Im ok if it isnt popular. It will keep compute costs lower for those using it as the norm is excessive usage

btilly 2y ago

That's because you're churning temporary memory. JS can't free it until garbage collection runs. Rust is able to do a lifetime analysis, and knows it can free it immediately. The same will happen on any function where you're calling functions over and over again that create transient data which later gets discarded.

throwitaway1123 2y ago

There are flags you can set to tune memory usage (notably V8's --max-old-space-size for Node and the --smol flag for Bun). And of course in advanced scenarios you can avoid holding strong references to objects with weak maps, weak sets, and weak refs.

palata 2y ago

> If everybody cared about optimizing for efficiency and performance The problem is that most developers are not capable of optimizing for efficiency and performance. Having more powerful hardware has allowed us to make software frameworks/libraries that make programming a lot more accessible. At the same time lowering the quality of said software. Doesn't mean that all software is bad. Most software is bad, that's all.