2 ms·

I wish this were true -- but it isn't. Partly because Java and scripting frameworks can easily get memory bound instead of CPU bound. I've worked for several

I wish this were true -- but it isn't. Partly because Java and scripting frameworks can easily get memory bound instead of CPU bound. I've worked for several large enterprises running externally accessible services in the cloud (and this is true of my current employer). Of course, all of our services have at least 2 "tasks" or instances to help with fault tolerance, yet most of them do, in fact, auto-scale up to more instances during a day based on either RAM or CPU load. (for the curious, apps which read from message queues tend to get CPU bound while web services in Java or a scripting language tend to get memory bound) If you must run 3 instances instead of 2, that is 50% more instance costs for that time period, etc. Moving a service from Java 11 to rust brought the memory load down sufficiently that we didn't need to autoscale at all. Some other service might autoscale, but to fewer max instances. And this adds up multiplied by days by services.

lmm 5y ago
It takes a pretty extreme level of fine tuning to hit that window - Java memory usage is generally about 2x equivalent Rust. (It's completely possible your rewrite saved much more than that - but in that case, doing the same rewrite in Java would probably have brought you similar gains). And unless your user number is very static, you're not going to be in that window for long - maybe you push back the point at which you have to implement autoscaling / do some serious profiling work / start shopping around cloud vendors / ... by six months, or a year, which is helpful but not a huge return on rewriting your whole system. A factor of 10x or 100x is maybe worth letting it determine your choice of language. But given that most performance differences come from algorithm selection rather than the low-level details of your runtime, a language that makes it easy to understand what your code is doing is definitely worth giving up a factor of 2x or even 5x for. For a mature codebase where your userbase is not really growing any more, that kind of marginal performance work can be worthwhile - maybe rather than putting off the point at which you have to scale out, it lets you avoid having to scale out at all and saves you 50% on your running costs forever. But that kind of environment is the same kind of environment where it makes sense to put the codebase into maintenance mode and get rid of most of the expensive programmers (perhaps replacing them with outsourcing).

lmm 5y ago

It takes a pretty extreme level of fine tuning to hit that window - Java memory usage is generally about 2x equivalent Rust. (It's completely possible your rewrite saved much more than that - but in that case, doing the same rewrite in Java would probably have brought you similar gains). And unless your user number is very static, you're not going to be in that window for long - maybe you push back the point at which you have to implement autoscaling / do some serious profiling work / start shopping around cloud vendors / ... by six months, or a year, which is helpful but not a huge return on rewriting your whole system. A factor of 10x or 100x is maybe worth letting it determine your choice of language. But given that most performance differences come from algorithm selection rather than the low-level details of your runtime, a language that makes it easy to understand what your code is doing is definitely worth giving up a factor of 2x or even 5x for. For a mature codebase where your userbase is not really growing any more, that kind of marginal performance work can be worthwhile - maybe rather than putting off the point at which you have to scale out, it lets you avoid having to scale out at all and saves you 50% on your running costs forever. But that kind of environment is the same kind of environment where it makes sense to put the codebase into maintenance mode and get rid of most of the expensive programmers (perhaps replacing them with outsourcing).