3 ms·
> If you (or your company) are paying cloud fees for hosting your services (including serverless), you are absolutely throwing money away You're almost certain
by lmm 5y ago
> If you (or your company) are paying cloud fees for hosting your services (including serverless), you are absolutely throwing money away
You're almost certainly not. You almost certainly are nowhere near the RPS limit. You almost certainly are bottlenecked on your own code far more than your framework/language. You're almost certainly running at a performance level of "just barely not bad enough to spend time improving" and would almost certainly be doing the same in any other language. You're probably using a slower framework than the fastest python or ruby framework, so the fact that faster frameworks exist in your language is irrelevant. And if you are in the (rare) case where you're handling enough requests for this choice to matter, then you're big enough to benchmark some proper prototypes that measure the kind of thing your application actually needs to do.
Almost anything is a better use of your time than looking at the benchmarks. And if you are going to look at the benchmarks, you shouldn't look at the RPS total; what you should do is first estimate how many requests your app needs to handle and then apply that as a yes/no filter to the frameworks you're considering. "It can handle 1000x as many requests as we need instead of 100x" is not a good reason to pick a framework, much less a language.
- ezekiel68 5y agoI wish this were true -- but it isn't. Partly because Java and scripting frameworks can easily get memory bound instead of CPU bound. I've worked for several large enterprises running externally accessible services in the cloud (and this is true of my current employer). Of course, all of our services have at least 2 "tasks" or instances to help with fault tolerance, yet most of them do, in fact, auto-scale up to more instances during a day based on either RAM or CPU load. (for the curious, apps which read from message queues tend to get CPU bound while web services in Java or a scripting language tend to get memory bound) If you must run 3 instances instead of 2, that is 50% more instance costs for that time period, etc. Moving a service from Java 11 to rust brought the memory load down sufficiently that we didn't need to autoscale at all. Some other service might autoscale, but to fewer max instances. And this adds up multiplied by days by services.
- lmm 5y agoIt takes a pretty extreme level of fine tuning to hit that window - Java memory usage is generally about 2x equivalent Rust. (It's completely possible your rewrite saved much more than that - but in that case, doing the same rewrite in Java would probably have brought you similar gains). And unless your user number is very static, you're not going to be in that window for long - maybe you push back the point at which you have to implement autoscaling / do some serious profiling work / start shopping around cloud vendors / ... by six months, or a year, which is helpful but not a huge return on rewriting your whole system. A factor of 10x or 100x is maybe worth letting it determine your choice of language. But given that most performance differences come from algorithm selection rather than the low-level details of your runtime, a language that makes it easy to understand what your code is doing is definitely worth giving up a factor of 2x or even 5x for. For a mature codebase where your userbase is not really growing any more, that kind of marginal performance work can be worthwhile - maybe rather than putting off the point at which you have to scale out, it lets you avoid having to scale out at all and saves you 50% on your running costs forever. But that kind of environment is the same kind of environment where it makes sense to put the codebase into maintenance mode and get rid of most of the expensive programmers (perhaps replacing them with outsourcing).