6 ms·

The difference in avg memory for first and second is substantial, 115.41 MiB Java vs 4.15 MiB Rust.

capableweb 5y ago
Indeed, but is relevant? If you have a machine with average memory installed, both are fine. If you have something with very little memory available, both would be too much.
- rossmohax 5y ago
  It matters for things like AWS Lambda
  - capableweb 5y ago
    If you're out after performance you wouldn't use AWS Lambda or even AWS at all. Go for dedicated hosting, better performance and cheaper.
    - Koiwai 5y ago
      Server management is a significant part of cost. Why would cloud provider be a business in the first place, do you think?
      - capableweb 5y ago
        Sure, it's up to you what you optimize for. Easy and fast to scale vertically or better numbers in terms of latency/through-output? With the former, go for cloud. For the latter, go for dedicated. Want best performance for each buck spent? Again, dedicated.
    - rossmohax 5y ago
      If I am already using Lambda because of infrastructure management costs, it is always nice to reduce my bill by using less memory and/or improving response time.
    - rualca 5y ago
      > If you're out after performance you wouldn't use AWS Lambda or even AWS at all. That point makes as much sense as complaining that if you were after performance you'd use a Formula1 car and not a Tesla. People who live in the real world and have to do real work need to use real world tools, and one of which is AWS Lambda. Let's put things in perspective: would it make any sense at all to advise a company to not only rewrite a whole application stack from scratch in your pet performant language but also jump head on to some boutique service provider? I mean, who in their right mind would get accountants involved in a goal to shave a few milliseconds over a few gRPC calls? Is a suggestion to improve performance expected to be considered even sane if it requires rewriting everything and change shop?
      - capableweb 5y ago
        > People who live in the real world and have to do real work need to use real world tools, and one of which is AWS Lambda. Another real tool is dedicated servers, something people used before "cloud" and something that people who care about performance still uses. AWS even offers dedicated servers themselves, so not sure why you would need to involve any "boutique service provider". Otherwise you have OVH, Hetzner and a range of others who compete well with AWS on dedicated instances as well, neither I'd say are "boutique". > rewrite a whole application stack from scratch in your pet performant language Not sure where this comes from, which one of these languages are "pet performant (SIC) languages"? > who in their right mind would get accountants involved in a goal to shave a few milliseconds over a few gRPC calls Hmm, unless the accountants are involved somehow in the API design (not sure what you're building), I don't know what the accountants have to do with anything here. In the end, AWS and Lambda absolutely does not fit every use case. Depending on your use case, and if it's important a few ms here and there, you chose different solutions. Since this benchmark is about throughoutput, I thought we were discussing the use case of needing the best throughoutput, otherwise this is all off-topic. And with that, I'm just sharing that if that is your focus, you would probably not be using Lambda in the first place, as you'll get very shitty throughoutput and you have to pay a lot, compared to other mature solutions for this that we already had for many many years.
        5y ago
- oaiey 5y ago
  Are not cloud resources often billed memory x seconds?
  - capableweb 5y ago
    If you're optimizing for pricing/billing you wouldn't use cloud in the first place.
- rualca 5y ago
  > Indeed, but is relevant? If you have a machine with average memory installed, both are fine. The whole point is that you don't. You literally pay for the memory your application requires, proportionally to the amount of memory. The difference in the resources required to run is around two orders of magnitude. Let's put things in perspective: in some platforms such as AWS Lambda, you are charged per memory used per second.
  - capableweb 5y ago
    Yeah, sure, that's one use case, to run it on AWS Lambda. Everyone does not, and if you really care about latency/throughoutput, you won't be anywhere near Lambda, and probably not even on AWS or any "cloud" for that matter. Paying for the amount of memory you use down to the MB, is hardly something everyone does, which makes it weird that you are now the third reply that assume this runs on AWS/Lambda.
    - rualca 5y ago
      > Yeah, sure, that's one use case, to run it on AWS Lambda. It's not an edge case. It is a clear and irrefutable example that you pay for the memory you use. Let's be very clear here: when you provision a VM anywhere in the world, you have to pick how much memory you require. You are charged for that memory, proportionally to the memory you require. If your app requires over 100x memory to run, that comes out of your wallet.
      - capableweb 5y ago
        > It's not an edge case Never said it was an edge case.... > when you provision a VM anywhere in the world, you have to pick how much memory you require Yes, thank you! That's exactly my point! You create a VM (or a dedicated instance) somewhere, they ask you for the memory usage up front. Usually they start at 128MB and go their way up from there. Even if you take the instance with the smallest amount of memory, you'll fit any of the benchmarked programs, effective making the "115.41 MiB Java vs 4.15 MiB Rust" statement not as important anymore. > You are charged for that memory, proportionally to the memory you require. If your app requires over 100x memory to run, that comes out of your wallet Hm, maybe on Lambda you pay more if the memory your app uses grows. But this is certainly not standard for "normal" hosting where you rent either a VM or proper instance. Then the memory grows until it cannot take any more memory, and the process either crashes, gets killed by OOM or does whatever operation you've designed it to take when running out of memory.
- adrianN 5y ago
  It matters when you do other stuff than rust running a benchmark. If every piece of your software uses 25x the memory you might end up needing 25x the servers for your application.
  - josefx 5y ago
    Most of that is probably up front cost for a jvm instance. So the 25x only happens because it is a micro benchmark that isn't doing anything useful. Might as well complain that the size of a C executable with an empty main is infinitely larger than a python script that does nothing.
    - jiofih 5y ago
      So you’re saying Java is a bad choice for something like a sidecar (most likely multiple) running along your main app, due to that JVM overhead. I think you’re all agreeing here.
      - josefx 5y ago
        You can probably cut down the default heap size, get it to use 32 bit pointers for a small heap, etc. . The JVM has quite a few startup options that you could use if the memory footprint of hundreds of tiny instances is a bottleneck for you. Might even speed things up a bit more.
        WhatIsDukkha 5y ago
        Unfortunately there is very very little information (last I looked) on the web about tuning down a jvm methodically and sensibly.
- alephu5 5y ago
  I worked in a web scraping team for a couple of years and memory was always the bottleneck since we ran hundreds of thousands of instances. Our scrapers were written in python and with the interpreter + all the imports we had 40MB reserved before even starting work. Not just that, but they were very sensitive to fluctuations.
- spacemanmatt 5y ago
  In practice you might get through the night with a memory-hogging process but that's not how you get to scale.
  - capableweb 5y ago
    How can people make generalized statements like this without anything about the use case or problem that would have to be solved? You have exactly 0 information available and yet you make a comment like this? Is this why we have cargo-culting? Not every solution to a problem needs to use as little memory as possible. And also not every solution can ignore memory usage. It depends, of course. Since this thread is about throughoutput and optimizing for that, I could tell you that most times I've been put in charge for optimizing throughoutput, having low memory usage have been pretty far down the list.
    - spacemanmatt 5y ago
      > Not every solution to a problem needs to use as little memory as possible Those problems are called "one-offs" relative to things I design for scale.
      - capableweb 5y ago
        ... Yes, "relative to things I design" You think that applies to everyone? You think that comes close to applying to people focusing on improving throughoutput specifically?
      - kaba0 5y ago
        Unless you talk about embedded, I have to disagree. RAM is the cheapest resource to scale and it will likely continue to grow without problems. There are server machines with 1TB of RAM available. And in the domain where Java is used, it uses totally apt amounts of RAM compared to other managed languages.
    - papaf 5y ago
      Not every solution to a problem needs to use as little memory as possible. But that's not the point. People aren't suggesting micro managing memory. Java has a well deserved reputation of being a memory hog and its using 20x what is needed in this benchmark.

Indeed, but is relevant? If you have a machine with average memory installed, both are fine. If you have something with very little memory available, both would be too much.

rossmohax 5y ago
It matters for things like AWS Lambda
- capableweb 5y ago
  If you're out after performance you wouldn't use AWS Lambda or even AWS at all. Go for dedicated hosting, better performance and cheaper.
  - Koiwai 5y ago
    Server management is a significant part of cost. Why would cloud provider be a business in the first place, do you think?
    - capableweb 5y ago
      Sure, it's up to you what you optimize for. Easy and fast to scale vertically or better numbers in terms of latency/through-output? With the former, go for cloud. For the latter, go for dedicated. Want best performance for each buck spent? Again, dedicated.
  - rossmohax 5y ago
    If I am already using Lambda because of infrastructure management costs, it is always nice to reduce my bill by using less memory and/or improving response time.
  - rualca 5y ago
    > If you're out after performance you wouldn't use AWS Lambda or even AWS at all. That point makes as much sense as complaining that if you were after performance you'd use a Formula1 car and not a Tesla. People who live in the real world and have to do real work need to use real world tools, and one of which is AWS Lambda. Let's put things in perspective: would it make any sense at all to advise a company to not only rewrite a whole application stack from scratch in your pet performant language but also jump head on to some boutique service provider? I mean, who in their right mind would get accountants involved in a goal to shave a few milliseconds over a few gRPC calls? Is a suggestion to improve performance expected to be considered even sane if it requires rewriting everything and change shop?
    - capableweb 5y ago
      > People who live in the real world and have to do real work need to use real world tools, and one of which is AWS Lambda. Another real tool is dedicated servers, something people used before "cloud" and something that people who care about performance still uses. AWS even offers dedicated servers themselves, so not sure why you would need to involve any "boutique service provider". Otherwise you have OVH, Hetzner and a range of others who compete well with AWS on dedicated instances as well, neither I'd say are "boutique". > rewrite a whole application stack from scratch in your pet performant language Not sure where this comes from, which one of these languages are "pet performant (SIC) languages"? > who in their right mind would get accountants involved in a goal to shave a few milliseconds over a few gRPC calls Hmm, unless the accountants are involved somehow in the API design (not sure what you're building), I don't know what the accountants have to do with anything here. In the end, AWS and Lambda absolutely does not fit every use case. Depending on your use case, and if it's important a few ms here and there, you chose different solutions. Since this benchmark is about throughoutput, I thought we were discussing the use case of needing the best throughoutput, otherwise this is all off-topic. And with that, I'm just sharing that if that is your focus, you would probably not be using Lambda in the first place, as you'll get very shitty throughoutput and you have to pay a lot, compared to other mature solutions for this that we already had for many many years.
      - 5y ago
oaiey 5y ago
Are not cloud resources often billed memory x seconds?
- capableweb 5y ago
  If you're optimizing for pricing/billing you wouldn't use cloud in the first place.
rualca 5y ago
> Indeed, but is relevant? If you have a machine with average memory installed, both are fine. The whole point is that you don't. You literally pay for the memory your application requires, proportionally to the amount of memory. The difference in the resources required to run is around two orders of magnitude. Let's put things in perspective: in some platforms such as AWS Lambda, you are charged per memory used per second.
- capableweb 5y ago
  Yeah, sure, that's one use case, to run it on AWS Lambda. Everyone does not, and if you really care about latency/throughoutput, you won't be anywhere near Lambda, and probably not even on AWS or any "cloud" for that matter. Paying for the amount of memory you use down to the MB, is hardly something everyone does, which makes it weird that you are now the third reply that assume this runs on AWS/Lambda.
  - rualca 5y ago
    > Yeah, sure, that's one use case, to run it on AWS Lambda. It's not an edge case. It is a clear and irrefutable example that you pay for the memory you use. Let's be very clear here: when you provision a VM anywhere in the world, you have to pick how much memory you require. You are charged for that memory, proportionally to the memory you require. If your app requires over 100x memory to run, that comes out of your wallet.
    - capableweb 5y ago
      > It's not an edge case Never said it was an edge case.... > when you provision a VM anywhere in the world, you have to pick how much memory you require Yes, thank you! That's exactly my point! You create a VM (or a dedicated instance) somewhere, they ask you for the memory usage up front. Usually they start at 128MB and go their way up from there. Even if you take the instance with the smallest amount of memory, you'll fit any of the benchmarked programs, effective making the "115.41 MiB Java vs 4.15 MiB Rust" statement not as important anymore. > You are charged for that memory, proportionally to the memory you require. If your app requires over 100x memory to run, that comes out of your wallet Hm, maybe on Lambda you pay more if the memory your app uses grows. But this is certainly not standard for "normal" hosting where you rent either a VM or proper instance. Then the memory grows until it cannot take any more memory, and the process either crashes, gets killed by OOM or does whatever operation you've designed it to take when running out of memory.
adrianN 5y ago
It matters when you do other stuff than rust running a benchmark. If every piece of your software uses 25x the memory you might end up needing 25x the servers for your application.
- josefx 5y ago
  Most of that is probably up front cost for a jvm instance. So the 25x only happens because it is a micro benchmark that isn't doing anything useful. Might as well complain that the size of a C executable with an empty main is infinitely larger than a python script that does nothing.
  - jiofih 5y ago
    So you’re saying Java is a bad choice for something like a sidecar (most likely multiple) running along your main app, due to that JVM overhead. I think you’re all agreeing here.
    - josefx 5y ago
      You can probably cut down the default heap size, get it to use 32 bit pointers for a small heap, etc. . The JVM has quite a few startup options that you could use if the memory footprint of hundreds of tiny instances is a bottleneck for you. Might even speed things up a bit more.
      - WhatIsDukkha 5y ago
        Unfortunately there is very very little information (last I looked) on the web about tuning down a jvm methodically and sensibly.
alephu5 5y ago
I worked in a web scraping team for a couple of years and memory was always the bottleneck since we ran hundreds of thousands of instances. Our scrapers were written in python and with the interpreter + all the imports we had 40MB reserved before even starting work. Not just that, but they were very sensitive to fluctuations.
spacemanmatt 5y ago
In practice you might get through the night with a memory-hogging process but that's not how you get to scale.
- capableweb 5y ago
  How can people make generalized statements like this without anything about the use case or problem that would have to be solved? You have exactly 0 information available and yet you make a comment like this? Is this why we have cargo-culting? Not every solution to a problem needs to use as little memory as possible. And also not every solution can ignore memory usage. It depends, of course. Since this thread is about throughoutput and optimizing for that, I could tell you that most times I've been put in charge for optimizing throughoutput, having low memory usage have been pretty far down the list.
  - spacemanmatt 5y ago
    > Not every solution to a problem needs to use as little memory as possible Those problems are called "one-offs" relative to things I design for scale.
    - capableweb 5y ago
      ... Yes, "relative to things I design" You think that applies to everyone? You think that comes close to applying to people focusing on improving throughoutput specifically?
    - kaba0 5y ago
      Unless you talk about embedded, I have to disagree. RAM is the cheapest resource to scale and it will likely continue to grow without problems. There are server machines with 1TB of RAM available. And in the domain where Java is used, it uses totally apt amounts of RAM compared to other managed languages.
  - papaf 5y ago
    Not every solution to a problem needs to use as little memory as possible. But that's not the point. People aren't suggesting micro managing memory. Java has a well deserved reputation of being a memory hog and its using 20x what is needed in this benchmark.