4 ms·

Except when 10s of thousands of requests can be handled in 32GB of RAM versus in Java where you need an instance four times as big to handle a load one fourth t

by ttfkam 2y ago

Except when 10s of thousands of requests can be handled in 32GB of RAM versus in Java where you need an instance four times as big to handle a load one fourth the size. That's annoying when you only need one or two instances. A real dent to your operating costs when the instance count gets much larger. https://instances.vantage.sh/ https://instances.vantage.sh/ A 2GB RAM C7G medium instance costs 2.4¢/hr on a reserved instance. A C7G 16xlarge with 128GB RAM costs $1.53/hr on a reserved instance. It adds up quick. Do you want to pay for the work getting done or pay a Java memory tax?

riku_iki 2y ago
> It adds up quick. Do you want to pay for the work getting done or pay a Java memory tax? for some corp "$1.53/hr" is very small expenses, and it is easier to make things done in more feature rich ecosystem. Also, diffs in memory footprint is closer to 2 times based on various benchmarks: https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/go.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
- neonsunset 2y ago
  Too bad it doesn't include C# by default :) https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/go-csharpcore.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... (it also keeps AOT[0] a specialized workload so you can't directly compare, which brings down the baseline RAM usage from 30 to 19MiB[1], though that is also problematic as the benchmark suite compiles AOT to x86-x64-v1 aka SSE2 (!) which would not run code with SSE4 or AVX2 SIMD implementations, and runs much slower for implementations expecting vectorization in general) https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/csharpcore-csharpaot.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... [1] I have no idea where 30MiB for most cases comes from, make a simple program and run it today and you'll see the RAM use will be below 10 MiB as you launch it (2-4MiB for AOT). The usage gets higher as you start loading dlls/dylibs/so's but those pages can be de-duplicated by OS so not much of an issue
  - igouy 2y ago
    Please provide example C# AOT compilation (Intel® SSE4.1, Intel® SSE4.2, Intel® AVX) [0] https://benchmarksgame-team.pages.debian.net/benchmarksgame/measurements/csharpaot.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/measurements/go.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... [1] rusage, maximum resident set size
    - neonsunset 2y ago
      NativeAOT defaults to x86-x64 (sans v2) unless specified otherwise. If we compare to C and C++ submissions, they mostly use -march=ivybridge with -O3 optimization level. In order to make such comparison fair, you could use the following build options in .NET: <OptimizationPreference>Speed</OptimizationPreference> <IlcInstructionSet>x86-x64-v2</IlcInstructionSet> // (or 'native', to match march=native) These can also be passed as arguments to `dotnet publish` if necessary. Reference: - https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot/optimizing https://learn.microsoft.com/en-us/dotnet/core/deploying/nati... - https://github.com/dotnet/runtime/blob/main/src/coreclr/nativeaot/docs/optimizing.md#options-related-to-code-generation https://github.com/dotnet/runtime/blob/main/src/coreclr/nati... -https://github.com/dotnet/runtime/blob/5b4e770daa190ce69f402c1cf92f388b07d3e144/src/coreclr/tools/Common/JitInterface/ThunkGenerator/InstructionSetDesc.txt https://github.com/dotnet/runtime/blob/5b4e770daa190ce69f402... (full list of recognized keys for IlcInstructionSet)
      - igouy 2y ago
        Done but just + - not a difference.
        neonsunset 2y ago
        Thanks a lot! Now the numbers are much more in line with what is expected from startup cost sensitive benchmarks: https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/csharpcore-csharpaot.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... and supposedly certain AVX-dependent (AVX1 that is, the only kind supported by ivy bridge) implementations no longer fail. If only there was an option to add side-by-side arbitrary options (e.g. Go vs C# AOT), but I suppose that's what contributions are for :)
- igouy 2y ago
  > GraalVM is three times worse by latency during cold starts and double the RAM of Go. GraalVM & Java https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/java-graalvm.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... However, that's for tiny tiny tiny programs like fannkuch-redux and n-body and … that's default settings memory use. When the work requires memory to be allocated k-nucleotide and reverse-complement and … there's less difference between GraalVM & Java.
- ttfkam 2y ago
  Have a nice day. https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/rust-java.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... And that $1.53/hr is per instance. $13,402.80/yr per instance. Now calculate 100 instances. 200. 1,000. Now find a CFO or CTO who wouldn't gladly cut that in half with Go or even more with Rust. At a small enough scale, all technologies work and are affordable. You could run a very small site from your home Comcast network with a Raspberry Pi and bash scripts invoked by CGI. In fact, a lot of sites could do this. But we're talking about software engineering. It's the difference between a steel plate covering a hole in the pavement and a highway bridge. Any ole steel will work just by slapping in down as long as it's a half inch thick. Once you're building bridges, you need to be more and more discriminating in your choices as the size/length increases. Java's advantages in 2005 over C, PHP, Perl, etc. just don't carry the same weight in comparison to alternatives in 2024.
  - riku_iki 2y ago
    > Now calculate 100 instances. 200. 1,000. and 1%, 0.1% and 0.01% of corps need to handle such traffic. And you would still need 50, 100, 500 instances for your Go backend, and such project indeed is better to implement in Rust or cpp. So, the space where it is reasonable to apply Go base on its memory footprint is like 0.002% of cases.

by ttfkam 2y ago

riku_iki 2y ago

> It adds up quick. Do you want to pay for the work getting done or pay a Java memory tax? for some corp "$1.53/hr" is very small expenses, and it is easier to make things done in more feature rich ecosystem. Also, diffs in memory footprint is closer to 2 times based on various benchmarks: https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/go.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

neonsunset 2y ago
Too bad it doesn't include C# by default :) https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/go-csharpcore.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... (it also keeps AOT[0] a specialized workload so you can't directly compare, which brings down the baseline RAM usage from 30 to 19MiB[1], though that is also problematic as the benchmark suite compiles AOT to x86-x64-v1 aka SSE2 (!) which would not run code with SSE4 or AVX2 SIMD implementations, and runs much slower for implementations expecting vectorization in general) https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/csharpcore-csharpaot.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... [1] I have no idea where 30MiB for most cases comes from, make a simple program and run it today and you'll see the RAM use will be below 10 MiB as you launch it (2-4MiB for AOT). The usage gets higher as you start loading dlls/dylibs/so's but those pages can be de-duplicated by OS so not much of an issue
- igouy 2y ago
  Please provide example C# AOT compilation (Intel® SSE4.1, Intel® SSE4.2, Intel® AVX) [0] https://benchmarksgame-team.pages.debian.net/benchmarksgame/measurements/csharpaot.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... https://benchmarksgame-team.pages.debian.net/benchmarksgame/measurements/go.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... [1] rusage, maximum resident set size
  - neonsunset 2y ago
    NativeAOT defaults to x86-x64 (sans v2) unless specified otherwise. If we compare to C and C++ submissions, they mostly use -march=ivybridge with -O3 optimization level. In order to make such comparison fair, you could use the following build options in .NET: <OptimizationPreference>Speed</OptimizationPreference> <IlcInstructionSet>x86-x64-v2</IlcInstructionSet> // (or 'native', to match march=native) These can also be passed as arguments to `dotnet publish` if necessary. Reference: - https://learn.microsoft.com/en-us/dotnet/core/deploying/native-aot/optimizing https://learn.microsoft.com/en-us/dotnet/core/deploying/nati... - https://github.com/dotnet/runtime/blob/main/src/coreclr/nativeaot/docs/optimizing.md#options-related-to-code-generation https://github.com/dotnet/runtime/blob/main/src/coreclr/nati... -https://github.com/dotnet/runtime/blob/5b4e770daa190ce69f402c1cf92f388b07d3e144/src/coreclr/tools/Common/JitInterface/ThunkGenerator/InstructionSetDesc.txt https://github.com/dotnet/runtime/blob/5b4e770daa190ce69f402... (full list of recognized keys for IlcInstructionSet)
    - igouy 2y ago
      Done but just + - not a difference.
      - neonsunset 2y ago
        Thanks a lot! Now the numbers are much more in line with what is expected from startup cost sensitive benchmarks: https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/csharpcore-csharpaot.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... and supposedly certain AVX-dependent (AVX1 that is, the only kind supported by ivy bridge) implementations no longer fail. If only there was an option to add side-by-side arbitrary options (e.g. Go vs C# AOT), but I suppose that's what contributions are for :)
igouy 2y ago
> GraalVM is three times worse by latency during cold starts and double the RAM of Go. GraalVM & Java https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/java-graalvm.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... However, that's for tiny tiny tiny programs like fannkuch-redux and n-body and … that's default settings memory use. When the work requires memory to be allocated k-nucleotide and reverse-complement and … there's less difference between GraalVM & Java.
ttfkam 2y ago
Have a nice day. https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/rust-java.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... And that $1.53/hr is per instance. $13,402.80/yr per instance. Now calculate 100 instances. 200. 1,000. Now find a CFO or CTO who wouldn't gladly cut that in half with Go or even more with Rust. At a small enough scale, all technologies work and are affordable. You could run a very small site from your home Comcast network with a Raspberry Pi and bash scripts invoked by CGI. In fact, a lot of sites could do this. But we're talking about software engineering. It's the difference between a steel plate covering a hole in the pavement and a highway bridge. Any ole steel will work just by slapping in down as long as it's a half inch thick. Once you're building bridges, you need to be more and more discriminating in your choices as the size/length increases. Java's advantages in 2005 over C, PHP, Perl, etc. just don't carry the same weight in comparison to alternatives in 2024.
- riku_iki 2y ago
  > Now calculate 100 instances. 200. 1,000. and 1%, 0.1% and 0.01% of corps need to handle such traffic. And you would still need 50, 100, 500 instances for your Go backend, and such project indeed is better to implement in Rust or cpp. So, the space where it is reasonable to apply Go base on its memory footprint is like 0.002% of cases.