23 ms·

“Sustainability with Rust” post misleading about Go

190 points by 0xedb 5y ago

cyb_ 5y ago
Alternative way to read the Twitter thread: https://nitter.net/_rsc/status/1496352325157457922 https://nitter.net/_rsc/status/1496352325157457922
- kevincox 5y ago
  That is a lot cleaner than the original. I still prefer https://threadreaderapp.com/thread/1496352325157457922.html https://threadreaderapp.com/thread/1496352325157457922.html though which removes almost all of the chrome between sentences.
belter 5y ago
jonathanstrange 5y ago
This doesn't surprise me. Rust's toxic community was one of several reasons that made me turn away from the language. When people ask on a Go forum (e.g. r/golang) whether they should switch to or write a project in Go, they tend to get good and professional advice, and often this advice is to stick with the existing language and approach. I don't know why this is so, but with Rust you mostly get zealots with zero ability to evaluate the Pros and Cons of their language of choice.
- Xorlev 5y ago
  Painting an entire community with the same brush seems to itself be fairly toxic behavior berift of signal. Let's keep focused on AWS's poor greenwashing post. You could replace all the names of the technologies and it's still just as bad.
  - geodel 5y ago
    > Painting an entire community with the same brush seems to itself be fairly toxic behavior berift of signal. Communities are painted toxic all the time rightly or wrongly. To say community have feeling and are to treated as person does not seem to be appropriate. Also calling people toxic who called some community toxic is akin to name calling and I dare say much more common in Rust circles.
- mcronce 5y ago
  That's not really consistent with my experience in the Rust community at all. Memes aside, rewrites for the sake of rewrites are typically discouraged. When language comparisons are made, they tend to clearly highlight objective and subjective pros/cons. In particular, the referenced AWS article is typically called out as a bad "study" Maybe it was different a few years ago, but this is what I've seen since late 2019ish.
  - mohanmcgeek 5y ago
    Rewrites for the sake of rewrites is a HN thing. This used to happen with other languages too. In 2015, Front page posts on a given day would have "xyz in Go" and it was JavaScript before that. People tend to reimplement systems they know in languages they are learning and it's perfectly alright. But, these are engineers at huge companies rewriting core system components for dubious reasons AND writing a deliberately misleading blog post about it. Now more engineers with an agenda would cite these posts to their managers and get their rewrites.
    - lanstin 5y ago
      Rewriting a small thing you know well is a fabulous way to understand the new language, because your focus isn't on "what should the code do" but "how do I make the code do this thing." I've had senior tech leadership explain that rewriting a big system into a new language is done because the real goal is to rewrite the old system that no one understands well enough and is gradually becoming scary to change; people are enthused for the shiny new language, and at the end the current organization understands this crucial system for a few more years. People in companies write the blog posts because it helps get promotions ("visibility" "industry leadership").
      - mohanmcgeek 5y ago
        > People in companies write the blog posts because it helps get promotions ("visibility" "industry leadership"). This is definitely toxic work culture that needs to be fixed
    - 5y ago
- pohl 5y ago
  I think Rust has grown way beyond the point where it makes sense to characterize the community as a whole. I imagine you could get entirely different experiences depending upon whether you were engaging with people in some HN thread, or Reddit, or IRC, or Discord,...and even day by day or thread by thread. Where specifically did you encounter the toxicity?
- the__alchemist 5y ago
  You can use the language and ignore the parts of the community you don't like. I've made similar observations, but still conclude the language is top-notch. It also seems like a large portion of the posts I see in embedded Rust are about generics and/or async. I have no interest in them. (Most of the embedded code I've seen using them can be written more plainly without). It also turns new people off the language due to the syntax clutter. So, they think "Rust is hard to write/messy". It's not the language - it's how parts of the community use it. I also dislike the condescending attitude when C code comes up .
  - mustache_kimono 5y ago
    I'll say as someone who leans more towards Rust, and its community, in this somewhat manufactured debate -- I think it's pretty hard not to be a little prickly when most of the criticism amounts to "I hate the hype!" or "I don't like change!" My opinion has always been: Be Confident Enough to Allow Others to Disagree. Some in the Rust community sometimes have a problem with this, but, my God, it's not as if the arguments against Rust are very good, or as if people don't love to troll Rust, like the parent to your comment. Most criticisms are embarrassingly flimsy.
    - Yoric 5y ago
      (to complete what mustache_kimono wrote) Which is not to say that Rust doesn't have a number of downsides. And I'm writing this as a Rust enthusiast and (very minor) contributor to the Rust compiler.
    - qsdf38100 5y ago
      I hate the hype. I like honesty, pros vs cons. Rust community seems full of zealots and evangelists. Those people are very vocal about pros, and condescending. As soon as someone brings any con or isn’t convinced, they are just bad programmers, with bad arguments, and even « embarrassingly flimsy » criticism…
      - mustache_kimono 5y ago
- dataangel 5y ago
  If you read the full context, the misunderstanding comes from research not done by the Rust community. You just found a way to reinforce your own bias.
- 0xFEE1DEAD 5y ago
  Every interaction I’ve had with the rust community was toxic in nature. Also for whatever reason all those „rust vs go“ articles are always coming from the rust corner. It’s like the rust community is frustrated about go success and they individually feel the need to attack their perceived competitor meanwhile the go community has no issue with any other language.
  - pyjarrett 5y ago
    [deleted]
    - kitkat_new 5y ago
      I don't get the logic of dropping a language you find great and you are productive in because of a few people
      - busterarm 5y ago
        You'll eventually have to cooperate with those people, whether it's on OSS or in the workplace. It's even worse when really senior figures in a community are also the source of the toxicity.
        deleted 5y ago
        [deleted]
      - sirwhinesalot 5y ago
        Those people will (potentially) be your future co-workers... So it is a valid concern. That said, I have not interacted with the Rust community and cannot make any comments in that regard. Language is pretty nice though.
        deleted 5y ago
        [deleted]
      - pyjarrett 5y ago
- tashoecraft 5y ago
  I think Rust has attracted a lot of people who don't really program in it often, but try to advocate for it. My experiences with the community have always been good, and they are consistently rated as having a really great community, but a growing community will always have issues.
  - ziml77 5y ago
    I’m fairly certain this is the case. The people who seem to want everything rewritten in Rust are not necessarily also people who actually use the language. They likely just heard that it improves security and stability to use Rust over C. Would be similar to the people who bring up Ada as an alternative; the chances that they’ve actually tried it are minuscule. And to be very clear I’m not trying to say there is no overlap at all, but I don’t think you can call the Rust community toxic because of those people. (I wonder if the people who treat Rust like a religion realize how harmful their behavior is. I want Rust to be seen as a reasonable option, but that’s difficult when you have the evangelists scaring people away)
- kitkat_new 5y ago
  In my experience, the Rust community isn't toxic at all. To the contrary: when I started to explore Rust, it was very welcoming, helped finding the answer to all my nooby questions, which they probably heard a dozen times - despite having a lot of C++ and JS baggage
  - MisterTea 5y ago
    You have a veery one sided perspective. Someone adopting/learning a language will be met with plenty of positivity and encouragement. The toxicity comes in to play once you post something critical of the language, even if it has merit.
    - Yoric 5y ago
      So you're saying that people leaving comments on (I assume) blogs are typically less civil than people who take the time to work with newcomers and actually make the community work? That is certainly true, but for pretty much every community that I know, not limited to any programming language or technology.
  - tayo42 5y ago
    Did you see what happened to the actix project. That was embarrassing
    - Yoric 5y ago
      What happened to the actix project? Asking, because I use it daily :)
      - tayo42 5y ago
        It used to use a bunch of unsafe, someone decided that wasn't acceptable and it blew up into a whole thing where people started dog piling on the project and the maintainer. everyone got emotional, he eventually quit and gave up actix to someone else i think. pretty unnecessary and childish the way people acted towards him imo. a lot happened on reddit and github so it should be easy to find.
- odiroot 5y ago
  I don't know which community is toxic and wouldn't even risk using such strong term. Still, I have to say, it's really weird to see (last few months) everything being rewritten into Rust, just for the sake of pride(?). Haven't we learned anything from the NodeJS craze?
  - josephcsible 5y ago
    > just for the sake of pride(?). It's to prevent all memory safety bugs.
    - kangalioo 5y ago
      Partly true, but this doesn't apply to many rewritten tools, like cat or ls where memory safety bugs are tolerable
      - ansible 5y ago
        Well, I don't know about the cat command (though that _is_ appropriate for something like the Redox OS, which aspires to be a pure-Rust desktop operating system). But there is a broad-based desire to rewrite libraries in Rust for a couple (IMHO valid) reasons. One is the API. While it is possible to have a nice and idiomatic Rust API to a C library, it will often be nicer to use when it is designed from the ground-up in Rust. Traits, interfaces, etc. It is also often desirable to avoid C when cross-compiling (such as for embedded systems), as that is generally a hassle to set up and get working seamlessly.
      - detaro 5y ago
        Doesn't sound like things where you'd want bugs either. But also the rewrites for this kind of thing are usually not that "serious". A few people do it because they can or because they think it's interesting, but it's not like there are large pushes to replace such tools (E.g. there's been "lets rewrite basic tools in Go" projects too, and nobody is surprised that they aren't in widespread use"). The winners usually are not in that category, but tools that stand alone.
      - 5y ago
- lnxg33k1 5y ago
  To some extent I agree with you, I don't like the rust community generically, as I feel the toxicity too, but I mean the language and the offering of a fast but easy and checked platform to be are still good enough to let me just ignore them, I mean on IRC you always find people who are going to be bad, and people who will just help, and who cares about those who are bad, I guess just feel a few seconds of pity for them and move on
  - Yoric 5y ago
    Out of curiosity, where did you see any toxicity? I'm both on the Rust forums and on the Rust Matrix community and I haven't seen anything toxic at all. Everything I've seen that looked like it could become toxic was handled gently by someone with a bit of seniority.
- burnished 5y ago
  Its weird, because the negative stuff I see around the Rust community is posts like this with name calling or histrionics, and the positive stuff is like 'thoughtful post mortem about why Rust compile times are so bad (but are improving)'.
- Yoric 5y ago
  Really? I've never found a community as nice as the Rust one. Where did you ask?
jerf 5y ago
When I first read that, I thought the core conclusions were still mostly correct, but the delta claimed for Rust over Go was too large. Rust will certainly beat Go on this metric (I expect Rust to even beat C and C++ at scale [1]), but by a factor of 3 is probably on the large side. There are other things that I would consider questionable as well; Haskell might be capable of beating Go if you really, really work at it, but what you might call "casually written Haskell" isn't going to perform anywhere near as well as casually written Go. And the degree of knowledge required to write high-performance Haskell is much higher than high-performance Go. You have to have more than just knowledge of Haskell as a language, you have to have a fairly deep understanding of exactly how GHC compiles and optimizes it. This can be scaled to the Shootout, but scaling it to real code is very difficult. Still, I think the general point is well-taken. I'm less worried about the environmental impact of my code because my code runs at scales where even if my code was running for "free" from an environmental impact, it wouldn't save much of anything. But having used Go for a while, I'd have a really difficult time moving back to a language that is literally dozens of times slower, and, yes, consumes dozens of times more energy as a result. Part of the reason why my code runs at such small scales is that it's reasonably efficient code. I have a couple of services that I consider trivial, but if I were writing them in Python, would be non-trivial. Still not exactly monsters that would be eating clusters of machines at a time, but there's still a lot of practical operational difference between "This occasionally spikes to 25% of a CPU for a minute at high loads" and "This occasionally spikes to consuming 10 full CPUs for a minute at high load", and that's a fairly reasonable multiplier. I have several services that would have deltas like that. [1]: A previous comment of mine: https://news.ycombinator.com/item?id=25625362 https://news.ycombinator.com/item?id=25625362
- newaccount2021 5y ago
- melony 5y ago
  Go is at about the same abstraction level as Java, maybe a fraction slower due to lack of runtime VM optimisations.
  - 5y ago
rectang 5y ago
> Most obviously, if your study claims that C++ uses 34% more energy, 56% more time, and 14% more memory than C, it’s time to reexamine your assumptions. Approximately every C program is a valid C++ program, so C++ can’t lose, especially not that badly! Benchmarking software is so easy to do wrong. It's so easy to manipulate the results if you are so inclined, and even when you're trying to be evenhanded it's so easy to disadvantage one of the systems under test. Different approaches are called for with different tools, and the classic error we see over and over again in benchmarking studies is the inappropriate application of idioms which suit one of the competitors across the others.
- gpderetta 5y ago
  Never trust a benchmark you didn't falsify yourself. Benchmarks can be useful as long as you can actually understand the reasons for the performance differences to be able to evaluate the pro and cons of the benchmarked systems.
  - PeterCorless 5y ago
    > "Never trust a benchmark you didn't falsify yourself." Stealing this.
    - gpderetta 5y ago
      For what is worth, I stole it myself ;)
      - PeterCorless 5y ago
        I am wondering if James Kanze is the source: As my friend James Kanze put it, "never trust a benchmark you didn't falsify yourself." https://stackoverflow.com/questions/5326269/is-c-sharp-really-slower-than-say-c https://stackoverflow.com/questions/5326269/is-c-sharp-reall...
        gpderetta 5y ago
tialaramex 5y ago
Russ argues that since you can compile lots of valid C programs as C++, surely C++ "can't lose" but this is a revealing error. It's true that lots of valid C programs are also valid C++ programs but it is not true that those programs will mean the same thing when treated as C++, nor does it mean that all C++ compilers are automatically just as good at optimising all possible valid programs as C compilers, and so the code which you get out and then measure may be quite different.
- laurent123456 5y ago
  I think he means that if you keep optimising a C++ program, or certain part of it, you eventually get to something very close to C, thus with similar energy and memory consumption. But indeed that's the problem with that kind of comparison - what's being compared exactly? How optimised is each program?
  - loeg 5y ago
    No, I think RSC means that C89 programs are literally valid C++ programs, and more recent C dialects nearly so.
    - int_19h 5y ago
      But C89 programs aren't literally valid C++ programs, either. Although I can't see any reason why those differences would affect performance. With newer C versions, there is "restrict", which can lead to better optimizations. And VLAs to avoid some heap allocations (albeit at the cost of potential stack overflow).
- qsort 5y ago
  It is also not necessarily true that a C compiler will be better than a C++ compiler at optimizing the same code. Even more so considering that both gcc and clang are basically C++ compilers regardless. In most cases, giving the compiler more information to work with makes the resulting code faster. C++ is generally slightly faster than equivalent C, even though the difference is usually very small. I have to agree with the OP, C being 34% faster than C++ is simply an absurd result.
kgeist 5y ago
The versioning scheme used in Go may be somewhat misleading to outsiders. There are often significant runtime changes between versions, but only the minor version is bumped. For example, adding generics to C# (among other things) prompted the C# team to bump the version of the language from 1 to 2. Go adds generics in 1.18, and if you don't follow the news, you won't know it's not just a minor improvement over 1.17, but one can say a whole paradigm shift (we've been waiting for generics for 13 years). So perhaps many people think there's nothing wrong about using old benchmarks for Go 1.10 because it doesn't feel like a lot must have changed between 1.10 and 1.18
- pjmlp 5y ago
  That was one of the reasons why by Java 9, the dual versioning scheme, e.g. 8 vs 1.8, was abandoned.
  - cpeterso 5y ago
    Sun did the same with Solaris’s versioning, jumping from Solaris 2.6 to 7 (i.e. 2.7).
  - tptacek 5y ago
    Go preserves major version bumps for things that can break compatibility. It's an engineering indicator, not a marketing tool.
    - Dylan16807 5y ago
      Marking very major new features is also an engineering indicator.
      - tptacek 5y ago
        Go preserves major version bumps for things that can break compatibility.
        5y ago
fulafel 5y ago
To what extent are the resulting co2 emissions of a program linked to its runtime resource use? On a succesful SV "web scale" app they are no doubt notable, but for 99.999% of programs the costs of getting it written surely dominate. The CO2 footprint then comes from the resulting consumer spending and other downstream economic activity, quantifiable by carbon intensity of the economy ($/kg/CO2e). [1] As a thought exercise, think what happens when a company engages a consultancy to make a Postgres backed Java or Python app running in AWS to replace a business process run previously in a overgrown Excel sheet and brain of a soon to be retired domain expert. Programmers are hired, (second) cars are bought, overseas vacations are had, what happens to co2 emissions? The choice between Java or Rust makes zero difference. The choice between Java and Excel might, but not because of runtime performance... TLDR; almost always it's the development budget, not CPU use. [1] You can see this number by country at https://ourworldindata.org/grapher/co2-intensity https://ourworldindata.org/grapher/co2-intensity - about 0.1 - 0.6 kg/$ in industrualized places
- jjice 5y ago
  The original post felt like when companies like Coca-cola tell you to recycle and act like the burden is with the individual instead of the massive corporation. Since this data was either posted (or at least referenced, can't remember) by AWS, it seems fitting. Amazon couldn't possibly have the power to deal with these rising emissions, it's the code that our customers run being written in Python that does it! I'd like to see tech industry emissions as a whole, just for running software. I have to imagine it's pretty minuscule in the grand scheme, but still interesting. What does a product like Instagram that has a huge user base and an application written (mostly?) in Python create in emissions and energy use? I guess we'd also have to consider that the societal benefit of Instagram isn't as high as trucks that deliver food to grocery stores, so truck emissions may be seen as more necessary emissions. It's a really deep rabbit hole. I find the concepts interested, though don't put too much thought into these issues I don't directly control.
  - wmf 5y ago
    AWS has huge incentives to optimize their infrastructure as much as possible. One that's done, all AWS resource consumption is on behalf of their customers.
new_stranger 5y ago
I'm really glad to have Rust and Go. I was worried the only way out of startups using scripting languages was Java or C++. Thanks to everyone that has contributed to these great languages! It's so nice to be able to simple things that are such an after-thought in other languages.
- brabel 5y ago
  Exactly! We are spoiled for choice. And Go and Rust are kind of appealing to totally opposite audiences who want the same thing: a fast binary. Rust is a complex language with a lot of features, making it very difficult to master. Go is at the very other end of the spectrum: very simple language, very few features, really easy to learn and write. Some people like one, some people like the other: it's great that we have both, so both kinds of people can write fast software!! Sure, Rust is faster, I don't think anyone will argue against that, but Go is still plenty fast and lightweight. So I am really glad my only choice are not only C++ and Rust, and that Go exists (as well as OCaml, Java, Kotlin, Dart, Haskell, D, Zig, they all have something going for them and people who get attracted to their ethos which just expands the number of people who can produce something using a language they can relate to!).
deleted 5y ago
[deleted]
infogulch 5y ago
Benchmark sites really need categories like speedruns. The finish line is something like "takes a dynamic input and gets the right answer". Then you have categories that do it under some additional constraints: any% (most languages call out to pre-written machine code, probably) language% (anything possible using just that language) idiomatic%, casual%, singlethreaded%, memory%, compile-run%?, safe%? etc. Then at least we'd be able to argue about which categories a particular benchmark result would apply to, and separately argue about which categories matter in a particular domain.
- 015a 5y ago
  Something like leetcode is the perfect platform for this kind of data; crowdsourced submissions for each language, with automated I/O systems to verify accuracy. In fact, when you submit a solution to a given problem, it tells you your completion time and memory usage percentile relative to other submissions in that language. It'd be very cool if they published statistics for each problem, for each language, but I'm not seeing that data right now.
  - davidkunz 5y ago
    Indeed, fast code in an idiomatic style doesn't mean it wasn't optimised for that benchmark. The only code I trust is code written by people having no idea about the benchmark.
- tialaramex 5y ago
  Each speedrun community decides their own rules. The people in the community are best placed to decide what it is exactly they're comparing and why. So there's no comparison from one community to another. But such comparisons are exactly what we'd be talking about for programming languages. Asking Rust programmers to set the rules for what counts as "idiomatic" Perl makes no sense, just like asking Cuphead players what counts as a "major" glitch in Java Minecraft or Portal 2. But if there is no agreement you haven't really improved anything on the measuring front.
  - infogulch 5y ago
ctvo 5y ago
A link back to the post so folks who haven't read it thoroughly can follow along: https://aws.amazon.com/blogs/opensource/sustainability-with-rust/ https://aws.amazon.com/blogs/opensource/sustainability-with-...
speedgoose 5y ago
The numbers are also very suspicious with JavaScript and TypeScript by the way.
- igouy 5y ago
  No one transliterated those JavaScript programs into type annotated TypeScript that passes type check. Different TypeScript programs were contributed.
- igouy 5y ago
  The data tables published with that 2017 paper, show a *15x* difference between the measured times of the selected JavaScript and TypeScript fannkuch-redux programs. https://sites.google.com/view/energy-efficiency-languages/results https://sites.google.com/view/energy-efficiency-languages/re... That outlier should explain the Typescript and Javascript difference in Table 4. Today with `node-v17.6.0` those same programs don't show that magnitude of difference: 43.268 seconds Node JS 77.029 seconds tsc --strict --noEmitOnError
laerus 5y ago
The Golang fanboyism is getting out of hand. Rust may not be the flawless language zealots tend to believe but at least it has something new to offer that was missing from the PL ecosystem and many other features that the industry didn't take seriously till lately cause they were "only for research languages". I don't even get why Golang is considered a good language. The new goto primitive for concurrency, the half baked error handling, the fact that it's designers think that programmers are not smart enough and made a language that looks like Java and feels like Python (but hey, no class inheritance and the compile times are faaast) are just a few of the examples that don't make it good enough for me. I wonder how popular Golang would have been if it wasn't backed up by it's company. What is this tendency for the software industry to try and make everything "easy"? Trying to oversimplify something that is inherently complex only adds accidental complexity.
- SkyMarshal 5y ago
  I agree, though will point out that one other factor in go’s favor besides fast compiles is relatively easy cross-compiling to other architectures. At least back when it came out, that was a noticeable feature to folks evaluating it. But yeah it seems to be more of a competitor vs Python and Ruby than vs Rust.
  - zozbot234 5y ago
    To be fair, the combination of cargo and llvm makes cross-compiling just as easy in Rust. But yes it's a great improvement over clunky old C/C++ toolchains. Go is not a bad language after all, in fact it has plenty of things going for it.
    - thinkharderdev 5y ago
      I think a big factor in Go adoption was exactly this. If you wanted a language that could be used for system-y programming (i.e. something that AOT compiles to a native binary) then Go was the only game in town (before Rust came onto the scene) aside from C/C++. And the tooling for C/C++ development is pretty atrocious. I hadn't done any C++ dev in a long time until relatively recently and was pretty shocked to see how bad it was. It's amazing that there is still no reliable way to manage dependencies and the various C++ package managers out there (vcpkg, conan, etc) are still extremely finicky.
lowbloodsugar 5y ago
Really comes down to "Do you want GC spikes or not?" and the tail latencies that go with it (clearly visible in the images in the tweet thread). The alternative, before Rust, was to use C or C++, and I'd never consider writing a Java or Go service in C++. Rust doesn't just bring safety to C, it brings an ecosystem that brings a "system language" up to the point where I'd consider it for general services. Edit: deleted java vs go comment, and replaced with the evidence at hand
- Thaxll 5y ago
  Java is definitely not faster than Go.
  - ithrow 5y ago
    The benchmark in the tweet show it is.
  - azth 5y ago
    Real world programs say otherwise.
sblom 5y ago
I struggled for years with a chronic misunderstanding of Go. I kept trying to compare it to the usual suspects in the "next systems language" crowd (D, Rust, Zig). That wasn't quite right. Once I realized that its sweet spot is large-ish services that you want closer to the metal than Java but for which would be silly to go as low level as C++, Go fans started making a whole lot more sense to me. I'm still not a Go fan personally—my taste in programming languages slants toward the more expressive end of the scale (C#, Rust, Scala). But at least I'm no longer confused by trying to make Go make sense where it doesn't. It's a very good tool for its kinds of jobs. Other languages also have their points of unreasonable effectiveness, too.
- azth 5y ago
  Define "closer to the metal than Java". What exactly does that mean, and how exactly is that an advantage?
  - PeterCorless 5y ago
    "Go performance is excellent since Go compiles to bare metal and there is no runtime between the metal and the language as is the case in other languages I used in this exercise (Java & Clojure: compile to JVM bytecode, and JVM runs the bytecode, Python & Javascript: interpreted languages and the runtime runs the code (though also Python compiles to bytecode)." Source: https://www.karimarttila.fi/languages/2018/11/13/go-good-productivity-with-bare-metal.html https://www.karimarttila.fi/languages/2018/11/13/go-good-pro...
    - azth 5y ago
      That statement is misleading. You can also compile Java natively. Secondly, there is most definitely a runtime "between the metal and the language" in golang, the GC being the most obvious. This proves my thoughts that these claims are not valid.
      - nemothekid 5y ago
        >You can also compile Java natively. This is also misleading. You are implying that GraalVM isn't relatively new and that Java native compilation isn't rare. I've yet to see anyone deploy service or CLI tools that are natively compiled in Java. That statement is about as true as saying you can compile Python natively. Just because its possible doesn't mean the language is actually suited for it.
PeterCorless 5y ago
Has anyone done a separate study to corroborate, contradict, or otherwise update the work of this 2017 paper? https://greenlab.di.uminho.pt/wp-content/uploads/2017/10/sleFinal.pdf https://greenlab.di.uminho.pt/wp-content/uploads/2017/10/sle... Also, a suggestion for anyone who plans to do an update: please put a version number on the languages you use, and also note which specific versions of compilers, VMs, GCs, etc. were used.
- Tobu 5y ago
  Indeed! This followup used sample code from the Rosetta Code wiki: https://haslab.github.io/SAFER/scp21.pdf https://haslab.github.io/SAFER/scp21.pdf The rankings move a little bit, but the relative positions of Go and Rust don't.
- igouy 5y ago
  On page 258, the 2017 paper tells you where to find that information: The measuring framework and the complete set of results are publicly available at https://sites.google.com/view/energy-efficiency-languages
scottlamb 5y ago
I agree with rsc about that study having problems, [1] but I'm not sure this part is accurate: > Second, the summary of the Discord post about switching from Go to Rust is incredibly misleading. The original Discord post showed a single graph plotting both a Go server and the equivalent Rust server. The Rust line had more predictable performance and avoided the latency spikes in the Go line, but the lines were roughly comparable. Instead, the AWS post displayed the Go graph next to a graph of the Rust server after a significant rewrite to use new data structures and more RAM, circling “ms” vs “µs” time scales. This is either a complete misunderstanding of the Discord post or blatant dishonesty. The discord post [2] says this: > After the service ran successfully for a few days, we decided it was time to re-raise the LRU cache capacity. In the Go version, as mentioned above, raising the cap of the LRU cache resulted in longer garbage collections. We no longer had to deal with garbage collection, so we figured we could raise the cap of the cache and get even better performance. We increased the memory capacity for the boxes, optimized the data structure to use even less memory (for fun), and increased the cache capacity to 8 million Read States. He paraphrased "optimized the data structures to use even less memory (for fun)" as "a significant rewrite to use new data structures". The post doesn't say how involved the optimization was, but I got the impression it might have been pretty trivial to shave a few bytes off the most common allocation or some such. And as for increasing the cache capacity, that's something Rust allowed them to do without problems but Go 1.10 didn't, so I think it's entirely fair to include it in the comparison. Maybe Go 1.18 would also have been able to do this, but unless Discord dusts off their old server to aid people choosing a language in 2022, we're not going to really know... [1] and even made a similar argument here: https://news.ycombinator.com/item?id=30313943 https://news.ycombinator.com/item?id=30313943 [2] https://discord.com/blog/why-discord-is-switching-from-go-to-rust https://discord.com/blog/why-discord-is-switching-from-go-to...
- Mathnerd314 5y ago
  The Discord and AWS posts are both about GC vs no GC. The GC in Go prevented scaling the cache due to latency spikes. Rust doesn't have a GC and so the cache could be grown from x million to 8 million without increasing latency. The Discord post says just simply rewriting the service from Go to Rust increased performance in all metrics (but doesn't give numbers). But Discord also made other changes during the rewrite - changing from hash table to B-tree, data structure layout optimizations, reducing memory copying, etc. And the graphs are with those changes included. So the Discord graphs aren't a pure Rust vs Go comparison. I guess I can agree with rsc that it's "misleading" to cite the graphs during a Rust vs. Go comparison. But the headings of the Discord post imply the latency change for average response time from milliseconds to microseconds is purely due to increasing the cache size, which was only possible due to eliminating the GC spikes. So it is fair to include that as an issue in a Go 1.10 vs Rust comparison. rsc's opinion seems to be that Go 1.18 will fix the latency issue - I disagree, but the only way to tell is to redo the measurements.
jcranberry 5y ago
I really don't see how it's misleading. It shows their rust implementation after some tuning vs their go implementation, which I'm sure was tuned as well.
allisdust 5y ago
I don't know. This reads more like a case of sour grapes. The discord article clearly states that some of the optimisations (like data structure improvements and especially the larger cache bucket sizes) weren't possible at all due to the way Go's garbage collection works (scanning for the entire structure rather than few changed bits). May be that improved with the latest Go version? Even if it did, it's not possible to have the same predictable performance from any GC languages for that particular use case which seem to be suited for a non GC language. Fair points about c++ and the overall deficits in the original study that doesn't use comparable programs across languages.
- igouy 5y ago
  See https://news.ycombinator.com/item?id=30505343 https://news.ycombinator.com/item?id=30505343
pornel 5y ago
> Approximately every C program is a valid C++ program, so C++ can’t lose, especially not that badly! He doesn’t seem to be aware that The Benchmarks Game has a moderator who requires the contest entries to be “idiomatic” for each language (details of what counts as idiomatic are of course contentious), so a C program that doesn’t use C++ features/idioms would not be accepted in the game.
- igouy 5y ago
  The project CONTRIBUTING.md says — You will probably come across people saying that the programs are not idiomatic ("enough"). So read the description and write your own idiomatic program, without programming tricks. That's not the same as "requiring". More like — if you aren't going to contribute idiomatic programs then don't complain when we don't show idiomatic programs.
- igouy 5y ago
  fwiw "… nontrivial C programs will not compile as C++ code without modification." https://en.wikipedia.org/wiki/Compatibility_of_C_and_C++ https://en.wikipedia.org/wiki/Compatibility_of_C_and_C++
- igouy 5y ago
  Meanwhile we can do better than "every C program is a valid C++ program, so C++ can’t lose" defensiveness. Look at the data tables published with that 2017 paper: https://sites.google.com/view/energy-efficiency-languages/re https://sites.google.com/view/energy-efficiency-languages/re... There's an order of magnitude difference between the times of the selected C and C++ programs, for one thing — regex-redux. That is enough to explain all of the difference between the C and C++ time measurements shown in "Table 4. Normalized global results". (Seems like an outlier which could have been excluded.)

190 points by 0xedb 5y ago

cyb_ 5y ago

Alternative way to read the Twitter thread: https://nitter.net/_rsc/status/1496352325157457922 https://nitter.net/_rsc/status/1496352325157457922

kevincox 5y ago
That is a lot cleaner than the original. I still prefer https://threadreaderapp.com/thread/1496352325157457922.html https://threadreaderapp.com/thread/1496352325157457922.html though which removes almost all of the chrome between sentences.

belter 5y ago

jonathanstrange 5y ago

This doesn't surprise me. Rust's toxic community was one of several reasons that made me turn away from the language. When people ask on a Go forum (e.g. r/golang) whether they should switch to or write a project in Go, they tend to get good and professional advice, and often this advice is to stick with the existing language and approach. I don't know why this is so, but with Rust you mostly get zealots with zero ability to evaluate the Pros and Cons of their language of choice.

Xorlev 5y ago
Painting an entire community with the same brush seems to itself be fairly toxic behavior berift of signal. Let's keep focused on AWS's poor greenwashing post. You could replace all the names of the technologies and it's still just as bad.
- geodel 5y ago
  > Painting an entire community with the same brush seems to itself be fairly toxic behavior berift of signal. Communities are painted toxic all the time rightly or wrongly. To say community have feeling and are to treated as person does not seem to be appropriate. Also calling people toxic who called some community toxic is akin to name calling and I dare say much more common in Rust circles.
mcronce 5y ago
That's not really consistent with my experience in the Rust community at all. Memes aside, rewrites for the sake of rewrites are typically discouraged. When language comparisons are made, they tend to clearly highlight objective and subjective pros/cons. In particular, the referenced AWS article is typically called out as a bad "study" Maybe it was different a few years ago, but this is what I've seen since late 2019ish.
- mohanmcgeek 5y ago
  Rewrites for the sake of rewrites is a HN thing. This used to happen with other languages too. In 2015, Front page posts on a given day would have "xyz in Go" and it was JavaScript before that. People tend to reimplement systems they know in languages they are learning and it's perfectly alright. But, these are engineers at huge companies rewriting core system components for dubious reasons AND writing a deliberately misleading blog post about it. Now more engineers with an agenda would cite these posts to their managers and get their rewrites.
  - lanstin 5y ago
    Rewriting a small thing you know well is a fabulous way to understand the new language, because your focus isn't on "what should the code do" but "how do I make the code do this thing." I've had senior tech leadership explain that rewriting a big system into a new language is done because the real goal is to rewrite the old system that no one understands well enough and is gradually becoming scary to change; people are enthused for the shiny new language, and at the end the current organization understands this crucial system for a few more years. People in companies write the blog posts because it helps get promotions ("visibility" "industry leadership").
    - mohanmcgeek 5y ago
      > People in companies write the blog posts because it helps get promotions ("visibility" "industry leadership"). This is definitely toxic work culture that needs to be fixed
  - 5y ago
pohl 5y ago
I think Rust has grown way beyond the point where it makes sense to characterize the community as a whole. I imagine you could get entirely different experiences depending upon whether you were engaging with people in some HN thread, or Reddit, or IRC, or Discord,...and even day by day or thread by thread. Where specifically did you encounter the toxicity?
the__alchemist 5y ago
You can use the language and ignore the parts of the community you don't like. I've made similar observations, but still conclude the language is top-notch. It also seems like a large portion of the posts I see in embedded Rust are about generics and/or async. I have no interest in them. (Most of the embedded code I've seen using them can be written more plainly without). It also turns new people off the language due to the syntax clutter. So, they think "Rust is hard to write/messy". It's not the language - it's how parts of the community use it. I also dislike the condescending attitude when C code comes up .
- mustache_kimono 5y ago
  I'll say as someone who leans more towards Rust, and its community, in this somewhat manufactured debate -- I think it's pretty hard not to be a little prickly when most of the criticism amounts to "I hate the hype!" or "I don't like change!" My opinion has always been: Be Confident Enough to Allow Others to Disagree. Some in the Rust community sometimes have a problem with this, but, my God, it's not as if the arguments against Rust are very good, or as if people don't love to troll Rust, like the parent to your comment. Most criticisms are embarrassingly flimsy.
  - Yoric 5y ago
    (to complete what mustache_kimono wrote) Which is not to say that Rust doesn't have a number of downsides. And I'm writing this as a Rust enthusiast and (very minor) contributor to the Rust compiler.
  - qsdf38100 5y ago
    I hate the hype. I like honesty, pros vs cons. Rust community seems full of zealots and evangelists. Those people are very vocal about pros, and condescending. As soon as someone brings any con or isn’t convinced, they are just bad programmers, with bad arguments, and even « embarrassingly flimsy » criticism…
    - mustache_kimono 5y ago
dataangel 5y ago
If you read the full context, the misunderstanding comes from research not done by the Rust community. You just found a way to reinforce your own bias.
0xFEE1DEAD 5y ago
Every interaction I’ve had with the rust community was toxic in nature. Also for whatever reason all those „rust vs go“ articles are always coming from the rust corner. It’s like the rust community is frustrated about go success and they individually feel the need to attack their perceived competitor meanwhile the go community has no issue with any other language.
- pyjarrett 5y ago
  [deleted]
  - kitkat_new 5y ago
    I don't get the logic of dropping a language you find great and you are productive in because of a few people
    - busterarm 5y ago
      You'll eventually have to cooperate with those people, whether it's on OSS or in the workplace. It's even worse when really senior figures in a community are also the source of the toxicity.
      - deleted 5y ago
        [deleted]
    - sirwhinesalot 5y ago
      Those people will (potentially) be your future co-workers... So it is a valid concern. That said, I have not interacted with the Rust community and cannot make any comments in that regard. Language is pretty nice though.
      - deleted 5y ago
        [deleted]
    - pyjarrett 5y ago
tashoecraft 5y ago
I think Rust has attracted a lot of people who don't really program in it often, but try to advocate for it. My experiences with the community have always been good, and they are consistently rated as having a really great community, but a growing community will always have issues.
- ziml77 5y ago
  I’m fairly certain this is the case. The people who seem to want everything rewritten in Rust are not necessarily also people who actually use the language. They likely just heard that it improves security and stability to use Rust over C. Would be similar to the people who bring up Ada as an alternative; the chances that they’ve actually tried it are minuscule. And to be very clear I’m not trying to say there is no overlap at all, but I don’t think you can call the Rust community toxic because of those people. (I wonder if the people who treat Rust like a religion realize how harmful their behavior is. I want Rust to be seen as a reasonable option, but that’s difficult when you have the evangelists scaring people away)
kitkat_new 5y ago
In my experience, the Rust community isn't toxic at all. To the contrary: when I started to explore Rust, it was very welcoming, helped finding the answer to all my nooby questions, which they probably heard a dozen times - despite having a lot of C++ and JS baggage
- MisterTea 5y ago
  You have a veery one sided perspective. Someone adopting/learning a language will be met with plenty of positivity and encouragement. The toxicity comes in to play once you post something critical of the language, even if it has merit.
  - Yoric 5y ago
    So you're saying that people leaving comments on (I assume) blogs are typically less civil than people who take the time to work with newcomers and actually make the community work? That is certainly true, but for pretty much every community that I know, not limited to any programming language or technology.
- tayo42 5y ago
  Did you see what happened to the actix project. That was embarrassing
  - Yoric 5y ago
    What happened to the actix project? Asking, because I use it daily :)
    - tayo42 5y ago
      It used to use a bunch of unsafe, someone decided that wasn't acceptable and it blew up into a whole thing where people started dog piling on the project and the maintainer. everyone got emotional, he eventually quit and gave up actix to someone else i think. pretty unnecessary and childish the way people acted towards him imo. a lot happened on reddit and github so it should be easy to find.
odiroot 5y ago
I don't know which community is toxic and wouldn't even risk using such strong term. Still, I have to say, it's really weird to see (last few months) everything being rewritten into Rust, just for the sake of pride(?). Haven't we learned anything from the NodeJS craze?
- josephcsible 5y ago
  > just for the sake of pride(?). It's to prevent all memory safety bugs.
  - kangalioo 5y ago
    Partly true, but this doesn't apply to many rewritten tools, like cat or ls where memory safety bugs are tolerable
    - ansible 5y ago
      Well, I don't know about the cat command (though that _is_ appropriate for something like the Redox OS, which aspires to be a pure-Rust desktop operating system). But there is a broad-based desire to rewrite libraries in Rust for a couple (IMHO valid) reasons. One is the API. While it is possible to have a nice and idiomatic Rust API to a C library, it will often be nicer to use when it is designed from the ground-up in Rust. Traits, interfaces, etc. It is also often desirable to avoid C when cross-compiling (such as for embedded systems), as that is generally a hassle to set up and get working seamlessly.
    - detaro 5y ago
      Doesn't sound like things where you'd want bugs either. But also the rewrites for this kind of thing are usually not that "serious". A few people do it because they can or because they think it's interesting, but it's not like there are large pushes to replace such tools (E.g. there's been "lets rewrite basic tools in Go" projects too, and nobody is surprised that they aren't in widespread use"). The winners usually are not in that category, but tools that stand alone.
    - 5y ago
lnxg33k1 5y ago
To some extent I agree with you, I don't like the rust community generically, as I feel the toxicity too, but I mean the language and the offering of a fast but easy and checked platform to be are still good enough to let me just ignore them, I mean on IRC you always find people who are going to be bad, and people who will just help, and who cares about those who are bad, I guess just feel a few seconds of pity for them and move on
- Yoric 5y ago
  Out of curiosity, where did you see any toxicity? I'm both on the Rust forums and on the Rust Matrix community and I haven't seen anything toxic at all. Everything I've seen that looked like it could become toxic was handled gently by someone with a bit of seniority.
burnished 5y ago
Its weird, because the negative stuff I see around the Rust community is posts like this with name calling or histrionics, and the positive stuff is like 'thoughtful post mortem about why Rust compile times are so bad (but are improving)'.
Yoric 5y ago
Really? I've never found a community as nice as the Rust one. Where did you ask?

jerf 5y ago

When I first read that, I thought the core conclusions were still mostly correct, but the delta claimed for Rust over Go was too large. Rust will certainly beat Go on this metric (I expect Rust to even beat C and C++ at scale [1]), but by a factor of 3 is probably on the large side. There are other things that I would consider questionable as well; Haskell might be capable of beating Go if you really, really work at it, but what you might call "casually written Haskell" isn't going to perform anywhere near as well as casually written Go. And the degree of knowledge required to write high-performance Haskell is much higher than high-performance Go. You have to have more than just knowledge of Haskell as a language, you have to have a fairly deep understanding of exactly how GHC compiles and optimizes it. This can be scaled to the Shootout, but scaling it to real code is very difficult. Still, I think the general point is well-taken. I'm less worried about the environmental impact of my code because my code runs at scales where even if my code was running for "free" from an environmental impact, it wouldn't save much of anything. But having used Go for a while, I'd have a really difficult time moving back to a language that is literally dozens of times slower, and, yes, consumes dozens of times more energy as a result. Part of the reason why my code runs at such small scales is that it's reasonably efficient code. I have a couple of services that I consider trivial, but if I were writing them in Python, would be non-trivial. Still not exactly monsters that would be eating clusters of machines at a time, but there's still a lot of practical operational difference between "This occasionally spikes to 25% of a CPU for a minute at high loads" and "This occasionally spikes to consuming 10 full CPUs for a minute at high load", and that's a fairly reasonable multiplier. I have several services that would have deltas like that. [1]: A previous comment of mine: https://news.ycombinator.com/item?id=25625362 https://news.ycombinator.com/item?id=25625362

newaccount2021 5y ago
melony 5y ago
Go is at about the same abstraction level as Java, maybe a fraction slower due to lack of runtime VM optimisations.
- 5y ago

rectang 5y ago

> Most obviously, if your study claims that C++ uses 34% more energy, 56% more time, and 14% more memory than C, it’s time to reexamine your assumptions. Approximately every C program is a valid C++ program, so C++ can’t lose, especially not that badly! Benchmarking software is so easy to do wrong. It's so easy to manipulate the results if you are so inclined, and even when you're trying to be evenhanded it's so easy to disadvantage one of the systems under test. Different approaches are called for with different tools, and the classic error we see over and over again in benchmarking studies is the inappropriate application of idioms which suit one of the competitors across the others.

gpderetta 5y ago
Never trust a benchmark you didn't falsify yourself. Benchmarks can be useful as long as you can actually understand the reasons for the performance differences to be able to evaluate the pro and cons of the benchmarked systems.
- PeterCorless 5y ago
  > "Never trust a benchmark you didn't falsify yourself." Stealing this.
  - gpderetta 5y ago
    For what is worth, I stole it myself ;)
    - PeterCorless 5y ago
      I am wondering if James Kanze is the source: As my friend James Kanze put it, "never trust a benchmark you didn't falsify yourself." https://stackoverflow.com/questions/5326269/is-c-sharp-really-slower-than-say-c https://stackoverflow.com/questions/5326269/is-c-sharp-reall...
      - gpderetta 5y ago

tialaramex 5y ago

Russ argues that since you can compile lots of valid C programs as C++, surely C++ "can't lose" but this is a revealing error. It's true that lots of valid C programs are also valid C++ programs but it is not true that those programs will mean the same thing when treated as C++, nor does it mean that all C++ compilers are automatically just as good at optimising all possible valid programs as C compilers, and so the code which you get out and then measure may be quite different.

laurent123456 5y ago
I think he means that if you keep optimising a C++ program, or certain part of it, you eventually get to something very close to C, thus with similar energy and memory consumption. But indeed that's the problem with that kind of comparison - what's being compared exactly? How optimised is each program?
- loeg 5y ago
  No, I think RSC means that C89 programs are literally valid C++ programs, and more recent C dialects nearly so.
  - int_19h 5y ago
    But C89 programs aren't literally valid C++ programs, either. Although I can't see any reason why those differences would affect performance. With newer C versions, there is "restrict", which can lead to better optimizations. And VLAs to avoid some heap allocations (albeit at the cost of potential stack overflow).
qsort 5y ago
It is also not necessarily true that a C compiler will be better than a C++ compiler at optimizing the same code. Even more so considering that both gcc and clang are basically C++ compilers regardless. In most cases, giving the compiler more information to work with makes the resulting code faster. C++ is generally slightly faster than equivalent C, even though the difference is usually very small. I have to agree with the OP, C being 34% faster than C++ is simply an absurd result.

kgeist 5y ago

The versioning scheme used in Go may be somewhat misleading to outsiders. There are often significant runtime changes between versions, but only the minor version is bumped. For example, adding generics to C# (among other things) prompted the C# team to bump the version of the language from 1 to 2. Go adds generics in 1.18, and if you don't follow the news, you won't know it's not just a minor improvement over 1.17, but one can say a whole paradigm shift (we've been waiting for generics for 13 years). So perhaps many people think there's nothing wrong about using old benchmarks for Go 1.10 because it doesn't feel like a lot must have changed between 1.10 and 1.18

pjmlp 5y ago
That was one of the reasons why by Java 9, the dual versioning scheme, e.g. 8 vs 1.8, was abandoned.
- cpeterso 5y ago
  Sun did the same with Solaris’s versioning, jumping from Solaris 2.6 to 7 (i.e. 2.7).
- tptacek 5y ago
  Go preserves major version bumps for things that can break compatibility. It's an engineering indicator, not a marketing tool.
  - Dylan16807 5y ago
    Marking very major new features is also an engineering indicator.
    - tptacek 5y ago
      Go preserves major version bumps for things that can break compatibility.
      - 5y ago

fulafel 5y ago

To what extent are the resulting co2 emissions of a program linked to its runtime resource use? On a succesful SV "web scale" app they are no doubt notable, but for 99.999% of programs the costs of getting it written surely dominate. The CO2 footprint then comes from the resulting consumer spending and other downstream economic activity, quantifiable by carbon intensity of the economy ($/kg/CO2e). [1] As a thought exercise, think what happens when a company engages a consultancy to make a Postgres backed Java or Python app running in AWS to replace a business process run previously in a overgrown Excel sheet and brain of a soon to be retired domain expert. Programmers are hired, (second) cars are bought, overseas vacations are had, what happens to co2 emissions? The choice between Java or Rust makes zero difference. The choice between Java and Excel might, but not because of runtime performance... TLDR; almost always it's the development budget, not CPU use. [1] You can see this number by country at https://ourworldindata.org/grapher/co2-intensity https://ourworldindata.org/grapher/co2-intensity - about 0.1 - 0.6 kg/$ in industrualized places

jjice 5y ago
The original post felt like when companies like Coca-cola tell you to recycle and act like the burden is with the individual instead of the massive corporation. Since this data was either posted (or at least referenced, can't remember) by AWS, it seems fitting. Amazon couldn't possibly have the power to deal with these rising emissions, it's the code that our customers run being written in Python that does it! I'd like to see tech industry emissions as a whole, just for running software. I have to imagine it's pretty minuscule in the grand scheme, but still interesting. What does a product like Instagram that has a huge user base and an application written (mostly?) in Python create in emissions and energy use? I guess we'd also have to consider that the societal benefit of Instagram isn't as high as trucks that deliver food to grocery stores, so truck emissions may be seen as more necessary emissions. It's a really deep rabbit hole. I find the concepts interested, though don't put too much thought into these issues I don't directly control.
- wmf 5y ago
  AWS has huge incentives to optimize their infrastructure as much as possible. One that's done, all AWS resource consumption is on behalf of their customers.

new_stranger 5y ago

I'm really glad to have Rust and Go. I was worried the only way out of startups using scripting languages was Java or C++. Thanks to everyone that has contributed to these great languages! It's so nice to be able to simple things that are such an after-thought in other languages.

brabel 5y ago
Exactly! We are spoiled for choice. And Go and Rust are kind of appealing to totally opposite audiences who want the same thing: a fast binary. Rust is a complex language with a lot of features, making it very difficult to master. Go is at the very other end of the spectrum: very simple language, very few features, really easy to learn and write. Some people like one, some people like the other: it's great that we have both, so both kinds of people can write fast software!! Sure, Rust is faster, I don't think anyone will argue against that, but Go is still plenty fast and lightweight. So I am really glad my only choice are not only C++ and Rust, and that Go exists (as well as OCaml, Java, Kotlin, Dart, Haskell, D, Zig, they all have something going for them and people who get attracted to their ethos which just expands the number of people who can produce something using a language they can relate to!).

deleted 5y ago

[deleted]

infogulch 5y ago

Benchmark sites really need categories like speedruns. The finish line is something like "takes a dynamic input and gets the right answer". Then you have categories that do it under some additional constraints: any% (most languages call out to pre-written machine code, probably) language% (anything possible using just that language) idiomatic%, casual%, singlethreaded%, memory%, compile-run%?, safe%? etc. Then at least we'd be able to argue about which categories a particular benchmark result would apply to, and separately argue about which categories matter in a particular domain.

015a 5y ago
Something like leetcode is the perfect platform for this kind of data; crowdsourced submissions for each language, with automated I/O systems to verify accuracy. In fact, when you submit a solution to a given problem, it tells you your completion time and memory usage percentile relative to other submissions in that language. It'd be very cool if they published statistics for each problem, for each language, but I'm not seeing that data right now.
- davidkunz 5y ago
  Indeed, fast code in an idiomatic style doesn't mean it wasn't optimised for that benchmark. The only code I trust is code written by people having no idea about the benchmark.
tialaramex 5y ago
Each speedrun community decides their own rules. The people in the community are best placed to decide what it is exactly they're comparing and why. So there's no comparison from one community to another. But such comparisons are exactly what we'd be talking about for programming languages. Asking Rust programmers to set the rules for what counts as "idiomatic" Perl makes no sense, just like asking Cuphead players what counts as a "major" glitch in Java Minecraft or Portal 2. But if there is no agreement you haven't really improved anything on the measuring front.
- infogulch 5y ago

ctvo 5y ago

A link back to the post so folks who haven't read it thoroughly can follow along: https://aws.amazon.com/blogs/opensource/sustainability-with-rust/ https://aws.amazon.com/blogs/opensource/sustainability-with-...

speedgoose 5y ago

The numbers are also very suspicious with JavaScript and TypeScript by the way.

igouy 5y ago
No one transliterated those JavaScript programs into type annotated TypeScript that passes type check. Different TypeScript programs were contributed.
igouy 5y ago
The data tables published with that 2017 paper, show a *15x* difference between the measured times of the selected JavaScript and TypeScript fannkuch-redux programs. https://sites.google.com/view/energy-efficiency-languages/results https://sites.google.com/view/energy-efficiency-languages/re... That outlier should explain the Typescript and Javascript difference in Table 4. Today with `node-v17.6.0` those same programs don't show that magnitude of difference: 43.268 seconds Node JS 77.029 seconds tsc --strict --noEmitOnError

laerus 5y ago

The Golang fanboyism is getting out of hand. Rust may not be the flawless language zealots tend to believe but at least it has something new to offer that was missing from the PL ecosystem and many other features that the industry didn't take seriously till lately cause they were "only for research languages". I don't even get why Golang is considered a good language. The new goto primitive for concurrency, the half baked error handling, the fact that it's designers think that programmers are not smart enough and made a language that looks like Java and feels like Python (but hey, no class inheritance and the compile times are faaast) are just a few of the examples that don't make it good enough for me. I wonder how popular Golang would have been if it wasn't backed up by it's company. What is this tendency for the software industry to try and make everything "easy"? Trying to oversimplify something that is inherently complex only adds accidental complexity.

SkyMarshal 5y ago
I agree, though will point out that one other factor in go’s favor besides fast compiles is relatively easy cross-compiling to other architectures. At least back when it came out, that was a noticeable feature to folks evaluating it. But yeah it seems to be more of a competitor vs Python and Ruby than vs Rust.
- zozbot234 5y ago
  To be fair, the combination of cargo and llvm makes cross-compiling just as easy in Rust. But yes it's a great improvement over clunky old C/C++ toolchains. Go is not a bad language after all, in fact it has plenty of things going for it.
  - thinkharderdev 5y ago
    I think a big factor in Go adoption was exactly this. If you wanted a language that could be used for system-y programming (i.e. something that AOT compiles to a native binary) then Go was the only game in town (before Rust came onto the scene) aside from C/C++. And the tooling for C/C++ development is pretty atrocious. I hadn't done any C++ dev in a long time until relatively recently and was pretty shocked to see how bad it was. It's amazing that there is still no reliable way to manage dependencies and the various C++ package managers out there (vcpkg, conan, etc) are still extremely finicky.

lowbloodsugar 5y ago

Really comes down to "Do you want GC spikes or not?" and the tail latencies that go with it (clearly visible in the images in the tweet thread). The alternative, before Rust, was to use C or C++, and I'd never consider writing a Java or Go service in C++. Rust doesn't just bring safety to C, it brings an ecosystem that brings a "system language" up to the point where I'd consider it for general services. Edit: deleted java vs go comment, and replaced with the evidence at hand

Thaxll 5y ago
Java is definitely not faster than Go.
- ithrow 5y ago
  The benchmark in the tweet show it is.
- azth 5y ago
  Real world programs say otherwise.

sblom 5y ago

I struggled for years with a chronic misunderstanding of Go. I kept trying to compare it to the usual suspects in the "next systems language" crowd (D, Rust, Zig). That wasn't quite right. Once I realized that its sweet spot is large-ish services that you want closer to the metal than Java but for which would be silly to go as low level as C++, Go fans started making a whole lot more sense to me. I'm still not a Go fan personally—my taste in programming languages slants toward the more expressive end of the scale (C#, Rust, Scala). But at least I'm no longer confused by trying to make Go make sense where it doesn't. It's a very good tool for its kinds of jobs. Other languages also have their points of unreasonable effectiveness, too.

azth 5y ago
Define "closer to the metal than Java". What exactly does that mean, and how exactly is that an advantage?
- PeterCorless 5y ago
  "Go performance is excellent since Go compiles to bare metal and there is no runtime between the metal and the language as is the case in other languages I used in this exercise (Java & Clojure: compile to JVM bytecode, and JVM runs the bytecode, Python & Javascript: interpreted languages and the runtime runs the code (though also Python compiles to bytecode)." Source: https://www.karimarttila.fi/languages/2018/11/13/go-good-productivity-with-bare-metal.html https://www.karimarttila.fi/languages/2018/11/13/go-good-pro...
  - azth 5y ago
    That statement is misleading. You can also compile Java natively. Secondly, there is most definitely a runtime "between the metal and the language" in golang, the GC being the most obvious. This proves my thoughts that these claims are not valid.
    - nemothekid 5y ago
      >You can also compile Java natively. This is also misleading. You are implying that GraalVM isn't relatively new and that Java native compilation isn't rare. I've yet to see anyone deploy service or CLI tools that are natively compiled in Java. That statement is about as true as saying you can compile Python natively. Just because its possible doesn't mean the language is actually suited for it.

PeterCorless 5y ago

Has anyone done a separate study to corroborate, contradict, or otherwise update the work of this 2017 paper? https://greenlab.di.uminho.pt/wp-content/uploads/2017/10/sleFinal.pdf https://greenlab.di.uminho.pt/wp-content/uploads/2017/10/sle... Also, a suggestion for anyone who plans to do an update: please put a version number on the languages you use, and also note which specific versions of compilers, VMs, GCs, etc. were used.

Tobu 5y ago
Indeed! This followup used sample code from the Rosetta Code wiki: https://haslab.github.io/SAFER/scp21.pdf https://haslab.github.io/SAFER/scp21.pdf The rankings move a little bit, but the relative positions of Go and Rust don't.
igouy 5y ago
On page 258, the 2017 paper tells you where to find that information: The measuring framework and the complete set of results are publicly available at https://sites.google.com/view/energy-efficiency-languages

scottlamb 5y ago

I agree with rsc about that study having problems, [1] but I'm not sure this part is accurate: > Second, the summary of the Discord post about switching from Go to Rust is incredibly misleading. The original Discord post showed a single graph plotting both a Go server and the equivalent Rust server. The Rust line had more predictable performance and avoided the latency spikes in the Go line, but the lines were roughly comparable. Instead, the AWS post displayed the Go graph next to a graph of the Rust server after a significant rewrite to use new data structures and more RAM, circling “ms” vs “µs” time scales. This is either a complete misunderstanding of the Discord post or blatant dishonesty. The discord post [2] says this: > After the service ran successfully for a few days, we decided it was time to re-raise the LRU cache capacity. In the Go version, as mentioned above, raising the cap of the LRU cache resulted in longer garbage collections. We no longer had to deal with garbage collection, so we figured we could raise the cap of the cache and get even better performance. We increased the memory capacity for the boxes, optimized the data structure to use even less memory (for fun), and increased the cache capacity to 8 million Read States. He paraphrased "optimized the data structures to use even less memory (for fun)" as "a significant rewrite to use new data structures". The post doesn't say how involved the optimization was, but I got the impression it might have been pretty trivial to shave a few bytes off the most common allocation or some such. And as for increasing the cache capacity, that's something Rust allowed them to do without problems but Go 1.10 didn't, so I think it's entirely fair to include it in the comparison. Maybe Go 1.18 would also have been able to do this, but unless Discord dusts off their old server to aid people choosing a language in 2022, we're not going to really know... [1] and even made a similar argument here: https://news.ycombinator.com/item?id=30313943 https://news.ycombinator.com/item?id=30313943 [2] https://discord.com/blog/why-discord-is-switching-from-go-to-rust https://discord.com/blog/why-discord-is-switching-from-go-to...

Mathnerd314 5y ago
The Discord and AWS posts are both about GC vs no GC. The GC in Go prevented scaling the cache due to latency spikes. Rust doesn't have a GC and so the cache could be grown from x million to 8 million without increasing latency. The Discord post says just simply rewriting the service from Go to Rust increased performance in all metrics (but doesn't give numbers). But Discord also made other changes during the rewrite - changing from hash table to B-tree, data structure layout optimizations, reducing memory copying, etc. And the graphs are with those changes included. So the Discord graphs aren't a pure Rust vs Go comparison. I guess I can agree with rsc that it's "misleading" to cite the graphs during a Rust vs. Go comparison. But the headings of the Discord post imply the latency change for average response time from milliseconds to microseconds is purely due to increasing the cache size, which was only possible due to eliminating the GC spikes. So it is fair to include that as an issue in a Go 1.10 vs Rust comparison. rsc's opinion seems to be that Go 1.18 will fix the latency issue - I disagree, but the only way to tell is to redo the measurements.

jcranberry 5y ago

I really don't see how it's misleading. It shows their rust implementation after some tuning vs their go implementation, which I'm sure was tuned as well.

allisdust 5y ago

I don't know. This reads more like a case of sour grapes. The discord article clearly states that some of the optimisations (like data structure improvements and especially the larger cache bucket sizes) weren't possible at all due to the way Go's garbage collection works (scanning for the entire structure rather than few changed bits). May be that improved with the latest Go version? Even if it did, it's not possible to have the same predictable performance from any GC languages for that particular use case which seem to be suited for a non GC language. Fair points about c++ and the overall deficits in the original study that doesn't use comparable programs across languages.

igouy 5y ago
See https://news.ycombinator.com/item?id=30505343 https://news.ycombinator.com/item?id=30505343

pornel 5y ago

> Approximately every C program is a valid C++ program, so C++ can’t lose, especially not that badly! He doesn’t seem to be aware that The Benchmarks Game has a moderator who requires the contest entries to be “idiomatic” for each language (details of what counts as idiomatic are of course contentious), so a C program that doesn’t use C++ features/idioms would not be accepted in the game.

igouy 5y ago
The project CONTRIBUTING.md says — You will probably come across people saying that the programs are not idiomatic ("enough"). So read the description and write your own idiomatic program, without programming tricks. That's not the same as "requiring". More like — if you aren't going to contribute idiomatic programs then don't complain when we don't show idiomatic programs.
igouy 5y ago
fwiw "… nontrivial C programs will not compile as C++ code without modification." https://en.wikipedia.org/wiki/Compatibility_of_C_and_C++ https://en.wikipedia.org/wiki/Compatibility_of_C_and_C++
igouy 5y ago
Meanwhile we can do better than "every C program is a valid C++ program, so C++ can’t lose" defensiveness. Look at the data tables published with that 2017 paper: https://sites.google.com/view/energy-efficiency-languages/re https://sites.google.com/view/energy-efficiency-languages/re... There's an order of magnitude difference between the times of the selected C and C++ programs, for one thing — regex-redux. That is enough to explain all of the difference between the C and C++ time measurements shown in "Table 4. Normalized global results". (Seems like an outlier which could have been excluded.)