5 ms·
Data Processing Benchmark Featuring Rust, Go, Swift, Zig, Julia etc.
- pron 9mo agoI was surprised to see that Java was slower than C++, but the Java code is run with `-XX:+UseSerialGC`, which is the slowest GC, meant to be used only on very small systems, and to optimise for memory footprint more than performance. Also, there's no heap size, which means it's hard to know what exactly is being measured. Java allows trading off CPU for RAM and vice-versa. It would be meaningful if an appropriate GC were used (Parallel, for this batch job) and with different heap sizes. If the rules say the program should take less than 8GB of RAM, then it's best to configure the heap to 8GB (or a little lower). Also, System.gc() shouldn't be invoked. Don't know if that would make a difference, but that's how I'd run it, because in Java, the heap/GC configuration is an important part of the program and how it's actually executed. Of course, the most recent JDK version should be used (I guess the most recent compiler version for all languages).
- rockwotj 9mo agoIt’s so hard to actually benchmark languages because it so much depends on the dataset, I am pretty sure with simdjson and some tricks I could write C++ (or Rust) that could top the leaderboard (see some of the techniques from the billion row challenge!). tbh for silly benchmarks like this it will ultimately be hard to beat a language that compiles to machine code, due to jit warmup etc. It’s hard to due benchmarks right, for example are you testing IO performance? are OS caches flushed between language runs? What kind of disk is used etc? Performance does not exist in a vacuum of just the language or algorithm.
- pron 9mo ago> due to jit warmup I think this harness actually uses JMH, which measures after warmup.
- clawsyndicate 9mo ago[dead]
- KerrAvon 9mo agoWhy are you surprised? Java always suffers from abstraction penalty for running on a VM. You should be surprised (and skeptical) if Java ever beats C++ on any benchmark.
- woooooo 9mo agoFor the most naive code, if you're calling "new" multiple times per row, maybe Java benefits from out of band GC while C++ calls destructors and free() inline as things go out of scope? Of course, if you're optimizing, you'll reuse buffers and objects in either language.
- igouy 9mo ago> maybe Java benefits from out of band GC benchmarks game uses BenchExec to take 'care of important low-level details for accurate, precise, and reproducible measurements' …. BenchExec uses the cgroups feature of the Linux kernel to correctly handle groups of processes and uses Linux user namespaces to create a container that restricts interference of [each program] with the benchmarking host.
- woooooo 9mo agoI'm talking about memory management in-process, I dont think cgroups would affect that?
- pron 9mo agoThe only "abstraction penalty" of "running on a VM" (by which I think you mean using a JIT compiler), is the warmup time of waiting for the JIT.
- andersmurphy 9mo agoIts a statement of our times that this is getting down voted. JIT is so underrated.
- stefs 9mo ago
- Imustaskforhelp 9mo agoThis is really interesting. Julia is a beast compared to python. Nowadays whenever I see benchmarks of different languages. I really compare it to benjdd.com/languages or benjdd.com/languages2 Ended up creating a visualization of this data if anybody's interested https://serjaimelannister.github.io/data-processing-benchmark/ https://serjaimelannister.github.io/data-processing-benchmar... (Given credits to both sources in the description of this repo) (Also fair disclosure but it was generated just out of curiosity of how this benchmark data might look if it was on benjdd's ui and I used LLM's for this use case for prototyping purposes. The result looks pretty simiar imo for visualization so full credits to benjdd's awesome visualization, I just wanted this to be in that to see for myself but ended up having it open source/on github pages) I think benjdd's on hackernews too so hi ben! Your websites really cool!
- gus_massa 9mo agoSomeone replied to me in an old comment that for fast Python you have to use numpy. In the folder there is a program in plain python, another with numpy and another with numba. I'm not sure why only one is shown in the data. Disclaimer: I used numpy and numba, but my level is quite low. Almost as if I just type `import numpy as np` and hope the best.
- another_twist 9mo ago> Almost as if I just type `import numpy as np` and hope the best. As do we all. If you browse through deep learning code a large majority is tensor juggling.
- SatvikBeri 9mo agoFor what it's worth, I've ported a lot of heavily optimized numpy code to Julia for work, and consistently gotten 10x-100x speedups, largely due to how much easier it is to control memory allocations and parallelize more effectively.
- Vaslo 9mo agoSo in the D vs Zig vs Rust vs C fight - learn d if speed is your thing?
- Ygg2 9mo agoThat only applies in an apples-to-apples comparison, i.e., same data structures, same algorithm, etc. You can't compare sorting in C and Python, but use bubble sort in C and radix sort in Python. In here there are different data structures being used. > D[HO] and Julia [HO] footnote: Uses specialized datastructures meant for demonstration purposes: more ↩ ↩2
- makapuf 9mo agoYou're right of course but it also depends on how long you want to spend on it. If Python gives you radix sort directly and the C implementation you can have with the same time is bubble sort because you spent much time setting up the project and finding the right libs it kinda makes sense.
- Ygg2 9mo agoPython doesn't come with Radix sort, and Julia doesn't come with [[deps.SuperDataStructures]] git-tree-sha1 = "7222b821efcee6dcdc9e652455da09c665d8afc1" repo-rev = "main" repo-subdir = "SuperDataStructures.jl"
- hiccuphippo 9mo agoDon't know about D but C, Zig and Rust use LLVM so there should be no difference.
- vips7L 9mo agoDepends on the D compiler. The reference compiler optimizes for compilation speed. LDC is backed by llvm and gdc by gcc.
- jhack 9mo agoD gets no respect. It's a solid language with a lot of great features and conveniences compared to C++ but it barely gets a passing mention (if that) when language discussions pop up. I'd argue a lot of the problems people have with C++ are addressed with D but they have no idea.
- rsyring 9mo agoCould say the same for Nim. But popularity/awareness/ecosystem matter.
- elcritch 9mo agoThat's the great thing about LLMs. Especially with Nim it's so easy to make quality libraries with a Codex/ClaudeCode and a couple hours as a hobby. Especially when they run fast. I just made Metal bindings and got 120 FPS demos with SDF bitmaps running yesterday while eating Saturday brunch.
- freeopinion 9mo agoI don't really get the idea that LLMs lower the level of familiarity one needs to have with a language. A standup comedian from Australia should not assume that the audience in the Himalayas is laughing because the LLM the comedian used 20 minutes before was really good at translating the comedian's routine. But I suppose it is normal for developers to assume that a compiler translated their Haskell into x86_64 instructions perfectly, then turned around and did the same for three different flavors of Arm instructions. So why shouldn't an LLM turn piles of oral descriptions into perfectly architected Nim? For some reason I don't feel the same urgency to double-check the details of the Arm instructions as I feel about inspecting the Nim or Haskell or whatever the LLM generated.
- elcritch 8mo agoI don’t trust them. I run tests and I review the code generated by the LLMs. About 1/5 times I’ll just git reset the changes and try again. You have to push for them to add tests. It also helps if you can have the LLM just translate from C++ to Nim. We’re certainly not at the age of LLMs generating code on the fly each time.
- ekianjo 9mo agoData processing benchmark but somehow R is not even mentioned?
- mcdermott 9mo agoIt would be the slowest language result on the list.
- ekianjo 9mo agoSlower than Python? I seriously doubt that
- mcdermott 9mo agoPort the script to R, benchmark and report your results. Python is slow, but R is generally much slower.
- ekianjo 9mo agoI will have a look, but R has much better data structures than Python for data processing (everything is a vector in R) EDIT: they have one script related.R in their repo, which is 3 years old, and uses jsonlite as a package which is notoriously slow. Using a package such as yyjsonr yields 10x performance, so something tells me what whoever wrote this piece of code has never heard of R before.
- pyrolistical 9mo agoThat’s odd zig concurrent got slower
- another_twist 9mo agoContention overhead likely. Performance is more than just the langauge.
- pyrolistical 9mo agoAlso 3 years old. Zig has been rewritten in that time
- piskov 9mo agoC# is very fast (see multicore rating). Implementation based on simd (vector), memory spans, stackalloc, source generators and what have you — modern C# allows you go very low-level and very fast. Probably even faster under .net 10. Though using stopwatch for benchmark is killing me :-) Wonder if multiple runs via benchmarkdotnet would show better times (also due to jit optimizations). For example, Java code had more warm-up iterations before measuring
- von_lohengramm 9mo agoThis entire benchmark is frankly a joke. As other commenters have pointed out, the compiler flags make no sense, they use pretty egregious ways to measure performance, and ancient versions are being used across the board. Worst of all, the code quality in each sample is extremely variable and some are _really_ bad.
- another_twist 9mo agoI mean this is only meant to be an iteration if I understand correctly. Its not like someone is going around citing this benchmark yelling rewrite everything in Julia / D. Imo this is a good starting point if you are doubtful or fall into the trap of Java is not fast. For most workloads we can clearly see, Java trades off the control of C++ for "about the same speed" and much much larger and well managed ecosystem. (Except for the other day, when someones OpenJDK PR was left hanging for a month which I am not sure why).
- nnevatie 9mo agoIf you get the same speeds for C++ and Java, I'd like to point out that the C++ implementation is likely very sub-optimal. This can obviously be true for toy problems, but tends not to generalize.
- inkyoto 9mo agoQuality does vary wildly because the languages vary wildly in terms of language constructs and standard libraries. Proficiency in every.single.language. used in the benchmark perhaps should not be taken for granted. But it is an GitHub repository and the repository owner appears to accept PR's and allows people to raise an issue to provide their feedback, or… it can be forked and improved upon. Feel free to jump in and contribute to make it a better benchmark that will not be «frankly a joke» or «_really_ bad».
- von_lohengramm 9mo agoI'm completely alright with just having fun and hosting your own little sandboxes online, but what good does it do to post and share this with others in its current state? The picture it paints is certainly not representative, and this sort of thing has been done a million times over with much better consistency. Again, I think it's great to hack around in every language and document your journey all the way, but sharing this is borderline misinformation. It's certainly not my duty to right the wrongs of this benchmark.
- deleted 9mo ago[deleted]
- deleted 9mo ago[deleted]
- matthewfcarlson 9mo agoI see some questions around the methodology of the testing. But is this representative of Ruby? Several minutes total when most finish under a second?
- deleted 9mo ago[deleted]
- igouy 9mo agofyi https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/comparable.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
- aatd86 9mo agoIsn't that measuring the speed of json encoding instead?
- KerrAvon 9mo agoGenuine question: Are GitHub workflows stable enough to be used for benchmarking? Like CPU time quantum scheduling is guaranteed to be the same from run to run?
- vlovich123 9mo agoNo, it’s sloppy benchmarking
- jasonjmcghee 9mo agoWhat's up with the massive jump for 20k to 60k for nearly all languages?
- foota 9mo agoMy guess would be cache related. 5k probably fits in L1-L2 cache, whereas 20k might put you into L3.
- XJ6w9dTdM 9mo agoI was very surprised to see the results for common lisp. As I scrolled down I just figured that the language was not included until I saw it down there. I would have guessed SBCL to be much faster. I checked it out locally and got: Rust 9ms, D: 16ms, and CL: 80ms. Looking at the implementation, only adding type annotations, there was a ~10% improvement. Then the tag-map using vectors as values which is more appropriate than lists (imo) gave a 40% improvement over the initial version. By additionally cutting a few allocations, the total time is halved. I'm guessing other languages will have similar easy improvements.
- sergiotapia 9mo agoI wrote a script (now an app basically haha) to migrate data from EMR #1 to EMR #2 and I chose Nim because it feels like Python but it's fast as hell. Claude Code did a fine job understanding and writing Nim especially when I gave it more explicit instructions in the system prompt.
- gethly 9mo agoGo being beaten by C# in multicore is quite hard to believe. Also Zig and Odin doing so "poorly" in single core is strange.
- osmsucks 9mo agoThe quality of the benchmark code is... not great. This seems like Zig written by someone who doesn't know Zig or asked Claude to write it for them. Hell, actually Claude might do a better job here. In short, I wouldn't trust these results for anything concrete. If you're evaluating which language is a better fit for your problem, craft your own benchmark tailored for that problem instead.
- gethly 9mo agoSo far, the best benchmark seems to be the https://plummerssoftwarellc.github.io/PrimeView/ https://plummerssoftwarellc.github.io/PrimeView/ Although it is very single-thread biased test.
- igouy 9mo agohttps://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/fannkuchredux.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
- DeathArrow 9mo agoGo is beaten constantly by C# in both Benchmark Game and Techempower benchmarks.
- mrsmrtss 9mo agoI don't know why this is downvoted, because the statement is not wrong (https://benchmarksgame-team.pages.debian.net/benchmarksgame/box-plot-summary-charts.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...). Times have changed, modern .NET is very fast and is getting faster still (https://devblogs.microsoft.com/dotnet/performance-improvements-in-net-10/ https://devblogs.microsoft.com/dotnet/performance-improvemen...).
- hgs3 9mo agoWhy is there no C benchmark? The C++ benchmark appears to be "modern C++" which isn't a substitute.
- deleted 9mo ago[deleted]
- jakobnissen 9mo agoThe fact that Julia “highly optimized” is 30x faster than the normal Julia implementation, yet still fails to reach for some pretty obvious optimizations, and uses a joke package called “SuperDataStructures” tells me that maybe this benchmark shouldn’t be taken all that seriously. Benchmarks like this can still be fun and informative
- stu2421 9mo agoFor comparison here's one from Dec '25 https://niklas-heer.github.io/speed-comparison https://niklas-heer.github.io/speed-comparison Certainly does "look" very interesting.
- boroboro4 9mo agoThis one doesn’t even have warmup for Java, which makes results complete non sense. Those benchmarks should be just forbidden for their misleading nature.
- igouy 9mo agoHow much difference does it make for tiny programs? https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/nbodyinprocess.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
- boroboro4 9mo agoIt's not an issue of warmup time, it's an issue of jit compilation. On my server (AMD EPYC 7252): 1) base time of the java program from the repo is 3.23s (which is ~2 worse than the one in linked page, so I assume my cpu is about 2 slower, and corresponding best c++ result will be ~450ms 2) if you count from inside of java program you get 3.17s (so about 60ms of overhead) 3) but if you run it 10 times (inside of same java program) you cut this time to 1570ms It's still much slower than c++ version, but it's between rust and go. And this is not me optimizing something, it's only measuring things correctly. update: running vector version of java code from same repo brings runtime to 392ms which is literally fastest out of all solutions including c++. update2: ran c++ version on same hardware, it takes 400ms, so I would say it's fair to say c++ and vectorized java are on par (and given "allows vectorization" comment in cpp code I assume that's the best one can get out of it).
- igouy 9mo ago> the java program Which java program?
- debois 9mo agoThe study seems to be “solve this the obvious way, don’t think too hard about it”. Then the systems languages (C, Zig, C++) are pretty close, the GC languages are around an order of magnitude slower (C#, Java doing pretty good at ca. 3x), and the scripting languages around two orders of magnitude slower. But note the HO-variants: with better algorithms, you can shave off two orders of magnitude. So if you’re open to thinking a bit harder about the problem, maybe your badly benchmarking language is just fine after all.
- dadoum 9mo agoD is a GC language too so the pattern does not hold that well.
- dvfjsdhgfv 9mo ago> Then the systems languages (C, Zig, C++) are pretty close I'm sorry, I don't see C among the results.
- debois 9mo agoMy mistake, sorry. Same for D above. Point stands, though: if your language is too far down the list, better algorithms might be enough.
- igouy 9mo agoon HN 2 years ago https://news.ycombinator.com/item?id=37848571 https://news.ycombinator.com/item?id=37848571 ? unchanged from 7 months ago
- Someone 9mo ago> Rules: > MUST > Support up to 100 tags > Represent tags as strings That doesn’t require the strings that represent the tags to be the tag strings, So, one can bend the rules by representing tags by single-character strings or, alternatively, by using fixed strings of length 0 through 99, and then doing the tag comparisons only on the first character of each string or, alternatively, the length of the string (if obtaining that is fast) Especially when tags have large common prefixes, that could speed up things tremendously. In languages that support string interning (https://en.wikipedia.org/wiki/String_interning https://en.wikipedia.org/wiki/String_interning), I suspect that also could be used to bend the rules.
- DeathArrow 9mo agoIf people don't find their preferred language on top, they will claim the benchmark is flawed. They will find a condition that is not satisfied by the benchmark. But if we operate outside of the benchmarks assumptions, all benchmarks are flawed since they cannot satisfy all possible conditions.
- deleted 8mo ago[deleted]