11 ms·
Benchmarking 20 programming languages on N-queens and matrix multiplication
- raister 3y agoI would expect Mojo to perform better in these settings.
- jorgelopes 3y agoMojo is on the list and performed well. Have a look
- vsskanth 3y agoits 2x slower than C in matmul which is something I wouldn't expect mojo to be slow at
- jorgelopes 3y ago2x slower than C while not optimal at all is not something I would call 'Slow' in this context. Still pretty strong considering how much more friendly it is to write code on for the average user of the languages in the context discussed here
- vsskanth 3y agoIMO it looks way too verbose compared to the (faster) Julia implementation in this repo.
- theteapot 3y agoPHP results: I was stupid enough to write some scientific code in PHP once so know how slow it can be - mostly around array access and manipulation. But if your going to try, use the HHVM interpreter. It's much faster and is a drop in replacement for the PHP interpreter. Hack (https://hacklang.org/ https://hacklang.org/) uses that under the hood by default.
- geek_at 3y agoAlso since PHP was built for web requests it builds The cache on the First run. would be interesting to see All languages of the benchmark with a second run
- habibur 3y agoI put the pure math codes in C extension of PHP. Building C extension for PHP is easier than most of the other high level languages. And then things get blazingly fast.
- brrrrrm 3y agodoesn't look that easy... zend_parse_parameters? pre-baked configure + make scripts? check out how a modern language deals with this stuff https://bun.sh/docs/api/ffi#usage https://bun.sh/docs/api/ffi#usage
- rafark 3y agoCheckout what? PHP also has support for FFI. Or am I missing something?
- EmberTwin 3y agoHHVM is not a drop-in replacement for a PHP interpreter. The semantics of Hack and PHP have diverged, typically in the direction of eliminating dynamic behavior from Hack that existed in PHP (examples: string -> function coercion, the PHP dual vector-hash-table array type, and non-throwing out-of-bounds array accesses are all gone from Hack). The semantics changes both simplify static analysis and make it easier to JIT fast code.
- Ayesh 3y agoThe state of PHP 8.3 and especially 8.4 is a lot better than HHVM. They have diverged quite a lot to a point that it's no longer a drop-in replacement. PHP added JIT in 8.0, and these math-heavy tasks can take advantage of it. It's not trivial to fine tine JIT configuration though. In PHP 8.4 (scheduled Nov 2024), there is a major upgrade to JIT as well.
- p-e-w 3y agoWhat is this supposed to demonstrate? There appears to be roughly the same code structure, ported to every language, while for some languages, arbitrary optimizations are introduced (such as using `array` instead of `list` in Python). But nobody working in Python uses matrix multiplication code written in Python. They use NumPy, which is a de facto standard library for people working in the relevant fields. It's as much part of "Python" as list comprehensions. Without taking such real-world conventions into account, such comparisons say essentially nothing about the languages involved (and their all-important ecosystems).
- morepedantic 3y agoIt demonstrates that Python needs libraries like NumPy. Few problems are more heavily optimized than matrix multiplication in practice, so comparing matrix multiplication benchmarks across languages with NumPy is not representative of real-world performance for most programming use cases. It also means that adding performance to an existing Python program requires dropping into a different language, which is not only complicated, but also requires engineers capable in both Python and C (or similar).
- lifthrasiir 3y agoYou don't need any C knowledge to use numpy. In fact, its conceptual similarity with Matlab is possibly the single most important reason for its popularity. Many other problems do need specialized treatments that would indeed require other languages, but numpy is not a good counterexample.
- jakobnissen 3y agoYou're missing the point. The point is that, for any application, Python needs an underlying C library to be fast. So if you need to solve problems where no such library exists, Python is slow. In other words, Python IS slow, but it can call fast code written in other languages.
- 3y ago
- lifthrasiir 3y ago> It is obvious that c[i], b[k] and a[i][k] can be moved out of the inner loop to reduce the frequency of matrix access. [...] However, most other languages cannot optimize this nested loop. If we manually move a[i][k] to the loop above it, we can often improve their performance. This is only true when three matrices are independent of each other, and also why C has a `restrict` qualifier that enables this assumption. The benchmark itself has no such assumption because all three variables are defined as `double **`, and it can be verified by assembly outputs. Clang's excellent performance in matmul is probably much more due to autovectorization.
- yxhuvud 3y agoIn some cases there are other reasons for hoiting the dereference too. For example, Crystal will check if the array access is out of bounds and by hoisting variables that will be done a lot less seldom, which can have huge effects for code that does a lot of that, like matmul.
- cm2187 3y agoDoesn’t that get mostly optimised away by the cpu branch prediction?
- beagle3 3y agoThe branch does, but you still have the comparison itself, and also the branch instruction to decide and skip (although thanks to branch prediction, your pipeline doesn’t get flushed). If you have enough idle execution units, you might not see a difference in wall clock time. But with many algorithms you can put those units to good use.
- yxhuvud 3y agoThe branch does, but not optimizations that could be done, like vectorization.
- jodrellblank 3y agoDoesn’t include Prolog which has decent constraint solver answers for n-queens and sudoku, which are pretty fast but I don’t know how they would compare in benchmarks: https://www.metalevel.at/queens/ https://www.metalevel.at/queens/ https://www.metalevel.at/sudoku/ https://www.metalevel.at/sudoku/
- trenchgun 3y agoMake a pull request.
- shpx 3y agoYou should add a chart of the number of gzip'd bytes of source code.
- morepedantic 3y agoIMO, uncompressed bytes is a better representation, because it can be used to compare relative expressive power for the particular problem. I'd bet Python cleans house here, but the write-only languages are a wild card.
- dawnofdusk 3y agoWhy would uncompressed bytes be better? Using a good compression algorithm better approximates the statistical entropy of the code which is at least correlated with e.g., Kolmogorov complexity.
- iopq 3y agobecause it often sits uncompressed on the drive
- deleted 3y ago[deleted]
- morepedantic 3y agoBecause humans read and write the uncompressed code. Gzip will hide problems like copy paste.
- igouy 3y agoAnd hide differences due to label-length personal-preferences.
- lifthrasiir 3y ago"Expressive power" is a very subjective term, and uncompressed size is a bad proxy as it includes too many variables specific to coding conventions. Compressed size with a stupid enough algorithm (here gzip) is meant to reduce these variables. The true Kolmogorov complexity in comparison can't be computed, and too smart algorithms can start to infer enough about the language itself.
- alberth 3y ago> ” Timing on Apple M1 Macbook Pro” Given that its become increasingly more common for CPUs to have both Performance & Efficency cores … how do benchmarks ensure they are only being run on the P-cores?
- sroussey 3y agoIt actually takes a bit of effort to run on the efficiency cores. I believe Game Mode will push the processes to e cores to keep a consistent game play without thermal throttling.
- vlovich123 3y agoSibling answered on macOS. Their scheduler prefers by default to run everything on performance and reserves efficiency for background OS tasks by default. On Linux you could set the affinity when running by pinning the cpu affinity either upfront through task set or programmatically at runtime (eg I have 32 cores and 0-15 are performance while the rest are efficiency)
- sakras 3y agoHow come not all benchmarks appear on all languages? For example, Zig's bar appears to be lower than C's by virtue of sudoku and bedconv being missing.
- tmtvl 3y agoSomeone needs to write sudoku and bedconv for Zig.
- Atotalnoob 3y agoAren’t JIT languages at a disadvantage since they are benchmarked through the CLI rather than using a benchmarking library to allow JIT to warmup?
- jakobnissen 3y agoYes, but the author claims the longest JIT warmup is 0.3 seconds, so it's not an important issue in these benchmarks that take several seconds.
- lifthrasiir 3y agoI strongly suspect that the author may have confused the JIT warmup (hard to measure, as you need to ensure that the performance figure have reached the stable point) from the startup overhead (easy to measure).
- vkazanov 3y ago...and many jitted vms do not even reach a stable point at all, there was a big paper on this a couple of years ago.
- nithril 3y ago0.3s is significant for a task that only takes 1.14s
- daxfohl 3y agoOdd that nqueens and sudoku have a high correlation but matmul seems to be largely doing its own thing. nqueen vs. sudoku: 0.531 matmul vs. sudoku: 0.362 matmul vs. nqueen: 0.127
- brrrrrm 3y agomatmul is old and useful -- there's a lot of hardware on a chip that makes it run much faster (prefetch, vectorization, instruction parallelism) and some of these languages have optimizations to expose those things automatically.
- yxhuvud 3y agoAnd some languages that perform array bounds check on each array deindex operation, which basically stops some of the above even if the optimizer can do all of it. Crystal is an example of that, where it would be possible and quite straightforward to write a specialized matrix class instead of the very generic dynamic array implementation.
- gnufx 3y agoThose will not simply give you close to optimal performance (>~90% of peak), even, say, POWER10's 4x4 matmul instruction. You need a structure matching the micro-architecture -- the cache and register structure [1]. That's not a triply-nested loop. Any remotely decent compiler will unroll and vectorize the micro-kernel appropriately, but you may still have to resort to assembler-level prefetch fiddling for the last 10s of percent performance (specifically on avx512). Compilers may recognize the loop structure and replace it with an optimal-ish implementation. I've an idea that FORTRAN H extended did that, but Polly does it in clang. 1. https://www.cs.utexas.edu/users/flame/pubs/blis1_toms_rev3.pdf https://www.cs.utexas.edu/users/flame/pubs/blis1_toms_rev3.p...
- quelsolaar 3y agoGreat post: Id be interested to see the difference between different C compilers, and to see what the difference is if the C code is compiled with a C++ compilers, if possible.
- metaltyphoon 3y agoWhy not run C# as AOT? It’s a one line change.
- borissk 3y agoDo people usually run C# code AOT compiled?
- romanovcode 3y agoIf it's for something stupid like benchmark comparison - yes.
- neonsunset 3y agoIt’s gaining popularity for CLI and serverless, some people use it to replace C++ for writing native libraries and plugins as well.
- mlhpdx 3y agoYes, everywhere I possible can. Particularly for targeting CLI programs and Linux daemons.
- metaltyphoon 3y agoI don't know about people but I do pretty much every where I can. I make such a difference that AWS is heavily investing on lambdas that uses AOT[0]. [0] - https://docs.aws.amazon.com/lambda/latest/dg/dotnet-native-aot.html https://docs.aws.amazon.com/lambda/latest/dg/dotnet-native-a...
- xnorswap 3y agoOr use BenchmarkDotNet which, among other things to get an accurate benchmark, does JIT warmup outside of measurement. ( https://github.com/dotnet/BenchmarkDotNet https://github.com/dotnet/BenchmarkDotNet ).
- latenightcoding 3y agoThese are not the best benchmarks, but Python is indeed as slow as Perl, which I find insane considering that Python has 100-1000 more people working on the interpreter and performance has been a big emphasis the last few years.
- d0mine 3y agoThe benchmark is not representative of how Python is actually used in practice. It ignores the existence of libraries (explicitly). For example, If numpy, pytorch were used for matmul, the results would be completely different.
- latenightcoding 3y agoPerl people would use PDL or Math::GSL. I look into these benchmarks because I'm interested in VMs, and yeah the Python VM can be quite slow if you are not calling C code.
- sneed_chucker 3y agoWell, for a long time speed was not a priority at all for the team developing cpython. In fact, Python 3 was still a bit slower than Python 2 until a few versions ago. Recently there have been some decent improvements to CPython's speed, but there are real upper limits to how fast you can make an interpreter. CPython will need JIT compilation if it is ever to break out of its current speed bracket. JavaScript has had a feature complete JIT reference implementation since 2008 which is a major part of the reason JS applications exploded so much in the 2010s.
- jakobnissen 3y agoNeat! The Julia matmul implementation has its rows and columns flipped though - unlike C, Julia uses row-major matrices. This has large implications for speed. Also, the code may be much faster if you enable SIMD in the function, which is disabled in the code because a) the code unnecessarily checks bounds at every index instead of at the top of the function, and b) float SIMD is opt-in since SIMD changes the rounding
- mafuy 3y agoYes. For this type of test, Julia should be nearly the exact same speed as C unless there is a mistake in the code. Which, to be fair, can happen very easily, especially with implicit typing.
- attractivechaos 3y agoJust come here to say this has been fixed in PR #2. The figure has been updated accordingly.
- borissk 3y agoN-queens and matrix benchmark without Fortran and Wolfram Mathematica?
- smitty1110 3y agoI was just popping in to make a similar comment. Fortran is probably doing the heavy lifting for most of the entries here, you might as well show how much your language-specific overhead is by including it.
- bjourne 3y agoNo, it isn't. OP is benchmarking language-native implementations of all algos.
- Haemm0r 3y agoFortran the king of matrices :-)
- daxfohl 3y ago[flagged]
- sa46 3y agoReliably benchmarking optimizing VMs like Java and JavaScript is a research problem. See “Virtual machine warmup blows hot and cold”. The authors run a variety of benchmarks (with a pristine methodology) and find 43% of them provide “bad inconsistent” results. Paper summary: https://blog.acolyer.org/2017/11/07/virtual-machine-warmup-blows-hot-and-cold/ https://blog.acolyer.org/2017/11/07/virtual-machine-warmup-b... Paper: https://dl.acm.org/doi/pdf/10.1145/3133876 https://dl.acm.org/doi/pdf/10.1145/3133876
- keyle 3y agoI love a good benchmark, thanks for putting this together! However, I have a bit of feedback. First, the graph is misleading, stacking times with languages that have half the implementation, they appear faster, until you dig in. I'd suggest producing an alternate graph that shows only the implemented puzzles in every language, or make a unique graph for every language:puzzle. Second, the examples are taken from rosetta code and are not necessarily what would be the best implementation, or even close to the best implementation, for benchmarking purposes. Finally, those examples should be reproduced across various hardware platforms, I'm on arm64 Darwin myself, but you might find different results on Intel platforms due to the various compiler optimizations available based on the hardware. More benchmarks would be interesting to see, such as actual real world operations, e.g. opening a file, reading it, parsing json, opening a socket server, etc.
- WhereIsTheTruth 3y agoThere is one for data processing here: https://github.com/jinyus/related_post_gen https://github.com/jinyus/related_post_gen
- cm2187 3y agoAlso it may or may not be relevant to not exclude the warm up time for JIT languages. If it is for a calculation you will do continuously, including the warm up time may bias the results materially (I know for .net it can be significant).
- attractivechaos 3y agoClarification: the examples are not taken from Rosetta code. Only the n-queens algorithm is inspired by (but still different from) an Rosetta code implementation; otherwise this benchmark has nothing to do with Rosetta code.
- account-5 3y agoGlad to see dart in there, it's normally overlooked. Not a bad result. I wonder what speed it would be if it had been AOT compiled instead of JIT.
- adamredwoods 3y agoAnd LuaJit surprised me quite a bit!
- igouy 3y agofwiw https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/dartexe-dartjit.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
- neonsunset 3y agoC# code should be using an official package for GEMM which is System.Numerics.Tensors, it is pure C# but runs at max hw efficiency (it is also idiomatic). Or at least use Vector<T> instead of scalar operations. I’d expect this applies to most other popular languages here too.
- attractivechaos 3y agoSystem.Numerics.Tensors is disqualified because it uses a different algorithm. If you have a faster matmul implementation in Vector<T>, a PR will be much appreciated. Thank you in advance.
- naet 3y agoSurprised how well JS (node and others) seem to come out when I've had firsthand experience of switching from JS to Go to speed up an algorithm type question and had the Go version crank through a bruteforce much much faster. Maybe I made an accidental optimization in my language translation, or maybe there are some operations that are much slower in JS and these benchmarks didn't hit any of them.
- lifthrasiir 3y agoGo's AOT compiler is actually not that sophiscated (comparable to -O1 in most C compilers, possibly even worse). Was your program running only for a fraction of a second? Then the JIT may haven't fully warmed up and even a basic AOT compiler has a better chance to win.
- nnx 3y agoNice to see Go performing well. Would be interesting to see the results with PGO enabled.
- 1letterunixname 3y agoFor the love of god, log graph tiny values with large values please. :)
- xnorswap 3y agoAnother alternative here is to plot the reciprocal (Op / Sec) which has the benefit of both the more natural "Bigger is Better" and easier comparison of the fastest runtimes.
- igouy 3y ago"Bigger is Better" may be "more natural" for Op / Sec but secs is "more natural" for "comparison of the fastest runtimes".
- airstrike 3y agoOr just make two charts ;-)
- hyperman1 3y agoI wondered why rust is so far behind C/nim/zig, they should have similar behaviour. The difference is mostly in matmul. I see how C, for an n x n matrix, does 2 allocations, while rust does n+1. C's matrix rows are right next to each other, rusts are probably all over the place. Didn't look at nim or zig. Maybe a slice of slice of double would perform better than a vec of vec of double? Then again, an argument can be made that rust pushes you to vec, so this impl is more honest for how a beginner would do it. Otoh, C has the optimization so why doesn't rust?
- nrabulinski 3y agoThe rust code is very unidiomatic, not only because of the Vec of Vecs which I’d say, even if it’s the obvious naive approach, no one experienced wouldn’t choose over a flat slice, the implementation itself is very naive and unidiomatic.
- hgomersall 3y agoAlso it's using checked indexing, which apart from being not idiomatic, is also going to slow things down. A fairer comparison would be to use the unchecked indexing variants.
- neonsunset 3y agoYeah, it's a similar problem in C# - ported code 1:1 is not representative since it now can't do out of bounds accesses and is therefore safe but comes at a cost of bounds checks which can be manually elided.
- vlovich123 3y agoUnchecked indexing is unidiomatic. A more fair comparison world be using iterators if possible but otherwise you have to live with that.
- tiehuis 3y agoZig had a similar issue which I made an MR to fix. I didn't actually notice Rust had the same issue but that's probably just my forgotten knowledge with how the vec! macros expand when nested.
- _giorgio_ 3y agoGiven the times we're in, it would be interesting to test some libraries too, like numpy and pytorch
- deleted 3y ago[deleted]
- brightball 3y agoI wonder if JRuby or Truffle Ruby would have a significant impact on this one for Ruby?
- igouy 3y agoPlease make and publish those measurements!
- akashcoach 3y agoWhy is c# so slow for Matmul compared to java?
- mlhpdx 3y agoThe implementation isn’t using any modern C# features?
- akashcoach 3y agoWhich modern features do you mean? Java and C# code looks similar to me.
- neonsunset 3y agoUnfortunately it doesn't. The newest and hottest way to do this is to either use bespoke matmul from System.Numerics.Tensors or at least using Vector<T> for SIMD (which is trivial and not "the last mile" optimization it often seems to be).
- useerup 3y agoI believe because the C# version has been written using rectangular arrays. This requires every array access to use a multiplication. The Java version uses array-of-arrays and hoisting the inner array out before accessing it in the inner loop. C# also has arrays-of-arrays, and could (should) be written in the same manner.
- akashcoach 3y agoThanks for the explanation. Yes this make sense to me.
- iruoy 3y agoI've just done this and it has been merged. The benchmarks table and image haven't been updated yet. But this should bring the C# result to ~2s instead of 4.67s
- doug_durham 3y agoI think this shows the value of programmer productivity over performance at all costs. Python is one of the most popular languages despite having performance issues for complex algorithms. Users value clarity and ease of expression over performance. That's why Python is primarily used a glue code in these complex tasks.
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- wslh 3y agoNowadays I would add lines of code and syntax complexity to the benchmark. It is apples and oranges comparison but you could prefer clarity than performance depending on the circumstances. Also, it is relatively easy to do with today language tools.
- igouy 3y agoFor example: https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/nbody-gz.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... "How source code size is measured" https://benchmarksgame-team.pages.debian.net/benchmarksgame/how-programs-are-measured.html#source-code https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
- mtreis86 3y agoA benchmark I would like to see is a comparison of languages in terms of how fast they are to beginners vs experts. I've been thinking about how to design it to get that result. What I think would work is taking something like these simple puzzles and have maybe a hundred people write up different solutions, so we can compare them using the programmer's level of expertise as one of the factors.
- igouy 3y agoHow would you decide who were beginners and who were experts? Here are naive line-by-line transliterations from an original C program: https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/simple.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... Here exhaustively-optimised + multicore + vector-instruction programs are included: https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/mandelbrot.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
- wk_end 3y agoThis is great, thanks for posting this. OP - are you interested in pull requests adding support for other languages?
- attractivechaos 3y agoOf course! Please implement at least nqueen and matmul as they have been implemented in every language in the benchmark.
- airstrike 3y agoVery interesting. My two cents: because the stacked bars for php, ruby, perl and py:cpy, it's impossible to compare the other languages in that first chart. All the chart says is that the benchmark is much slower in those 4 languages relative to "all other languages we tested". It would be nice to see those other languages in a chart that doesn't include the slower four. Alternatively, you could also show those slower four with "broken" columns like this https://peltiertech.com/broken-y-axis-in-excel-chart/ https://peltiertech.com/broken-y-axis-in-excel-chart/
- tmtvl 3y agoThe second chart is basically the first chart with the slow 4 removed (well, not removed but at such a scale that they're irrelevant).
- airstrike 3y agoAh, duh, I see that now. That's what I get for commenting too quickly ;-) The choice of stacked bar charts is also strange because the languages are not all comparable > Every language has nqueen and matmul implementations. Some languages do not have sudoku or bedcov implementations. In addition, I implemented most algorithms in plb2 and adapted a few contributed matmul and sudoku implementations in plb. As I am mostly a C programmer, implementations in other languages may be suboptimal and there are no implementations in functional languages. Pull requests are welcomed! So the point still stands that many charts doing one thing each would be better than fewer charts doing many things each
- pasc1878 3y agoSee also https://benchmarksgame-team.pages.debian.net/benchmarksgame/index.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/... for more algorithms and code hand tuned as well as plain.
- igouy 3y agoThanks, and they know: "Plb2 complements the Computer Language Benchmark Games."
- FrustratedMonky 3y agoIt has .NET generic. Would .NET with F# make big difference here? I'm little surprised Java beat .NET. Is that typical these days?
- montebicyclelo 3y agoThe reality is that it would be very hard to find Python code that does not use NumPy (or some tensor lib), for matmul. Including time to JIT compile is questionable, why not also include time to compile the compiled languages?
- igouy 3y ago"Nonetheless, because most benchmarks run for several seconds, including the startup time does not greatly affect the results." https://github.com/attractivechaos/plb2#startup-time https://github.com/attractivechaos/plb2#startup-time
- montebicyclelo 3y agoYes, I saw that and still consider the methodology questionable. A fairer approach might be: time from the cli, including compilation, for compiled languages. (Or warmup the jit compiled code.)
- igouy 3y agoThose are unfounded conjectures. When you show the results for those programs have been greatly affected … As-in "Wtf kind of benchmark counts the jvm startup time?" https://benchmarksgame-team.pages.debian.net/benchmarksgame/sometimes-people-just-make-up-stuff.html#jvm-startup-time https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
- taeric 3y agoThis is an area where "solvers" are heavily under utilized. For the absolute fastest solution, bespoke implementations are almost certainly required. However, translating Sudoku and N-Queens into a similar problem that you can feed into a solver can get you a long way. Even better, you can move that solver into whatever language gives you the best optimizations that you can work. Even better, there are almost certainly optimizations in common solvers that you don't want to deal with implementing on your on.
- bjourne 3y agoHaving checked the C implementations of the algorithms, I'm a little skeptical because the implementations aren't optimized. Tiling matrix multiplication exploiting SIMD could easily improve performance 100-fold or more. At those speeds the cost of memory transfer usually dominate so the languages that give you the most fine-grained control over how data is laid out in memory tend to win. And it may not be the same languages that are now winning in your benchmark.
- nerdponx 3y agoI think a benchmark of "naive" implementations is interesting too, because it shows you how fast your code is usually going to run, not how fast it theoretically could run at its best.
- igouy 3y ago> shows you how fast your code is usually going to run Why would you think that? Maybe the "naive" implementation just shows how fast the easily removed hotspot in your code is going to run. "Swap the order of two statements and see the Java code slow down … Swap globals for local variables in a function and see the Python code speed up. Swap language implementations and see the C code speed up." https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/toosimple.html https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
- GrumpySloth 3y agoA curious thing about Swift: after https://github.com/attractivechaos/plb2/pull/23 https://github.com/attractivechaos/plb2/pull/23, the matrix multiplication example is comparable to C and Rust. However, I don’t see a way to idiomatically optimise the sudoku example, whose main overhead is allocating several arrays each time solve() is called. Apparently, in Swift there is no such thing as static array allocation. That’s very unfortunate.
- attractivechaos 3y agoFigure updated. Now swift is pretty fast on nqueen+matmul but it has the longest green bar (i.e. longest running time for sudoku). This looks ... interesting. There are only 20 thousand array allocations in total, not a lot. Javascript also has these many arrays allocated/deallocated but it is 4 times as fast.
- Jaxan 3y agoOne thing I always wonder with these benchmarks is: do we want to know the best possible runtime for a language or the runtime of idiomatic/average implementations? Even with C there are loads of compiler flags (and compilers) to choose from.
- igouy 3y agoThe data has nothing definite to say about language implementations that were not measured. Let's focus on what we might say about those that we're measured.