4 ms·
How about extending this to comparing programming language efficiency from a developer's standpoint, looking at bytes of code, versus the execution time? I too
by compumike 4y ago
How about extending this to comparing programming language efficiency from a developer's standpoint, looking at bytes of code, versus the execution time?
I took the source code file size from the repository, and the runtime from the blog post.
Then I made an arbitrary overall "PAIN SCORE" (lower is better) by multiplying code size * runtime. I suggest this is a worthwhile metric simply because lower is better on both axes, but of course, in the "real world" there will be different economic costs to CPU time and developer time depending on the use case. Here's the sorted results, from least "pain" to most:
LANGUAGE FILENAME CODE SIZE RUNTIME PAIN SCORE (LOWER IS BETTER)
Shell optimized.sh 75 bytes 1.83 s 137.25
Crystal simple.cr 240 bytes 1.29 s 309.6
Nim simple.nim 424 bytes 0.77 s 326.48
Python simple.py 208 bytes 2.21 s 459.68
Ruby simple.rb 175 bytes 3.17 s 554.75
Go optimized.go 1514 bytes 0.40 s 605.6
Python optimized.py 464 bytes 1.33 s 617.12
Zig optimized.zig 2688 bytes 0.24 s 645.12
Go simple.go 688 bytes 1.12 s 770.56
Zig simple.zig 1394 bytes 0.55 s 766.7
Nim optimized.nim 1683 bytes 0.49 s 824.67
Shell simple.sh 60 bytes 14.81 s 888.6
Ruby optimized.rb 401 bytes 2.47 s 990.47
JavaScript simple.js 532 bytes 1.88 s 1000.16
C optimized.c 4360 bytes 0.23 s 1002.80
Rust optimized.rs 3065 bytes 0.43 s 1317.95
Swift simple.swift 317 bytes 4.23 s 1340.91
JavaScript optimized.js 1501 bytes 1.10 s 1651.1
C simple.c 2735 bytes 0.96 s 2625.6
Rust simple.rs 2239 bytes 1.38 s 3089.82
Sorting only by code size, the most concise implementations are: Shell, Ruby, Python, Crystal. Nobody was aiming to play code golf (i.e. minimize source code size), so these are fairly straightforward, idiomatic, readable implementations.
I am definitely a Crystal fan, in fact this afternoon I'm continuing to implement suggestions from my recent Show HN https://news.ycombinator.com/item?id=32081943 https://news.ycombinator.com/item?id=32081943 comments. :)
- igouy 4y ago> a worthwhile metric Previously — "Completely Random and Arbitrary Point System!, or CRAPS![TM]" https://web.archive.org/web/20010124100400/http://www.bagley.org/~doug/shootout/craps.shtml https://web.archive.org/web/20010124100400/http://www.bagley... Currently — fwiw https://benchmarksgame-team.pages.debian.net/benchmarksgame/box-plot-summary-charts.html#chart-fastest-simplest https://benchmarksgame-team.pages.debian.net/benchmarksgame/... > Sorting only by code size https://benchmarksgame-team.pages.debian.net/benchmarksgame/how-programs-are-measured.html#source-code https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
- adsharma 4y agoIt'd be interesting to write code in a concise, easy to learn/grok language and then transpile to a faster systems language and recompute the PAIN score. I spent a few months last year pursuing that approach (py2many on github). We'd need a stdlib that works across languages for the approach to be successful. nimpylib and similar libraries in other languages are a step in that direction. Alternatively, the python stdlib itself could be rewritten in python and transpiled to other languages. Perhaps it'd help alternative python implementations in terms of compatibility.
- optionalsquid 4y agoIt would probably be prudent to strip comments before measuring the code sizes: I did not check all files, but noticed that the simple rust code consists of about 60% comments (in terms of bytes), mostly due to a verbose blurb. It also spends about 15% of the non-comment bytes on printing nicer error messages, which is an interesting choice for a micro-benchmark. The simple C and FORTH versions similarly have a lot of comments. Meanwhile a lot of the other files have very few or no comments.
- burntsushi 4y agoI wrote the Rust version and the comments are a reflection of my style and my belief that benchmarks should come with some kind of analysis. The "simple" variant was also intended to demonstrate idiomatic code without a bunch of perf tricks. Nicer error messages in that case is fair game. Indeed, a naively analysis based on code size without taking comments into account is pretty obviously wrong.
- jandrese 4y agoI would expect the AWK version to have very low pain despite its reputation. This is the sort of problem that AWK is well optimized for.
- bestinterest 4y agoAlso a Crystal fan but from afar. The compile time speeds are a real killer of productivity.
- dureuill 4y agoProgramming language efficiency from a developer's standpoint is not measured in bytes written in the source code, because typing characters is never the bottleneck when writing code. Even if it were, most code isn't "fire and forget", and so for most code the cost is dominated by maintenance cost, which has more to see with reading than writing code. Languages that are concise are so because they either express information very densely, or express less information (via for example ignoring error handling or making it implicit with exceptions). In practice I find that such languages are much harder to maintain, because the missing information has to be rebuilt by the reader.
- norman784 4y agoIsn’t unfair to not count the runtime code size too for interpreted languages? Because for the compiled ones, they include all the necessary runtime in the executable.
- Too 4y agoExtra pain score should be awarded for 2 character abbreviations, one character flags, mode switches, special character operators, difficult documentation, lack of error handling, missing type safety, difficulty of introspecting/debugging halfway through program, environment dependencies. Most of these weigh a lot more than just line count.
- 1980phipsi 4y agoI think another interesting "pain score" is whether the code is written with built-ins from the language or the standard library (i.e. not new data structures or functions).