3 ms·
I've been hacking on a little scripting language of my own lately[1]. To get a handle on its perf, I have a few small benchmarks[2] I run that compare it agains
by munificent 13y ago
I've been hacking on a little scripting language of my own lately[1]. To get a handle on its perf, I have a few small benchmarks[2] I run that compare it against Lua (5.1.5), Python (2.7.5), and Ruby (2.0.0p247). (All of those and my language are bytecode-compiled, dynamically typed languages, hence that comparison.)
I happen to have Ruby 2.0 installed, and I was surprised to see it handily beat the other languages. I had expected Ruby to be the slowest of the pack, but they've apparently made big strides in 2.0.
On the off chance that anyone is curious, here's the results of running them on my machine:
binary_trees - wren 3531 0.28s
binary_trees - lua 1366 0.73s 258.40%
binary_trees - python 1420 0.70s 248.64%
binary_trees - ruby 3170 0.32s 111.39%
fib - wren 2688 0.37s
fib - lua 2998 0.33s 89.65%
fib - python 1390 0.72s 193.38%
fib - ruby 3645 0.27s 73.74%
for - wren 9092 0.11s
for - lua 9387 0.11s 96.86%
for - python 3171 0.32s 286.74%
for - ruby 8319 0.12s 109.30%
method_call - wren 4703 0.21s
method_call - lua 1780 0.56s 264.20%
method_call - python 806 1.24s 583.33%
method_call - ruby 3408 0.29s 138.02%
Those numbers are score, time, and percentage. Score is basically 1/time, just to yield a number where bigger is better. Time is best of ten runs. Percentage is how my language's score compares to that one. So, for example, the 264.20% lua method_call score means my language is about 2.6x faster than Lua on a benchmark designed to stress dynamic method call performance.
[1] https://github.com/munificent/wren https://github.com/munificent/wren
[2] https://github.com/munificent/wren/tree/master/benchmark https://github.com/munificent/wren/tree/master/benchmark
- StefanKarpinski 13y agoOh, man, this reminds me of the pain of writing even the most trivial microbenchmarks in seven or eight languages. How about throwing in some of the fast scripting languages for healthy competition? E.g. JavaScript, LuaJIT and Julia. Feel free to borrow (MIT licensed, but I'm willing to relicense whatever I can): https://github.com/JuliaLang/julia/tree/master/test/perf/micro https://github.com/JuliaLang/julia/tree/master/test/perf/mic...
- munificent 13y agoI'm certainly impressed by those implementations, but I don't think I'm competing with them. :) I had JS (using node) in there for a while, but it was regularly 10x compared to the other languages. Using a JIT makes it effectively a different class of implementation. It's like entering a car in a bike race. Thanks for the benchmarks! I might steal some of those. They may be more arithmetic focused than I need (which makes sense for Julia, but less sense for an embedding scripting language where most math is probably done in the host app).
- StefanKarpinski 13y agoIt was a turn of phrase, but really I just meant that in my experience it's healthy to have something to aspire to – that's why we always compare to fully optimized C and Fortran. It's easy to fall into an "interpreter bubble" and forget that there's a whole other realm of performance out there. And of course, interpreters can be quite fast – CPython is no slouch and Wren is giving it a run for its money, so kudos. Femtolisp [1] (which we use for Julia's parser) is a great example of a small and simple, yet fast and featureful interpreter. I really want Jeff to write a little book about it, walking the reader through the implementation. [1] https://github.com/JeffBezanson/femtolisp https://github.com/JeffBezanson/femtolisp
- munificent 13y ago> I just meant that in my experience it's healthy to have something to aspire to I had actually thought that's what I was doing when I compared it to Lua! I was quite surprised at how well Wren did in comparison. The trick will be to see if I can keep it that fast as the implementation gets more robust. It takes some shortcuts right now that don't scale well, but happen to be faster. > I really want Jeff to write a little book about it, walking the reader through the implementation. I would love that. I've skimmed femtolisp a few times and it's clean but suffers from the almost total lack of documentation that other programming language implementations also seem to have.
- 13y ago
- yareally 13y agoI know you were just going with what was already on your system most likely, but wouldn't it be a good idea to compare against a more up to date version of Python (version 3.3.x) and Lua (5.2.x)? Lua 5.1 initial release was in 2006[1] Python 2.7 initial release was in 2010[2] Ruby 2.0 initial release was in 2013[3] [1] http://www.lua.org/versions.html#5.1 http://www.lua.org/versions.html#5.1 [2] http://en.wikipedia.org/wiki/History_of_Python#Version_release_dates http://en.wikipedia.org/wiki/History_of_Python#Version_relea... [3] https://www.ruby-lang.org/en/news/2013/02/24/ruby-2-0-0-p0-is-released/ https://www.ruby-lang.org/en/news/2013/02/24/ruby-2-0-0-p0-i...
- munificent 13y agoGood call. I'm not sure how I ended up with that version of Lua. I'll upgrade to the latest of the others when I get a chance.