4 ms·

MRI Ruby seems to be making a lot of progress in the implementation details recently. Good to see.

munificent 13y ago
Anecdote time! I've been hacking on a little scripting language[1] lately. To see how its performance compares, I run a few benchmarks[2] against other similar (dynamically typed, bytecode compiled) languages: Lua, LuaJIT (interpreted), Python, and Ruby. Like others, I had internalized "Ruby is slow" through osmosis. But the version of Ruby I happen to have on my machine is 2.0.0. In my little benchmarks, it turns out Ruby is one of the fastest. I haven't compared to 1.8.7, but I'm guessing that was much slower. [1] https://github.com/munificent/wren https://github.com/munificent/wren [2] https://github.com/munificent/wren/tree/master/benchmark https://github.com/munificent/wren/tree/master/benchmark If you want the gory details, here's the results of running them against Lua 5.3.2, LuaJIT 2.0.2, Python 2.7.5, and Ruby. (Yes, I should try against Python 3. I will.): score time wren score relative binary_trees - wren 3193 0.31s binary_trees - lua 1366 0.73s 233.72% binary_trees - luajit 6256 0.16s 51.04% binary_trees - python 1376 0.73s 231.96% binary_trees - ruby 3003 0.33s 106.34% fib - wren 3078 0.32s fib - lua 2785 0.36s 110.52% fib - luajit 6900 0.14s 44.62% fib - python 1331 0.75s 231.37% fib - ruby 3548 0.28s 86.77% for - wren 6080 0.16s for - lua 1990 0.08s 50.71% for - luajit 5914 0.02s 13.24% for - python 2825 0.35s 215.25% for - ruby 6595 0.15s 92.19% method_call - wren 4707 0.21s method_call - lua 1674 0.60s 281.17% method_call - luajit 4221 0.24s 111.53% method_call - python 767 1.30s 613.69% method_call - ruby 3061 0.33s 153.77% As you can see, Ruby fares quite well.
- mcguire 13y ago
  "dynamically typed, bytecode compiled" I haven't followed Ruby development, but the last time I heard, Ruby 1.8 and prior was a tree-walking interpreter---simple to implement but not fast at all.
  - munificent 13y ago
    That's correct. I still think Ruby 1.8 would be a useful data point just because it's a widely used language that many people are familiar with, but in this case I'm comparing it to Ruby 2.0, so it is apples/apples. For what it's worth, I look at these benchmarks as validating feasibility more so than really trying to win some performance fight between languages. In order to argue that my language is suitable for real world use, I need to show it's performance is comparable to other languages that are widely used. From that angle, Ruby is a fine comparison, regardless of how it's implemented.
  - srd 13y ago
    This changed in 1.9. Ruby is now compiled into a stack machine based bytecode. Ruby 1.8 used the AST internally, you're right there. This made e.g. method calls very slow compared to the new byte code representation.
  - steveklabnik 13y ago
    1.8 -> 1.9 was an entire re-write, it's now a bytecode compiled language: http://www.ruby-doc.org/core-2.1.0/RubyVM/InstructionSequence.html http://www.ruby-doc.org/core-2.1.0/RubyVM/InstructionSequenc...
- alberth 13y ago
  >> "To see how its performance compares, I run a few benchmarks against ... LuaJIT (interpreted)" Why would you use LuaJIT in interpreted mode? That defeats the main reason for using LuaJIT, and you'll see massive improvement in speed if you run it not interpreted [1] It should also be noted that, even in the slower interpreted mode of LuaJIT - it was still the fastest in completing your benchmarks compared to all other languages/implementations. It will run even faster if you don't run LuaJIT in interpreted mode. [1] http://luajit.org/performance_x86.html http://luajit.org/performance_x86.html
  - munificent 13y ago
    Running LuaJIT interpreted is a deliberate choice. Some platforms, in particular iOS and game consoles, do not allow generated machine code. I want my language to be usable for games, and that's one reason I'm using bytecode. (Simplicity is the main reason.) Given that, I think it's most helpful for me to compare Wren's performance to other bytecode language implementations. Comparing to JIT compiled languages is a bit apples/oranges. They tend to be something like an order of magnitude faster (and an order of magnitude more complex!). Meanwhile, LuaJIT's interpreted mode is very fast as you note. I think it's one of the fastest bytecode interpreters around, so it's a great target for Wren to aim for. If you want a lot more detail on my thoughts about this, see here: https://github.com/munificent/wren/blob/master/doc/site/performance.markdown https://github.com/munificent/wren/blob/master/doc/site/perf...
- the8472 13y ago
  What makes ruby (MRI particularly) slow are metaprogramming things that slow it down when done at runtime (dynamically definding methods and classes), its global interpreter lock (preventing proper parallelism) and the single-threaded super-slow GC. Integer math is fast because they're using tagged pointers and thus don't need to allocate real objects for them. Method calls also are reasonably optimized, although they don't have inline caches afaik. You're basically testing a few tiny bits of the runtime that happen to be reasonably fast. The big frameworks like rails do a lot of stuff that is horribly inefficient. Combined the aforementioned single-threadedness and slow GC leads to ruby being slow in real-world applications even if a few microbenchmarks are ok for an interpreted language.

Anecdote time! I've been hacking on a little scripting language[1] lately. To see how its performance compares, I run a few benchmarks[2] against other similar (dynamically typed, bytecode compiled) languages: Lua, LuaJIT (interpreted), Python, and Ruby. Like others, I had internalized "Ruby is slow" through osmosis. But the version of Ruby I happen to have on my machine is 2.0.0. In my little benchmarks, it turns out Ruby is one of the fastest. I haven't compared to 1.8.7, but I'm guessing that was much slower. [1] https://github.com/munificent/wren https://github.com/munificent/wren [2] https://github.com/munificent/wren/tree/master/benchmark https://github.com/munificent/wren/tree/master/benchmark If you want the gory details, here's the results of running them against Lua 5.3.2, LuaJIT 2.0.2, Python 2.7.5, and Ruby. (Yes, I should try against Python 3. I will.): score time wren score relative binary_trees - wren 3193 0.31s binary_trees - lua 1366 0.73s 233.72% binary_trees - luajit 6256 0.16s 51.04% binary_trees - python 1376 0.73s 231.96% binary_trees - ruby 3003 0.33s 106.34% fib - wren 3078 0.32s fib - lua 2785 0.36s 110.52% fib - luajit 6900 0.14s 44.62% fib - python 1331 0.75s 231.37% fib - ruby 3548 0.28s 86.77% for - wren 6080 0.16s for - lua 1990 0.08s 50.71% for - luajit 5914 0.02s 13.24% for - python 2825 0.35s 215.25% for - ruby 6595 0.15s 92.19% method_call - wren 4707 0.21s method_call - lua 1674 0.60s 281.17% method_call - luajit 4221 0.24s 111.53% method_call - python 767 1.30s 613.69% method_call - ruby 3061 0.33s 153.77% As you can see, Ruby fares quite well.

mcguire 13y ago
"dynamically typed, bytecode compiled" I haven't followed Ruby development, but the last time I heard, Ruby 1.8 and prior was a tree-walking interpreter---simple to implement but not fast at all.
- munificent 13y ago
  That's correct. I still think Ruby 1.8 would be a useful data point just because it's a widely used language that many people are familiar with, but in this case I'm comparing it to Ruby 2.0, so it is apples/apples. For what it's worth, I look at these benchmarks as validating feasibility more so than really trying to win some performance fight between languages. In order to argue that my language is suitable for real world use, I need to show it's performance is comparable to other languages that are widely used. From that angle, Ruby is a fine comparison, regardless of how it's implemented.
- srd 13y ago
  This changed in 1.9. Ruby is now compiled into a stack machine based bytecode. Ruby 1.8 used the AST internally, you're right there. This made e.g. method calls very slow compared to the new byte code representation.
- steveklabnik 13y ago
  1.8 -> 1.9 was an entire re-write, it's now a bytecode compiled language: http://www.ruby-doc.org/core-2.1.0/RubyVM/InstructionSequence.html http://www.ruby-doc.org/core-2.1.0/RubyVM/InstructionSequenc...
alberth 13y ago
>> "To see how its performance compares, I run a few benchmarks against ... LuaJIT (interpreted)" Why would you use LuaJIT in interpreted mode? That defeats the main reason for using LuaJIT, and you'll see massive improvement in speed if you run it not interpreted [1] It should also be noted that, even in the slower interpreted mode of LuaJIT - it was still the fastest in completing your benchmarks compared to all other languages/implementations. It will run even faster if you don't run LuaJIT in interpreted mode. [1] http://luajit.org/performance_x86.html http://luajit.org/performance_x86.html
- munificent 13y ago
  Running LuaJIT interpreted is a deliberate choice. Some platforms, in particular iOS and game consoles, do not allow generated machine code. I want my language to be usable for games, and that's one reason I'm using bytecode. (Simplicity is the main reason.) Given that, I think it's most helpful for me to compare Wren's performance to other bytecode language implementations. Comparing to JIT compiled languages is a bit apples/oranges. They tend to be something like an order of magnitude faster (and an order of magnitude more complex!). Meanwhile, LuaJIT's interpreted mode is very fast as you note. I think it's one of the fastest bytecode interpreters around, so it's a great target for Wren to aim for. If you want a lot more detail on my thoughts about this, see here: https://github.com/munificent/wren/blob/master/doc/site/performance.markdown https://github.com/munificent/wren/blob/master/doc/site/perf...
the8472 13y ago
What makes ruby (MRI particularly) slow are metaprogramming things that slow it down when done at runtime (dynamically definding methods and classes), its global interpreter lock (preventing proper parallelism) and the single-threaded super-slow GC. Integer math is fast because they're using tagged pointers and thus don't need to allocate real objects for them. Method calls also are reasonably optimized, although they don't have inline caches afaik. You're basically testing a few tiny bits of the runtime that happen to be reasonably fast. The big frameworks like rails do a lot of stuff that is horribly inefficient. Combined the aforementioned single-threadedness and slow GC leads to ruby being slow in real-world applications even if a few microbenchmarks are ok for an interpreted language.