3 ms·

Is that really parallel? I thought Ruby had a GIL.

by clubhi 12y ago

ad_hominem 12y ago
MRI has a GIL, but the GIL doesn't block on I/O like HTTP requests (but you'll still be pegged to one core unless you do process forking yourself). JRuby and Rubinius use native threads so you can use all cores with just Thread.new.
- Arnor 12y ago
  > JRuby and Rubinius use native threads so you can use all cores with just Thread.new. Almost. JRuby uses JVM threads so the actual threading model depends on the underline JVM.
  - rubiquity 12y ago
    What versions of the JVM don't map JVM threads directly to Os threads? I thought green threading was removed a long time ago. I would love more info, thank you.
- krisdol 12y ago
  Green threads were removed from ruby when 1.9 was released, and we're almost at 2.2 now. Is there some other factor limiting you to one core? Edit: I just want to say I've done some more reading on the subject and have learned that the GIL does in fact prevent a single process from running on multiple cores. That said, if you wrote a thread-safe ruby application, then just fork it n-1 times (where n = number of cores) and everything should be fine.
pizza234 12y ago
The GIL doesn't always kick in! Here's a nice post with a couple of examples: http://ablogaboutcode.com/2012/02/06/the-ruby-global-interpreter-lock/ http://ablogaboutcode.com/2012/02/06/the-ruby-global-interpr...
bigdubs 12y ago
i re-wrote it to just loop over the urls and fetch, results below. >> time ruby parallel.rb http://google.com http://google.com: 301 Moved Permanently http://grantland.com http://grantland.com: 200 OK http://espn.go.com http://espn.go.com: 200 OK http://www.cnn.com http://www.cnn.com: 200 OK real 0m0.595s user 0m0.099s sys 0m0.045s ~/code/ruby >> time ruby serial.rb http://www.cnn.com http://www.cnn.com: 200 OK http://espn.go.com http://espn.go.com: 200 OK http://grantland.com http://grantland.com: 200 OK http://google.com http://google.com: 301 Moved Permanently real 0m1.495s user 0m0.116s sys 0m0.053s
masklinn 12y ago
The GIL is there to protect the interpreter's internal data structures, so it's only necessary to hold it while running the interpreter (while executing ruby code) or manipulating interpreter internals. This is all IO, there's no interpreter execution while you're waiting for an HTTP request to come back so the GIL is released before starting IO, and re-acquired after the request comes back (before resuming execution), same when executing native code which doesn't interact with the interpreter (e.g. image processing implemented in C). The GIL is an issue when you have multiple interpreted (not native) CPU-bound threads (and depending on the exact strategy it may also be an issue when there's an interpreted CPU-bound thread and multiple IO-bound threads, CPython 3's "new GIL" has/had that problem) Also MRI has a GIL, GILs are implementation details (with consequences, but still) and other implementations may or may not use a GIL.
krisdol 12y ago
It definitely is. I've benchmarked similar solutions where the time it took to perform 1000+ POSTs to a REST endpoint was cut in half with every new thread up to about 6 threads. The interpreter performance for any language is going to be a much smaller impediment than the speed of the network.

Is that really parallel? I thought Ruby had a GIL.

by clubhi 12y ago

Is that really parallel? I thought Ruby had a GIL.

MRI has a GIL, but the GIL doesn't block on I/O like HTTP requests (but you'll still be pegged to one core unless you do process forking yourself). JRuby and Rubinius use native threads so you can use all cores with just Thread.new.

Arnor 12y ago
> JRuby and Rubinius use native threads so you can use all cores with just Thread.new. Almost. JRuby uses JVM threads so the actual threading model depends on the underline JVM.
- rubiquity 12y ago
  What versions of the JVM don't map JVM threads directly to Os threads? I thought green threading was removed a long time ago. I would love more info, thank you.
krisdol 12y ago
Green threads were removed from ruby when 1.9 was released, and we're almost at 2.2 now. Is there some other factor limiting you to one core? Edit: I just want to say I've done some more reading on the subject and have learned that the GIL does in fact prevent a single process from running on multiple cores. That said, if you wrote a thread-safe ruby application, then just fork it n-1 times (where n = number of cores) and everything should be fine.

pizza234 12y ago

The GIL doesn't always kick in! Here's a nice post with a couple of examples: http://ablogaboutcode.com/2012/02/06/the-ruby-global-interpreter-lock/ http://ablogaboutcode.com/2012/02/06/the-ruby-global-interpr...

bigdubs 12y ago

i re-wrote it to just loop over the urls and fetch, results below. >> time ruby parallel.rb http://google.com http://google.com: 301 Moved Permanently http://grantland.com http://grantland.com: 200 OK http://espn.go.com http://espn.go.com: 200 OK http://www.cnn.com http://www.cnn.com: 200 OK real 0m0.595s user 0m0.099s sys 0m0.045s ~/code/ruby >> time ruby serial.rb http://www.cnn.com http://www.cnn.com: 200 OK http://espn.go.com http://espn.go.com: 200 OK http://grantland.com http://grantland.com: 200 OK http://google.com http://google.com: 301 Moved Permanently real 0m1.495s user 0m0.116s sys 0m0.053s

masklinn 12y ago

The GIL is there to protect the interpreter's internal data structures, so it's only necessary to hold it while running the interpreter (while executing ruby code) or manipulating interpreter internals. This is all IO, there's no interpreter execution while you're waiting for an HTTP request to come back so the GIL is released before starting IO, and re-acquired after the request comes back (before resuming execution), same when executing native code which doesn't interact with the interpreter (e.g. image processing implemented in C). The GIL is an issue when you have multiple interpreted (not native) CPU-bound threads (and depending on the exact strategy it may also be an issue when there's an interpreted CPU-bound thread and multiple IO-bound threads, CPython 3's "new GIL" has/had that problem) Also MRI has a GIL, GILs are implementation details (with consequences, but still) and other implementations may or may not use a GIL.

krisdol 12y ago

It definitely is. I've benchmarked similar solutions where the time it took to perform 1000+ POSTs to a REST endpoint was cut in half with every new thread up to about 6 threads. The interpreter performance for any language is going to be a much smaller impediment than the speed of the network.