3 ms·

I don't understand how people are using multithreaded ruby. Are these people doing something clever that I am not aware of? Or is it just insane amounts of test

by nahname 13y ago

I don't understand how people are using multithreaded ruby. Are these people doing something clever that I am not aware of? Or is it just insane amounts of testing? http://stackoverflow.com/questions/15184338/how-to-know-what-is-not-thread-safe-in-ruby http://stackoverflow.com/questions/15184338/how-to-know-what...

FooBarWidget 13y ago
Web stuff tends to be embarrassingly parallel: http://en.wikipedia.org/wiki/Embarrassingly_parallel http://en.wikipedia.org/wiki/Embarrassingly_parallel. Which means that it's extremely easy to make a framework that's reentrant: http://en.wikipedia.org/wiki/Reentrancy_(computing) http://en.wikipedia.org/wiki/Reentrancy_(computing). Reentrancy gives you thread-safety by default. Yeah, Ruby core data structures are not thread-safe. But that doesn't matter. You're not supposed to a lot of any data structures between requests anyway. Of the few that are shared (e.g. the Rails cache object), they are explicitly engineered to be thread-safe. There's also the story that Ruby cannot use multi-core. That's partially correct: on the MRI implementation, it doesn't matter how many threads you have, only one can be active at a time. However what you can have is multiple processes, each with multiple threads. Each process can use a different core. Furthermore, the JRuby and Rubinius implementations can handle multi-core with multithreading just fine. Both Phusion Passenger and Puma support JRuby and Rubinius. Maybe I'm just too familiar with Ruby and multithreading, but I don't understand why people have trouble with multithreaded Ruby. In my eyes it's pretty easy. If you have a library that's not thread-safe, then just don't share that library's objects between threads, or ensure that you grab a lock before using that object, and you'll be fine. In my opinion, the situation is not much different in Python, Java or C++.
- nahname 13y ago
  Being embarrassingly parallel and always being parallel are two separate things. I don't see how the risk of bugs that, generally, show themselves under load would ever make puma worth it. What is the advantage over unicorn? Slightly lower memory usage? My CPU and IO are always pegged way before memory is.
  - FooBarWidget 13y ago
    The advantages depends on the workload. For a lot of applications, i.e. applications with CPU-bound, fast-running requests, multi-process behind a buffering web server (Phusion Passenger default style, Unicorn style) is fine and works great. Don't move away from that. When it comes to this kind of workload, both Phusion Passenger (open source and Enterprise alike) and Unicorn work better than Puma. While Puma can be configured to use 1 thread, Phusion Passenger still has better management tools and fault-tolerance features than Puma, while Unicorn also has better fault-tolerance features than Puma. For applications that utilize more blocking I/O though, e.g. applications that perform a lot of HTTP calls, you need a multithreaded or an evented app server, as well as an app written to support that concurrency style. Otherwise you will run out of concurrency very soon, and your app will spend a lot of time waiting on I/O instead of doing useful work.
    - jrochkind1 13y ago
      I think that MOST web apps perform enough I/O to see huge advantages from concurrent request dispatch. Like, say, calls to the database. Most web apps do a significant amount of I/O, they are not strictly web bound. If the request-response loop spends a non-trivial amount of time waiting on I/O -- and most do -- then concurrent request dispatch has huge advantages. I tried to investigate/show that in these benchmarks: https://github.com/jrochkind/fake_work_app/blob/master/README.md https://github.com/jrochkind/fake_work_app/blob/master/READM... (And writing a ruby app to actually work right with evented/reactor-style request dispatch turns out to be way worse than multi-threading, in actual practice. http://www.slideshare.net/KyleDrake/hybrid-concurrency-patterns http://www.slideshare.net/KyleDrake/hybrid-concurrency-patte... ) (Your reference to 'blocking I/O' is confusing, I think -- that phrase means different and sometimes opposite things to different people/contexts. But it's not even neccesary to get into it for this discussion, you just mean 'I/O' at all, I think.)
      - FooBarWidget 13y ago
        Well, local I/O, and sometimes even LAN I/O, tend to behave close to non-blocking. The latency is very small, unless your database is under heavy load, but then you have other problems anyway. With blocking I/O I mean I/O calls which put the process to sleep for a while. This is in contrast to I/O calls which succeed immediately.
- dasil003 13y ago
  > Maybe I'm just too familiar with Ruby and multithreading, but I don't understand why people have trouble with multithreaded Ruby. [...] In my opinion, the situation is not much different in Python, Java or C++. I think the problem is a mix of language features and culture. Ruby does not have a history good thread-safe practices, and the language has some extremely convenient features which are death for thread-safety (eg. class instance variables). Combine that with prolific meta-programming and a lack of static analysis tools, and it can become very very difficult to be certain that a given application is thread safe. As long as all developers are well-versed and keep thread-safety front and center from the beginning of application developer then I agree it's not hard per se, but in practice that is so rarely the case that if I knew I needed heavy multi-threading for memory efficiency and CPU utilization I might disqualify Ruby on cultural reasons alone (and I say this as a full-time rubyist who knows no language better).
  - jrochkind1 13y ago
    I know what you mean and don't disagree in general.... but for a _web application_ specifically, the picture is quite a bit rosier. A web application doesn't usually share much in-memory state _between requests_ -- especially if we're talking about app-specific code and not framework code. If it does (modify class variables or other global state, etc) -- you've got to eliminate that or make it threadsafe, but that's not _that_ hard to identify. A web application is very rarely going to be doing class-modifying meta-programming _as part of request handling_ (as opposed to on boot, where it won't be a problem). Your actual app-specific code in a web application is highly likely to be MT-request-dispatch safe already, and if not is not too hard to get it there -- and worth the effort because of the extreme throughput increases you can get with an MT-request-dispatch app server. Now, what can definitely be trickier is framework and gem code. ActiveRecord, historically, has had quite a few problems with thread-safety under MT-request-dispatch, off and on (the database connections themselves are the shared state it's tricky to deal with, among other things). But it's gotten a LOT better and should be fairly robust now. I know what you mean about 'cultural reasons', but I think the ruby culture has been gradually changing for a while (jruby has a lot to do with it), and some of us hope is becoming more MT-friendly. But I'm not saying it's trivial or guaranteed problem free, but the potential gains are worth it. (I do run a rails app that uses multiple threads and ActiveRecord.)

by nahname 13y ago

FooBarWidget 13y ago

Web stuff tends to be embarrassingly parallel: http://en.wikipedia.org/wiki/Embarrassingly_parallel http://en.wikipedia.org/wiki/Embarrassingly_parallel. Which means that it's extremely easy to make a framework that's reentrant: http://en.wikipedia.org/wiki/Reentrancy_(computing) http://en.wikipedia.org/wiki/Reentrancy_(computing). Reentrancy gives you thread-safety by default. Yeah, Ruby core data structures are not thread-safe. But that doesn't matter. You're not supposed to a lot of any data structures between requests anyway. Of the few that are shared (e.g. the Rails cache object), they are explicitly engineered to be thread-safe. There's also the story that Ruby cannot use multi-core. That's partially correct: on the MRI implementation, it doesn't matter how many threads you have, only one can be active at a time. However what you can have is multiple processes, each with multiple threads. Each process can use a different core. Furthermore, the JRuby and Rubinius implementations can handle multi-core with multithreading just fine. Both Phusion Passenger and Puma support JRuby and Rubinius. Maybe I'm just too familiar with Ruby and multithreading, but I don't understand why people have trouble with multithreaded Ruby. In my eyes it's pretty easy. If you have a library that's not thread-safe, then just don't share that library's objects between threads, or ensure that you grab a lock before using that object, and you'll be fine. In my opinion, the situation is not much different in Python, Java or C++.

nahname 13y ago
Being embarrassingly parallel and always being parallel are two separate things. I don't see how the risk of bugs that, generally, show themselves under load would ever make puma worth it. What is the advantage over unicorn? Slightly lower memory usage? My CPU and IO are always pegged way before memory is.
- FooBarWidget 13y ago
  The advantages depends on the workload. For a lot of applications, i.e. applications with CPU-bound, fast-running requests, multi-process behind a buffering web server (Phusion Passenger default style, Unicorn style) is fine and works great. Don't move away from that. When it comes to this kind of workload, both Phusion Passenger (open source and Enterprise alike) and Unicorn work better than Puma. While Puma can be configured to use 1 thread, Phusion Passenger still has better management tools and fault-tolerance features than Puma, while Unicorn also has better fault-tolerance features than Puma. For applications that utilize more blocking I/O though, e.g. applications that perform a lot of HTTP calls, you need a multithreaded or an evented app server, as well as an app written to support that concurrency style. Otherwise you will run out of concurrency very soon, and your app will spend a lot of time waiting on I/O instead of doing useful work.
  - jrochkind1 13y ago
    I think that MOST web apps perform enough I/O to see huge advantages from concurrent request dispatch. Like, say, calls to the database. Most web apps do a significant amount of I/O, they are not strictly web bound. If the request-response loop spends a non-trivial amount of time waiting on I/O -- and most do -- then concurrent request dispatch has huge advantages. I tried to investigate/show that in these benchmarks: https://github.com/jrochkind/fake_work_app/blob/master/README.md https://github.com/jrochkind/fake_work_app/blob/master/READM... (And writing a ruby app to actually work right with evented/reactor-style request dispatch turns out to be way worse than multi-threading, in actual practice. http://www.slideshare.net/KyleDrake/hybrid-concurrency-patterns http://www.slideshare.net/KyleDrake/hybrid-concurrency-patte... ) (Your reference to 'blocking I/O' is confusing, I think -- that phrase means different and sometimes opposite things to different people/contexts. But it's not even neccesary to get into it for this discussion, you just mean 'I/O' at all, I think.)
    - FooBarWidget 13y ago
      Well, local I/O, and sometimes even LAN I/O, tend to behave close to non-blocking. The latency is very small, unless your database is under heavy load, but then you have other problems anyway. With blocking I/O I mean I/O calls which put the process to sleep for a while. This is in contrast to I/O calls which succeed immediately.
dasil003 13y ago
> Maybe I'm just too familiar with Ruby and multithreading, but I don't understand why people have trouble with multithreaded Ruby. [...] In my opinion, the situation is not much different in Python, Java or C++. I think the problem is a mix of language features and culture. Ruby does not have a history good thread-safe practices, and the language has some extremely convenient features which are death for thread-safety (eg. class instance variables). Combine that with prolific meta-programming and a lack of static analysis tools, and it can become very very difficult to be certain that a given application is thread safe. As long as all developers are well-versed and keep thread-safety front and center from the beginning of application developer then I agree it's not hard per se, but in practice that is so rarely the case that if I knew I needed heavy multi-threading for memory efficiency and CPU utilization I might disqualify Ruby on cultural reasons alone (and I say this as a full-time rubyist who knows no language better).
- jrochkind1 13y ago
  I know what you mean and don't disagree in general.... but for a _web application_ specifically, the picture is quite a bit rosier. A web application doesn't usually share much in-memory state _between requests_ -- especially if we're talking about app-specific code and not framework code. If it does (modify class variables or other global state, etc) -- you've got to eliminate that or make it threadsafe, but that's not _that_ hard to identify. A web application is very rarely going to be doing class-modifying meta-programming _as part of request handling_ (as opposed to on boot, where it won't be a problem). Your actual app-specific code in a web application is highly likely to be MT-request-dispatch safe already, and if not is not too hard to get it there -- and worth the effort because of the extreme throughput increases you can get with an MT-request-dispatch app server. Now, what can definitely be trickier is framework and gem code. ActiveRecord, historically, has had quite a few problems with thread-safety under MT-request-dispatch, off and on (the database connections themselves are the shared state it's tricky to deal with, among other things). But it's gotten a LOT better and should be fairly robust now. I know what you mean about 'cultural reasons', but I think the ruby culture has been gradually changing for a while (jruby has a lot to do with it), and some of us hope is becoming more MT-friendly. But I'm not saying it's trivial or guaranteed problem free, but the potential gains are worth it. (I do run a rails app that uses multiple threads and ActiveRecord.)