5 ms·

Threads (in Ruby): Enough Already

116 points by wycats 16y ago

Tichy 16y ago
Somehow the takeaway from all these posts seems to be to stick to JRuby. Am I reading that right?
- wycats 16y ago
  If you use C Ruby, you'll need one Ruby process per core, managed by something like Passenger (mod_rails) or Unicorn. If you use JRuby, you'll need one Ruby process per machine (for N cores), managed by the JVM. For boxes with a lot of cores, JRuby's larger memory footprint is overtaken by the ability to share that memory across a number of cores. This story is also somewhat complicated by Ruby Enterprise Edition, which adds copy-on-write semantics to Ruby's GC, and is built by the same guys as Passenger, making it possible to share SOME memory between processes. With all that said, we're really talking about marginal amounts of RAM. The real takeaway is that if you're running 6 processes per core (very common), you're doing something very wrong. FYI: At some point in the future (1.2?), Rubinius will also be able to run a single Ruby process per machine.
  - evanphx 16y ago
    Rubinius wise, I'm actively working on making in concurrent. Yes, a future release will incorporate that work, which allows one Rubinius process to fully use as many cores as a machine has.
    - extension 16y ago
      Concurrent GC too?
      - evanphx 16y ago
        Concurrent GC is not currently planned, but it should be noted that our generational GC does wonders to reduce GC pause time anyway. Additionally, even Hotspot typically defaults to their stop-the-world GC. This is because a concurrent GC typically spreads part of the GC time around, ie, performance is slower to reduced GC pause time. But even a concurrent GC typically has to stop all threads at some point to get everything consistent.
  - sandofsky 16y ago
    You will get some benefit from having more processes than cores, though perhaps not six. Ruby's "stop the world" garbage collector means a process will completely pause during GC, including all threads. You may be better off having twice as many processes at half the ulimit. Fewer threads will be paused at once, and there are fewer objects the VM has to traverse during a GC run. It's something to investigate when you tune your app. (This mostly applies to 1.8. I haven't investigated 1.9.)
  - pl00nk 16y ago
    which do you generally prefer?
ericb 16y ago
Ok, dumb question--I'm a bit naive on process forking. On Linux with Ruby, if I fork a process, must the new process stay on the same core? Forked processes begin by sharing memory initially in Ruby, correct? Can anyone enlighten me on the mechanics of forking and if there are scenarios where it helps with utilizing multiple cores (maybe relatively independent workers that just read shared settings?). Forking seems...a little weird to me coming from my previous Windows background.
- amock 16y ago
  The process will be scheduled by the OS just like any other process. I think fork() is similar to CreateProcess() on Windows, but I haven't done any Windows programming that needed to create more processes.
  - okmjuhb 16y ago
    CreateProcess can't implement fork() semantics. To my knowledge there's no good way to fork() on Windows.
    - koenigdavidmj 16y ago
      Section 5.6 of http://www.redhat.com/support/wpapers/cygnus/cygnus_cygwin/architecture.html http://www.redhat.com/support/wpapers/cygnus/cygnus_cygwin/a... is an interesting read about how the Cygwin team implemented fork.
- evanphx 16y ago
  The linux kernel will schedule the new process on whatever core it wants. Forked process in ruby have posix semantics, ie, NOT shared memory. Linux uses Copy-On-Write pages to conserve the amount of memory copying when the new process is created.
- wycats 16y ago
  Forking creates a new process, initially "sharing" memory via copy-on-write semantics (and therefore being eligible to run on another core). However, C Ruby's default mark-sweep garbage collector touches pretty much all memory immediately, eliminating the typical memory savings of a forking model. As I commented earlier, Ruby Enterprise Edition (made by the same guys as Passenger) ships an alternate GC that doesn't actually write to the memory space of the original objects in order to mark them. As a result, some memory (but not necessarily as much as you'd expect) can be shared between processes forked from the same parent and running on multiple cores. Sort of a middle ground between totally separate processes and shared memory via non-GIL'ed threads.
deleted 16y ago
[deleted]
tptacek 16y ago
This post is a useful rundown on where threads stand in Ruby now, but it hangs on a bit of a straw man argument. The argument for evented concurrency assumes threads work. Async design isn't a reaction to MRI's crappy green threads. It's also not a very compelling argument for threading to say "web programs don't share a lot of state, so you don't have to worry about synchronization". If all you do are CRUD apps, you can indeed punt scaling to the database. That doesn't mean threads are more effective than events; it means you made concurrency and synchronization someone else's problem. There's nothing wrong with that, but it's not a convincing demonstration of threading.
- Nwallins 16y ago
  > The argument for evented concurrency assumes threads work. Async design isn't a reaction to MRI's crappy green threads. Truly? To my understanding, event loops require inversion of control (and likely callbacks and broken exception handling). This is a large cost that requires a benefit to be worth it. I understand that benefit to be: you don't have to deal with threads (or bad implementations of such). This blog post comes to mind: http://www.unlimitednovelty.com/2010/08/multithreaded-rails-is-generally-better.html http://www.unlimitednovelty.com/2010/08/multithreaded-rails-...
  - tptacek 16y ago
    Not being a patterns guy or a Fowlerite, I can't tell you whether async "requires inversion of control". Apropos nothing, I'm also unlikely to recognize "dependency injection" when I see it, as my first language was C, not Java. Async code isn't "likely" to require callbacks; it will almost certainly involve callbacks, those being a central design feature of async code. You should play with the idea for a bit before forming an opinion about it. I don't know what you mean by "broken exception handling". This may be a Rails-ism I'm unfamiliar with. I'm very familiar with Rails (we ship a fairly large product built on it), but I've never tried to shoehorn async I/O into it; like Twitter and presumably every other large site, we do Rails on the front-end/UI and fast things on the backend, in our case with EventMachine. Like I said: I'm not arguing that Rails threading is bad, or even that Rails should have better async support. If I cared that much about the performance of my front end, I probably wouldn't be serving requests directly off Rails. Rails developers may very well be better off with threads. But that fact has little to do with the merits of threading and async as concepts.
samstokes 16y ago
This article is a good wake-up call, but even if Rails itself is now (since 2.2) thread-safe, I need to be fairly sure that all my libraries are before I can enable threaded request dispatch in my Rails app. That's a lot of code inspection / thorough regression testing / inevitable bugs. The Brightbox team tried to tackle the related problem for moving to Ruby 1.9 with a crowdsourced library review site: http://isitruby19.com/ http://isitruby19.com/ Does anyone know of a similar site where people can report whether libraries are thread safe? (The code for Is It Ruby 1.9 is available on github - http://github.com/brightbox/isitruby19 http://github.com/brightbox/isitruby19 - so it would be easy enough to set up a clone, but I can't commit to maintaining such a thing.)
- wycats 16y ago
  The Rails Plugins site (http://railsplugins.org http://railsplugins.org) that we built at Engine Yard has 406 registered plugins, and a criteria for thread safety (http://railsplugins.org/plugins?criteria[]=2 http://railsplugins.org/plugins?criteria[]=2). The way it works is that the author can specify whether or not he thinks it's threadsafe (yes/no/maybe), which is then verified by users who can specify whether they agree. If a user marks a plugin as not-threadsafe (or not-Rails 3, or not-JRuby, or not-Ruby 1.9), the author has 7 days to help the user come around before it sticks. So far, there are 60 plugins marked as threadsafe (which means that either the author said "yes" and nobody disagreed, or the author said "maybe" and all the votes so far say yes).
  - samstokes 16y ago
    That's awesome, and exactly what I was looking for. Thanks! I found that the postgres gem, the most immediately important for me, isn't listed. Do I need to be the gem author/maintainer to register a gem, or can I just add it?
    - mml 16y ago
      Wycats noted in the comments of the article that the pg driver is indeed safe for use in rails threaded mode. As a psql fan, it makes me wonder what the rails world would be like had dhh had picked psql from the start.
deleted 16y ago
[deleted]

116 points by wycats 16y ago

Tichy 16y ago

Somehow the takeaway from all these posts seems to be to stick to JRuby. Am I reading that right?

wycats 16y ago
If you use C Ruby, you'll need one Ruby process per core, managed by something like Passenger (mod_rails) or Unicorn. If you use JRuby, you'll need one Ruby process per machine (for N cores), managed by the JVM. For boxes with a lot of cores, JRuby's larger memory footprint is overtaken by the ability to share that memory across a number of cores. This story is also somewhat complicated by Ruby Enterprise Edition, which adds copy-on-write semantics to Ruby's GC, and is built by the same guys as Passenger, making it possible to share SOME memory between processes. With all that said, we're really talking about marginal amounts of RAM. The real takeaway is that if you're running 6 processes per core (very common), you're doing something very wrong. FYI: At some point in the future (1.2?), Rubinius will also be able to run a single Ruby process per machine.
- evanphx 16y ago
  Rubinius wise, I'm actively working on making in concurrent. Yes, a future release will incorporate that work, which allows one Rubinius process to fully use as many cores as a machine has.
  - extension 16y ago
    Concurrent GC too?
    - evanphx 16y ago
      Concurrent GC is not currently planned, but it should be noted that our generational GC does wonders to reduce GC pause time anyway. Additionally, even Hotspot typically defaults to their stop-the-world GC. This is because a concurrent GC typically spreads part of the GC time around, ie, performance is slower to reduced GC pause time. But even a concurrent GC typically has to stop all threads at some point to get everything consistent.
- sandofsky 16y ago
  You will get some benefit from having more processes than cores, though perhaps not six. Ruby's "stop the world" garbage collector means a process will completely pause during GC, including all threads. You may be better off having twice as many processes at half the ulimit. Fewer threads will be paused at once, and there are fewer objects the VM has to traverse during a GC run. It's something to investigate when you tune your app. (This mostly applies to 1.8. I haven't investigated 1.9.)
- pl00nk 16y ago
  which do you generally prefer?

ericb 16y ago

Ok, dumb question--I'm a bit naive on process forking. On Linux with Ruby, if I fork a process, must the new process stay on the same core? Forked processes begin by sharing memory initially in Ruby, correct? Can anyone enlighten me on the mechanics of forking and if there are scenarios where it helps with utilizing multiple cores (maybe relatively independent workers that just read shared settings?). Forking seems...a little weird to me coming from my previous Windows background.

amock 16y ago
The process will be scheduled by the OS just like any other process. I think fork() is similar to CreateProcess() on Windows, but I haven't done any Windows programming that needed to create more processes.
- okmjuhb 16y ago
  CreateProcess can't implement fork() semantics. To my knowledge there's no good way to fork() on Windows.
  - koenigdavidmj 16y ago
    Section 5.6 of http://www.redhat.com/support/wpapers/cygnus/cygnus_cygwin/architecture.html http://www.redhat.com/support/wpapers/cygnus/cygnus_cygwin/a... is an interesting read about how the Cygwin team implemented fork.
evanphx 16y ago
The linux kernel will schedule the new process on whatever core it wants. Forked process in ruby have posix semantics, ie, NOT shared memory. Linux uses Copy-On-Write pages to conserve the amount of memory copying when the new process is created.
wycats 16y ago
Forking creates a new process, initially "sharing" memory via copy-on-write semantics (and therefore being eligible to run on another core). However, C Ruby's default mark-sweep garbage collector touches pretty much all memory immediately, eliminating the typical memory savings of a forking model. As I commented earlier, Ruby Enterprise Edition (made by the same guys as Passenger) ships an alternate GC that doesn't actually write to the memory space of the original objects in order to mark them. As a result, some memory (but not necessarily as much as you'd expect) can be shared between processes forked from the same parent and running on multiple cores. Sort of a middle ground between totally separate processes and shared memory via non-GIL'ed threads.

deleted 16y ago

[deleted]

tptacek 16y ago

This post is a useful rundown on where threads stand in Ruby now, but it hangs on a bit of a straw man argument. The argument for evented concurrency assumes threads work. Async design isn't a reaction to MRI's crappy green threads. It's also not a very compelling argument for threading to say "web programs don't share a lot of state, so you don't have to worry about synchronization". If all you do are CRUD apps, you can indeed punt scaling to the database. That doesn't mean threads are more effective than events; it means you made concurrency and synchronization someone else's problem. There's nothing wrong with that, but it's not a convincing demonstration of threading.

Nwallins 16y ago
> The argument for evented concurrency assumes threads work. Async design isn't a reaction to MRI's crappy green threads. Truly? To my understanding, event loops require inversion of control (and likely callbacks and broken exception handling). This is a large cost that requires a benefit to be worth it. I understand that benefit to be: you don't have to deal with threads (or bad implementations of such). This blog post comes to mind: http://www.unlimitednovelty.com/2010/08/multithreaded-rails-is-generally-better.html http://www.unlimitednovelty.com/2010/08/multithreaded-rails-...
- tptacek 16y ago
  Not being a patterns guy or a Fowlerite, I can't tell you whether async "requires inversion of control". Apropos nothing, I'm also unlikely to recognize "dependency injection" when I see it, as my first language was C, not Java. Async code isn't "likely" to require callbacks; it will almost certainly involve callbacks, those being a central design feature of async code. You should play with the idea for a bit before forming an opinion about it. I don't know what you mean by "broken exception handling". This may be a Rails-ism I'm unfamiliar with. I'm very familiar with Rails (we ship a fairly large product built on it), but I've never tried to shoehorn async I/O into it; like Twitter and presumably every other large site, we do Rails on the front-end/UI and fast things on the backend, in our case with EventMachine. Like I said: I'm not arguing that Rails threading is bad, or even that Rails should have better async support. If I cared that much about the performance of my front end, I probably wouldn't be serving requests directly off Rails. Rails developers may very well be better off with threads. But that fact has little to do with the merits of threading and async as concepts.

samstokes 16y ago

This article is a good wake-up call, but even if Rails itself is now (since 2.2) thread-safe, I need to be fairly sure that all my libraries are before I can enable threaded request dispatch in my Rails app. That's a lot of code inspection / thorough regression testing / inevitable bugs. The Brightbox team tried to tackle the related problem for moving to Ruby 1.9 with a crowdsourced library review site: http://isitruby19.com/ http://isitruby19.com/ Does anyone know of a similar site where people can report whether libraries are thread safe? (The code for Is It Ruby 1.9 is available on github - http://github.com/brightbox/isitruby19 http://github.com/brightbox/isitruby19 - so it would be easy enough to set up a clone, but I can't commit to maintaining such a thing.)

wycats 16y ago
The Rails Plugins site (http://railsplugins.org http://railsplugins.org) that we built at Engine Yard has 406 registered plugins, and a criteria for thread safety (http://railsplugins.org/plugins?criteria[]=2 http://railsplugins.org/plugins?criteria[]=2). The way it works is that the author can specify whether or not he thinks it's threadsafe (yes/no/maybe), which is then verified by users who can specify whether they agree. If a user marks a plugin as not-threadsafe (or not-Rails 3, or not-JRuby, or not-Ruby 1.9), the author has 7 days to help the user come around before it sticks. So far, there are 60 plugins marked as threadsafe (which means that either the author said "yes" and nobody disagreed, or the author said "maybe" and all the votes so far say yes).
- samstokes 16y ago
  That's awesome, and exactly what I was looking for. Thanks! I found that the postgres gem, the most immediately important for me, isn't listed. Do I need to be the gem author/maintainer to register a gem, or can I just add it?
  - mml 16y ago
    Wycats noted in the comments of the article that the pg driver is indeed safe for use in rails threaded mode. As a psql fan, it makes me wonder what the rails world would be like had dhh had picked psql from the start.

deleted 16y ago

[deleted]