9 ms·

Pitchfork: Rack HTTP server for shared-nothing architecture

137 points by bkudria 4y ago

jbverschoor 4y ago
How does it compare to puma?
- aeontech 4y ago
  From the Reddit thread linked above (lots of context there, worth reading), the author answers: > what's the main benefit? Drastically reduced memory usage by improving Copy-on-Write performance. For mid-sized apps it can even use less memory than puma depending on how you set it up. See this synthetic benchmark. > Why did you decide to build reforking into a fork of Unicorn instead of contributing it to Puma? Mostly for simplicity. You can build this in Puma, but there are a few extra challenges. For instance you don't want to refork when another thread is currently processing a request, you'd risk leaving some global resource in a corrupted state. So you'd first need to stop accepting traffic and wait for all requests to complete. The problem is, Puma doesn't have a request timeout, so if the request never complete, what do you do? Lots of small challenges like that. But I think it would be awesome if Puma was to take back that idea. Puma's fork_worker feature was a big part of the inspiration after all, and I'd even be happy to help Nate or someone else do it.
- wgjordan 4y ago
  Puma is a multi-thread, multi-process Rack HTTP server that implements a request buffer. By comparison, Unicorn (the project upon which Pitchfork is based) is a single-thread, multi-process Rack HTTP server that does not buffer requests, so it's only designed for fast clients (or it needs to be paired with a proxy like Nginx to handle slow clients). The narrower design of Unicorn results in a simpler, less-flexible potentially more efficient architecture. The reforking feature that Pitchfork introduces to Unicorn was originally implemented in Puma [1], though they're controlled differently and the underlying implementations are entirely different. [1] https://github.com/puma/puma/blob/master/5.0-Upgrade.md#fork_worker https://github.com/puma/puma/blob/master/5.0-Upgrade.md#fork...
aeontech 4y ago
Looks very promising!
tiffanyh 4y ago
I’m still surprised that we haven’t seen much larger improvements in Ruby performance over the last decade given the large number of major tech companies using Rails. Yes, I'm aware of 3x3 but for the most common usage of Ruby which if for Rails apps - there hasn't been anything like the gains PHP saw from 5.x to 7.x. Especially from Microsoft, who has deep compiler/language expertise, and who owns GitHub (large Rails app).
- chucke 4y ago
  Not following what you mean. This server is from a shopify employer (major enough?). This server makes quite impressive improvements in terms of memory usage. Moreover, shopify is contributing a production grade JIT to the CRuby runtime.
- sosodev 4y ago
  There have been plenty of major performance improvements for Ruby but most of them require changes that just aren't feasible for a large codebase. For example, there are alternative Ruby interpreters that are far faster than the MRI but they're not usually a drop in replacement because they don't have full Gem or really general ecosystem compatibility. Also, some performance improvements like JIT just don't matter that much for Rails because the primary problem isn't single thread performance it's the lack of cheap parallelism.
- flavorjones 4y ago
  Howdy, I'm on the Shopify team that is working on both pitchfork and a few different performance-improvement projects for Ruby. There's a ton of activity around Ruby performance right now! I think we're entering a period of increased experimentation and rapid evolution as demonstrated by projects like YJIT[1][2], improved inline caching[3][4] and Object Shapes[5] (also used by V8), and variable-width allocation[6][7], and smaller improvements like better constant invalidation[8]. Significant investments in TruffleRuby[9] are still going on by Oracle, Shopify, and other companies. And recently, Takashi Kokubun gave a talk at Ruby Kaigi about the future of JIT compilers in Ruby that gives a peek at a whole new set of optimizations Ruby can work on (as well as some performance comparisons against other interpreted languages)[10]. You may be surprised to see how well Ruby (with the JIT enabled) performs compared to Python 3. All of which is to say, I think there's quite a bit of performance improvement being made in recent Rubies, and that trend will likely continue for quite some time. update And I forgot to mention that some very notable computer science researchers and their teams are working in the Ruby community now![11] [1]: https://news.ycombinator.com/item?id=28938446 https://news.ycombinator.com/item?id=28938446) [2]: https://speed.yjit.org/ https://speed.yjit.org/ [3]: https://bugs.ruby-lang.org/issues/18943 https://bugs.ruby-lang.org/issues/18943 [4]: https://bugs.ruby-lang.org/issues/18875 https://bugs.ruby-lang.org/issues/18875 [5]: https://bugs.ruby-lang.org/issues/18776 https://bugs.ruby-lang.org/issues/18776 [6]: https://bugs.ruby-lang.org/issues/18045 https://bugs.ruby-lang.org/issues/18045 [7]: https://bugs.ruby-lang.org/issues/18634 https://bugs.ruby-lang.org/issues/18634 [8]: https://bugs.ruby-lang.org/issues/18589 https://bugs.ruby-lang.org/issues/18589 [9]: https://eregon.me/blog/2022/01/06/benchmarking-cruby-mjit-yjit-jruby-truffleruby.html https://eregon.me/blog/2022/01/06/benchmarking-cruby-mjit-yj... [10]: https://speakerdeck.com/k0kubun/rubykaigi-2022 https://speakerdeck.com/k0kubun/rubykaigi-2022 [11]: https://shopify.engineering/shopify-ruby-at-scale-research-investment https://shopify.engineering/shopify-ruby-at-scale-research-i...
joshmn 4y ago
From r/ruby (including discussion from author): https://old.reddit.com/r/ruby/comments/xwcvty/a_new_web_server_github_shopifypitchfork/ https://old.reddit.com/r/ruby/comments/xwcvty/a_new_web_serv...
a-dub 4y ago
everything old is new again. this was the concurrency model for pre 2.0 apache. (i think in 2.0 you could pick)
- pedrocr 4y ago
  That was only forking the apache process. This is forking the whole ruby app that also happens to speak HTTP. The gain is in the app side not in the best way to serve HTTP where this is probably a loss over threading or event based solutions.
  - a-dub 4y ago
    many serious applications back then built by developers who cared about performance and latency made use of mod_perl. it put the entire perl interpreter (which then loaded perl modules that defined the application) into the preforked apache httpd processes. often times you'd put a proxy in front, serving static assets with a lighter weight http so that slow loads wouldn't tax limited server capacity for big httpd processes that included full perl interpreters and all application code loaded. so it seems it's almost exactly the same?
    - pedrocr 4y ago
      It's very similar to that setup. I don't think that was particularly common though and probably the reason prefork went away. Particularly as apache and later nginx were used as the user facing webservers proxying to something else inside (first with fastcgi and latter just http). The preforking advantage then moved to that server inside which is what this is. Nginx or something similar will still be running in front of this with a threading or event model instead of forking. So this is not going back to an old idea as it never really left, it just left apache because no one ran it as an application server and there are better ways to run an http frontend. Preforking is also not what's new here, this is just an optimization over the preforking model in unicorn which is over a decade old.
      - a-dub 4y ago
skrtskrt 4y ago
I have now worked at several significantly-sized companies that ripped Ruby out of everything as they scaled, usually replaced with Go, though none as large as Shopify. I wonder what that financial calculus looks like between these options for a large organization: 1. trying to basically reinvent the workings of an entire programming language and migrate your Ruby apps to these new tools/runtimes/typecheckers/whatever 2. incrementally rewriting critical paths to something more performant (Go/Rust) & just throwing more servers at the Ruby stuff you haven't managed to replace yet.
- tiffanyh 4y ago
  Probably not a financial decision. Tobias (Founder/CEO) was an early Rails core contributor. So Ruby on Rails is in Shopify DNA.
- jaxrtech 4y ago
  A lot of the time, I ask myself, "why don't we seem to have a tool with robust source-to-source transformation?" You could call it "cross-language refactoring". I wouldn't be surprised if someone with the code-base the size of Google has some internal tool. Maybe this is just wishful thinking. The closest thing I've seen is in academic research, e.g. [1]. [1]: Koppel, Solar-Lezama - Incremental parametric syntax for multi-language transformation (2017) - https://dl.acm.org/doi/10.1145/3135932.3135940 https://dl.acm.org/doi/10.1145/3135932.3135940
  - pedrocr 4y ago
    There are some tools specific to some transformations at least: https://c2rust.com/ https://c2rust.com/
  - dgl 4y ago
    I mean Google did essentially do this: https://opensource.googleblog.com/2017/01/grumpy-go-running-python.html https://opensource.googleblog.com/2017/01/grumpy-go-running-... Except without actually solving the problem of making the translation nice and readable. But this kind of thing at least gives you options for changing language.
tomcam 4y ago
What is Rack? Hard to search for
- _t4za 4y ago
  https://github.com/rack/rack https://github.com/rack/rack
  - tomcam 4y ago
    Thank you!
- deleted 4y ago
  [deleted]
compumike 4y ago
For those of you scaling Ruby on Rails: are you more constrained by memory or CPU consumption? If you could choose a 50% reduction in process memory footprint, versus a 50% reduction in CPU cycles to serve your average request, which would you pick?
- treeman79 4y ago
  Database locks is usual issue. Avoidable if planned better. Memory is second. In 15 years CPU has never been a real concern.
  - Puts 4y ago
    Second this. After spending several years of full time just troubleshooting others web apps I can say that not only are database locks in 99.99% of the cases the real bottleneck but also that developers in general are quite bad at managing databases and will try to micro optimizing just about anything before looking at their DB queries.
    - brigandish 4y ago
      Most don't know how to improve their DB queries, it's like a black box to them, so they look at the parts they do know. Comfort level is king.
  - jamesfinlayson 4y ago
    Yep agreed - in my experience slow queries eventually cause the web server to back up.
  - KronisLV 4y ago
    > Database locks is usual issue. Avoidable if planned better. I've often seen problems because of N+1 querying in particular, but maybe that's just the majority of codebases that I've seen. > Memory is second. In 15 years CPU has never been a real concern. In most stacks that I've worked in, memory seems to be the main cause of problems: be it a Ruby application, a PHP one, Python one or Java. Though perhaps that's because larger servers are expensive and development environments in particular tend to be on the more conservative side of things. Either that, or the apps that I've worked with are good examples of Wirth's law and refuse to even run with less than 2 GB of RAM (though maybe that's more along the lines of commentary about enterprise Java apps in particular). When your development/test server has something like 8 GB of RAM or your computer has 16 GB of RAM and you need to run 7 or 8 of those apps (as well as IDE instances), you run into problems. CPU is generally not an issue for most applications, though various databases and data stores instead. Perhaps it's batch migrations or just lots of in-database processing, or honestly just most cases where you use ElasticSearch - not only does it seem to eat as much RAM as you'll give it, but the same seems to apply to the CPU (especially when used as the data store for something like Skywalking APM). On the bright side: most of the modern tech stacks are generally reasonable and there are often micro frameworks (or just more lightweight ones) to be found, which can be suitable for some use cases. Don't want Rails, use Sinatra. Don't want Laravel, use Lumen. Don't want Django, use Flask. Don't want Spring Boot, use Quarkus. That said, Node with Express.js and .NET with ASP.NET are pretty lightweight out of the box, which is nice. Of course, if you go down this route, you might end up with something that is better suited to a bare API, instead of fancy features like server side rendering/templating etc.
wgjordan 4y ago
I'm the author of the original 'refork' feature introduced to Puma a couple years ago [1], that served as an inspiration for this project. As I understand it, Pitchfork is an implementation of a reforking feature on top of Unicorn, a single-threaded multi-process Rack HTTP server. The general idea of 'reforking' is that in addition to pre-forking worker processes at initialization time, you re-fork them on an ongoing basis, in order to optimize copy-on-write efficiency across processes, especially for memory allocations that might happen after initialization. The Pitchfork author mentions sharing YJIT machine code as a particularly important use-case [2], one that has become relevant in recent Ruby releases. I never saw much interest in the concept when I originally introduced the reforking feature to Puma, and it remained a kind of interesting experiment that never saw much production uptake. I'm thrilled that Shopify is iterating on the concept through this project and I hope the technique will continue to be refined and developed, and see broader adoption as a result. [1] https://github.com/puma/puma/pull/2099 https://github.com/puma/puma/pull/2099 ; https://github.com/puma/puma/blob/master/5.0-Upgrade.md#fork_worker https://github.com/puma/puma/blob/master/5.0-Upgrade.md#fork... [2] https://github.com/Shopify/pitchfork/pull/1#issuecomment-1251944727 https://github.com/Shopify/pitchfork/pull/1#issuecomment-125...
- byroot 4y ago
  Hey Will, Pitchfork author here. As mentioned in the Readme, thanks for your Puma PR, it was indeed quite instrumental in Pitchfork inception. As for why it wasn’t used much as a Puma feature I don’t know, but there are a few challenges with it that prevented me from putting it in production. The most important one being that new workers end up being grand children of the original process, so if the middle process die, you may end up with zombies etc. It’s also very scary to fork a process that may have live threads currently processing a request. I believe Pitchfork solves most of that.
  - bullen 4y ago
    Isn't HTTP/1.1 transfer encoding chunked a simpler solution to the same problem?
    - 4y ago
endorphine 4y ago
That's a great project, very interested to see how it plays out. In my previous job we used Unicorn but didn't hit any GVL contention issues (much smaller scale than Shopify's). As a suggestion to the authors: A section titled "Why not Unicorn?" explaining the rationale behind this and why one would want to choose it over Unicorn, would be very helpful. If not in the current experimental stage, then in the "stable" one.
- byroot 4y ago
  Pitchfork author here. GVL contention is an issue when using threads of fibers, so it isn’t an issue with Unicorn
nickjj 4y ago
I wonder how this will handle Websockets. With Rails and Hotwire Turbo, the story to support sprinkling in Websockets for broadcasting events is there. Puma can handle long running connections which makes it suitable to use for that (reverse proxied by nginx of course). Would Unicorn and Pitchfork work the same here in that as long as it's reverse proxied by nginx it'll be fine for Websockets, even with thousands of open connections? What about in development where you wouldn't likely have nginx in front of your app?

137 points by bkudria 4y ago

jbverschoor 4y ago

How does it compare to puma?

aeontech 4y ago
From the Reddit thread linked above (lots of context there, worth reading), the author answers: > what's the main benefit? Drastically reduced memory usage by improving Copy-on-Write performance. For mid-sized apps it can even use less memory than puma depending on how you set it up. See this synthetic benchmark. > Why did you decide to build reforking into a fork of Unicorn instead of contributing it to Puma? Mostly for simplicity. You can build this in Puma, but there are a few extra challenges. For instance you don't want to refork when another thread is currently processing a request, you'd risk leaving some global resource in a corrupted state. So you'd first need to stop accepting traffic and wait for all requests to complete. The problem is, Puma doesn't have a request timeout, so if the request never complete, what do you do? Lots of small challenges like that. But I think it would be awesome if Puma was to take back that idea. Puma's fork_worker feature was a big part of the inspiration after all, and I'd even be happy to help Nate or someone else do it.
wgjordan 4y ago
Puma is a multi-thread, multi-process Rack HTTP server that implements a request buffer. By comparison, Unicorn (the project upon which Pitchfork is based) is a single-thread, multi-process Rack HTTP server that does not buffer requests, so it's only designed for fast clients (or it needs to be paired with a proxy like Nginx to handle slow clients). The narrower design of Unicorn results in a simpler, less-flexible potentially more efficient architecture. The reforking feature that Pitchfork introduces to Unicorn was originally implemented in Puma [1], though they're controlled differently and the underlying implementations are entirely different. [1] https://github.com/puma/puma/blob/master/5.0-Upgrade.md#fork_worker https://github.com/puma/puma/blob/master/5.0-Upgrade.md#fork...

aeontech 4y ago

Looks very promising!

tiffanyh 4y ago

I’m still surprised that we haven’t seen much larger improvements in Ruby performance over the last decade given the large number of major tech companies using Rails. Yes, I'm aware of 3x3 but for the most common usage of Ruby which if for Rails apps - there hasn't been anything like the gains PHP saw from 5.x to 7.x. Especially from Microsoft, who has deep compiler/language expertise, and who owns GitHub (large Rails app).

chucke 4y ago
Not following what you mean. This server is from a shopify employer (major enough?). This server makes quite impressive improvements in terms of memory usage. Moreover, shopify is contributing a production grade JIT to the CRuby runtime.
sosodev 4y ago
There have been plenty of major performance improvements for Ruby but most of them require changes that just aren't feasible for a large codebase. For example, there are alternative Ruby interpreters that are far faster than the MRI but they're not usually a drop in replacement because they don't have full Gem or really general ecosystem compatibility. Also, some performance improvements like JIT just don't matter that much for Rails because the primary problem isn't single thread performance it's the lack of cheap parallelism.
flavorjones 4y ago
Howdy, I'm on the Shopify team that is working on both pitchfork and a few different performance-improvement projects for Ruby. There's a ton of activity around Ruby performance right now! I think we're entering a period of increased experimentation and rapid evolution as demonstrated by projects like YJIT[1][2], improved inline caching[3][4] and Object Shapes[5] (also used by V8), and variable-width allocation[6][7], and smaller improvements like better constant invalidation[8]. Significant investments in TruffleRuby[9] are still going on by Oracle, Shopify, and other companies. And recently, Takashi Kokubun gave a talk at Ruby Kaigi about the future of JIT compilers in Ruby that gives a peek at a whole new set of optimizations Ruby can work on (as well as some performance comparisons against other interpreted languages)[10]. You may be surprised to see how well Ruby (with the JIT enabled) performs compared to Python 3. All of which is to say, I think there's quite a bit of performance improvement being made in recent Rubies, and that trend will likely continue for quite some time. update And I forgot to mention that some very notable computer science researchers and their teams are working in the Ruby community now![11] [1]: https://news.ycombinator.com/item?id=28938446 https://news.ycombinator.com/item?id=28938446) [2]: https://speed.yjit.org/ https://speed.yjit.org/ [3]: https://bugs.ruby-lang.org/issues/18943 https://bugs.ruby-lang.org/issues/18943 [4]: https://bugs.ruby-lang.org/issues/18875 https://bugs.ruby-lang.org/issues/18875 [5]: https://bugs.ruby-lang.org/issues/18776 https://bugs.ruby-lang.org/issues/18776 [6]: https://bugs.ruby-lang.org/issues/18045 https://bugs.ruby-lang.org/issues/18045 [7]: https://bugs.ruby-lang.org/issues/18634 https://bugs.ruby-lang.org/issues/18634 [8]: https://bugs.ruby-lang.org/issues/18589 https://bugs.ruby-lang.org/issues/18589 [9]: https://eregon.me/blog/2022/01/06/benchmarking-cruby-mjit-yjit-jruby-truffleruby.html https://eregon.me/blog/2022/01/06/benchmarking-cruby-mjit-yj... [10]: https://speakerdeck.com/k0kubun/rubykaigi-2022 https://speakerdeck.com/k0kubun/rubykaigi-2022 [11]: https://shopify.engineering/shopify-ruby-at-scale-research-investment https://shopify.engineering/shopify-ruby-at-scale-research-i...

joshmn 4y ago

From r/ruby (including discussion from author): https://old.reddit.com/r/ruby/comments/xwcvty/a_new_web_server_github_shopifypitchfork/ https://old.reddit.com/r/ruby/comments/xwcvty/a_new_web_serv...

a-dub 4y ago

everything old is new again. this was the concurrency model for pre 2.0 apache. (i think in 2.0 you could pick)

pedrocr 4y ago
That was only forking the apache process. This is forking the whole ruby app that also happens to speak HTTP. The gain is in the app side not in the best way to serve HTTP where this is probably a loss over threading or event based solutions.
- a-dub 4y ago
  many serious applications back then built by developers who cared about performance and latency made use of mod_perl. it put the entire perl interpreter (which then loaded perl modules that defined the application) into the preforked apache httpd processes. often times you'd put a proxy in front, serving static assets with a lighter weight http so that slow loads wouldn't tax limited server capacity for big httpd processes that included full perl interpreters and all application code loaded. so it seems it's almost exactly the same?
  - pedrocr 4y ago
    It's very similar to that setup. I don't think that was particularly common though and probably the reason prefork went away. Particularly as apache and later nginx were used as the user facing webservers proxying to something else inside (first with fastcgi and latter just http). The preforking advantage then moved to that server inside which is what this is. Nginx or something similar will still be running in front of this with a threading or event model instead of forking. So this is not going back to an old idea as it never really left, it just left apache because no one ran it as an application server and there are better ways to run an http frontend. Preforking is also not what's new here, this is just an optimization over the preforking model in unicorn which is over a decade old.
    - a-dub 4y ago

skrtskrt 4y ago

I have now worked at several significantly-sized companies that ripped Ruby out of everything as they scaled, usually replaced with Go, though none as large as Shopify. I wonder what that financial calculus looks like between these options for a large organization: 1. trying to basically reinvent the workings of an entire programming language and migrate your Ruby apps to these new tools/runtimes/typecheckers/whatever 2. incrementally rewriting critical paths to something more performant (Go/Rust) & just throwing more servers at the Ruby stuff you haven't managed to replace yet.

tiffanyh 4y ago
Probably not a financial decision. Tobias (Founder/CEO) was an early Rails core contributor. So Ruby on Rails is in Shopify DNA.
jaxrtech 4y ago
A lot of the time, I ask myself, "why don't we seem to have a tool with robust source-to-source transformation?" You could call it "cross-language refactoring". I wouldn't be surprised if someone with the code-base the size of Google has some internal tool. Maybe this is just wishful thinking. The closest thing I've seen is in academic research, e.g. [1]. [1]: Koppel, Solar-Lezama - Incremental parametric syntax for multi-language transformation (2017) - https://dl.acm.org/doi/10.1145/3135932.3135940 https://dl.acm.org/doi/10.1145/3135932.3135940
- pedrocr 4y ago
  There are some tools specific to some transformations at least: https://c2rust.com/ https://c2rust.com/
- dgl 4y ago
  I mean Google did essentially do this: https://opensource.googleblog.com/2017/01/grumpy-go-running-python.html https://opensource.googleblog.com/2017/01/grumpy-go-running-... Except without actually solving the problem of making the translation nice and readable. But this kind of thing at least gives you options for changing language.

tomcam 4y ago

What is Rack? Hard to search for

_t4za 4y ago
https://github.com/rack/rack https://github.com/rack/rack
- tomcam 4y ago
  Thank you!
deleted 4y ago
[deleted]

compumike 4y ago

For those of you scaling Ruby on Rails: are you more constrained by memory or CPU consumption? If you could choose a 50% reduction in process memory footprint, versus a 50% reduction in CPU cycles to serve your average request, which would you pick?

treeman79 4y ago
Database locks is usual issue. Avoidable if planned better. Memory is second. In 15 years CPU has never been a real concern.
- Puts 4y ago
  Second this. After spending several years of full time just troubleshooting others web apps I can say that not only are database locks in 99.99% of the cases the real bottleneck but also that developers in general are quite bad at managing databases and will try to micro optimizing just about anything before looking at their DB queries.
  - brigandish 4y ago
    Most don't know how to improve their DB queries, it's like a black box to them, so they look at the parts they do know. Comfort level is king.
- jamesfinlayson 4y ago
  Yep agreed - in my experience slow queries eventually cause the web server to back up.
- KronisLV 4y ago
  > Database locks is usual issue. Avoidable if planned better. I've often seen problems because of N+1 querying in particular, but maybe that's just the majority of codebases that I've seen. > Memory is second. In 15 years CPU has never been a real concern. In most stacks that I've worked in, memory seems to be the main cause of problems: be it a Ruby application, a PHP one, Python one or Java. Though perhaps that's because larger servers are expensive and development environments in particular tend to be on the more conservative side of things. Either that, or the apps that I've worked with are good examples of Wirth's law and refuse to even run with less than 2 GB of RAM (though maybe that's more along the lines of commentary about enterprise Java apps in particular). When your development/test server has something like 8 GB of RAM or your computer has 16 GB of RAM and you need to run 7 or 8 of those apps (as well as IDE instances), you run into problems. CPU is generally not an issue for most applications, though various databases and data stores instead. Perhaps it's batch migrations or just lots of in-database processing, or honestly just most cases where you use ElasticSearch - not only does it seem to eat as much RAM as you'll give it, but the same seems to apply to the CPU (especially when used as the data store for something like Skywalking APM). On the bright side: most of the modern tech stacks are generally reasonable and there are often micro frameworks (or just more lightweight ones) to be found, which can be suitable for some use cases. Don't want Rails, use Sinatra. Don't want Laravel, use Lumen. Don't want Django, use Flask. Don't want Spring Boot, use Quarkus. That said, Node with Express.js and .NET with ASP.NET are pretty lightweight out of the box, which is nice. Of course, if you go down this route, you might end up with something that is better suited to a bare API, instead of fancy features like server side rendering/templating etc.

wgjordan 4y ago

I'm the author of the original 'refork' feature introduced to Puma a couple years ago [1], that served as an inspiration for this project. As I understand it, Pitchfork is an implementation of a reforking feature on top of Unicorn, a single-threaded multi-process Rack HTTP server. The general idea of 'reforking' is that in addition to pre-forking worker processes at initialization time, you re-fork them on an ongoing basis, in order to optimize copy-on-write efficiency across processes, especially for memory allocations that might happen after initialization. The Pitchfork author mentions sharing YJIT machine code as a particularly important use-case [2], one that has become relevant in recent Ruby releases. I never saw much interest in the concept when I originally introduced the reforking feature to Puma, and it remained a kind of interesting experiment that never saw much production uptake. I'm thrilled that Shopify is iterating on the concept through this project and I hope the technique will continue to be refined and developed, and see broader adoption as a result. [1] https://github.com/puma/puma/pull/2099 https://github.com/puma/puma/pull/2099 ; https://github.com/puma/puma/blob/master/5.0-Upgrade.md#fork_worker https://github.com/puma/puma/blob/master/5.0-Upgrade.md#fork... [2] https://github.com/Shopify/pitchfork/pull/1#issuecomment-1251944727 https://github.com/Shopify/pitchfork/pull/1#issuecomment-125...

byroot 4y ago
Hey Will, Pitchfork author here. As mentioned in the Readme, thanks for your Puma PR, it was indeed quite instrumental in Pitchfork inception. As for why it wasn’t used much as a Puma feature I don’t know, but there are a few challenges with it that prevented me from putting it in production. The most important one being that new workers end up being grand children of the original process, so if the middle process die, you may end up with zombies etc. It’s also very scary to fork a process that may have live threads currently processing a request. I believe Pitchfork solves most of that.
- bullen 4y ago
  Isn't HTTP/1.1 transfer encoding chunked a simpler solution to the same problem?
  - 4y ago

endorphine 4y ago

That's a great project, very interested to see how it plays out. In my previous job we used Unicorn but didn't hit any GVL contention issues (much smaller scale than Shopify's). As a suggestion to the authors: A section titled "Why not Unicorn?" explaining the rationale behind this and why one would want to choose it over Unicorn, would be very helpful. If not in the current experimental stage, then in the "stable" one.

byroot 4y ago
Pitchfork author here. GVL contention is an issue when using threads of fibers, so it isn’t an issue with Unicorn

nickjj 4y ago

I wonder how this will handle Websockets. With Rails and Hotwire Turbo, the story to support sprinkling in Websockets for broadcasting events is there. Puma can handle long running connections which makes it suitable to use for that (reverse proxied by nginx of course). Would Unicorn and Pitchfork work the same here in that as long as it's reverse proxied by nginx it'll be fine for Websockets, even with thousands of open connections? What about in development where you wouldn't likely have nginx in front of your app?