7 ms·

Ruby Concurrency Final Report

229 points by pvsukale3 7y ago

pantulis 7y ago
This is a great read and also provides useful links to get up to date to the state of MT and concurrency in Ruby-world.
ketamine__ 7y ago
I've run into an issue where a Ruby library is technically thread-safe but there is some C library code that blocks (i.e. connect with TinyTds). This makes it practically unusable with threading under special scenarios where your SQL servers are not guaranteed to be available.
- matharmin 7y ago
  Thread-safe just means it won't result in data corruption, race conditions or other similar issues when used from multiple threads. That's still quite different from being able to execute concurrently.
  - k__ 7y ago
    haha, in fact "not being able to execute concurrently" is the easiest way "to ensure thread-safety" :D
    - ioquatix 7y ago
      Actually, completely sequential programs can have parallelism bugs due to the underlying hardware being out of order and other such features. I know, maybe a bit of a stretch by your definitions, but I'm also not sure that "not being able to execute concurrently" implies "thread safety" in practice (even though theoretically pretty sound).
      - BubRoss 7y ago
        That would not be a bug in the software, that would be a bug in the CPU. Still, I haven't ever heard of this actually happening, do you have a link?
        ioquatix 7y ago
        Yes, agreed, all I'm trying to say is, as software engineers we have some assumptions about processors (and compilers/interpreters) that they do things in the order we specify... but that's not always the case, and sometimes that introduces non-determinism. Processors (and compilers) are not supposed to leak the implications of their non-determinism into user code... but unfortunately it does happen. Rewriting the order of instructions sometimes happens incorrectly (which is why we need memory barriers). It's not my area, but two things that come to mind, are out-of-order execution where the CPU will re-order micro-ops, and speculative execution, where the CPU will execute one or more branches before the result of a conditional is known. Both of these issues have lead to bugs in the past and will continue to be a source of bugs in the future. https://en.wikipedia.org/wiki/Spectre_(security_vulnerability) https://en.wikipedia.org/wiki/Spectre_(security_vulnerabilit...
        BubRoss 7y ago
        None of that has anything to do with software bugs, it doesn't change the behavior of a program. People can understand that or not, it doesn't affect the correctness of the software.
- thibaut_barrere 7y ago
  Maybe you are referring to https://github.com/rails-sqlserver/tiny_tds/issues/117 https://github.com/rails-sqlserver/tiny_tds/issues/117 ? If that's the case, it was fixed seven years ago :-)
  - ketamine__ 7y ago
    I am referring to that project but the issue I'm referring to is an issue with connecting to multiple servers concurrently. If the servers have poor connectivity then you can only connect to one at a time despite using threads.
    - thibaut_barrere 7y ago
      Ok! If you ever find the issue back, I'm definitely interested!
      - ketamine__ 7y ago
        You can replicate the issue by doing this: 1) Setup 100-500 sql servers. 2) Artificially increase the amount of time needed to finish connect sequence (> 10 seconds). I have no idea how to do this but it's a likely scenario in my use case of servers on cellular connections. 2) Setup X number of threads to make a connection with TinyTds. 3) Those threads should return immediately but with my testing they did not. Only one connection could be in progress at any given time.
        thibaut_barrere 7y ago
        Thanks! I'll keep this handy in case I multiply the number of connections in my cases.
- ioquatix 7y ago
  If you checked my background section, you'll see even the sqlite3 gem for Ruby blocks and doesn't release the GVL which is a big issue (https://github.com/sparklemotion/sqlite3-ruby/issues/287 https://github.com/sparklemotion/sqlite3-ruby/issues/287). So, completely agree that it's really tricky for users to understand what is going on and how to actually build scalable systems.
  - aardvark179 7y ago
    It's hard even for developers to fully grasp where and how execution might switch. Yes, releasing the GVL will allow other threads to run, but so could other calls which end up calling Ruby code for reasons you would never consider, and that code, or other threads may have effects you didn't consider. I sometimes imagine a warning poster saying, "Do you know whether your RARRAY_PTR is still valid?" I've been thinking for a while how we could allow individual C extensions to opt in to greater concurrency in TruffleRuby. At the moment we have a single mutex for all C extensions, but maybe we could allow an opt out at compile time so an extension could be built that won't claim that mutex. There's some fiddly details to get right round this though, because I wouldn't want the existence of the opt out to have a negative impact on performance.
    - ioquatix 7y ago
      I completely agree it is difficult and that's also my point - that shared multi-thread is very tricky to get right, and there is a significant benefit to isolated event loops (even if they are running on threads, they should be isolated). I like this article by Herb Sutter "The Trouble With Locks". http://gotw.ca/publications/mill36.htm http://gotw.ca/publications/mill36.htm I have used multi-(fiber/thread/process) extensively, and at the right place they are useful tools. But by far, threads which share data (especially opaque) are the most tricky to get right. The combinatorial explosion of program states is very hard to deal with in practice and "it works for me" is a very common testing strategy.
- pmontra 7y ago
  Yep, I almost inherited a Rails application which uses SQL Server. It's the first time I saw it paired with Ruby and I found many deadlocks in the logs, which are basically not existent with PostgreSQL and MySQL. I didn't investigate it deeply because hopefully other people will take care of that project (not a nice technical setup, not because of SQL Server) but I immediately thought of the database driver.
vadasambar 7y ago
I get connection timeout when I try to load the page :(
- ioquatix 7y ago
  As you can imagine, deploying a new technology to production can have issues :p I've been dogfooding falcon (https://github.com/socketry/falcon https://github.com/socketry/falcon) for the past week, which is built on top of async/Ruby. The HN hug of death + Reddit hug of death is a really great traffic test. I think actually it's been pretty solid, but something caused the instance to run out of swap space, even though it had plenty of free memory. It's something I'll have to try and reproduce so I can understand how it's happening.
nickbauman 7y ago
Working complex systems develop from working simple systems. This is trying to get a working complex system by creating another initial working complex system. If only ruby didn't have that darned assignment operator...
- ioquatix 7y ago
  I definitely agree with your first statement. However, I think the Fiber + Reactor approach is about as simple as it gets, taking into consideration actual practical, scalable, concurrency. Every approach has trade offs, but I think this design is pretty good. My goal was to build enough of the stack to prove that.
thibaut_barrere 7y ago
ioquatix (Samuel Williams, the author) is doing tons of great work for Ruby at the moment. You'll want to check out, for instance: - https://github.com/socketry/async https://github.com/socketry/async "An awesome asynchronous event-driven reactor for Ruby" - https://github.com/socketry/falcon https://github.com/socketry/falcon "A high-performance web server for Ruby, supporting HTTP/1, HTTP/2 and TLS" I can recommend following him on twitter too: http://twitter.com/ioquatix http://twitter.com/ioquatix
- giu 7y ago
  In the linked article, the author also included some videos of himself implementing some of the presented parts e.g. exclusive Fibers [0], including writing the necessary C code using the Ruby C API. Recording yourself writing code and then publishing it takes quite a lot of courage, even moreso if it's a complex topic such as the one presented. Additionally, it's such a valuable resource: You can follow the whole implementation step by step. That's just awesome. [0] https://www.youtube.com/watch?v=cNaqbeagqUw https://www.youtube.com/watch?v=cNaqbeagqUw
  - tylerscott 7y ago
    Thanks for the link. This is great!
  - djohnston 7y ago
    This is really cool. Do you have any recs for similar vid of someone hacking on c python API? I want to gain a better understanding of how these langs interface w their implementation, but Ruby isn't really in my wheelhouse.
    - ioquatix 7y ago
      The internals aren't THAT different, so while I can't really answer your question (I'm not working on CPython), you might still find it interesting from the POV as to how interpreters work.
  - psychoslave 7y ago
bepvte 7y ago
This thin font is very difficult to read
- ioquatix 7y ago
  Welcome suggestions for alternatives.
  - bloudermilk 7y ago
    Bootstrap adopted a native font stack that looks great and doesn't require any web fonts. https://getbootstrap.com/docs/4.0/content/reboot/#native-font-stack https://getbootstrap.com/docs/4.0/content/reboot/#native-fon...
    - ioquatix 7y ago
      Okay, I will check it out, thanks!
    - ptx 7y ago
      Is "adopting a native font stack" different in some way from simply not using web fonts and doing it the old-fashioned way instead? Edit: Other than using the magic font names "-apple-system" and "BlinkMacSystemFont" it looks like it's just specifying the Windows, Android and Mac fonts in order. This is not going to use the native system font on other platforms or if the user happens to have Segoe UI installed on Android.
    - saagarjha 7y ago
      That stack should lead with system-ui.
- psadauskas 7y ago
  I noticed this, too. On Fedora KDE, it looks fine in Chromium, but poor in Firefox. It seems Firefox can't find the "PT Sans" font, even though I have it installed, and Chromium uses it correctly.
NickGerleman 7y ago
I haven't touched Ruby in a while, but are there any common multithreaded use cases? It seemed like the direction was to go multi-process for web workloads (E.g. with Unicorn).
- thibaut_barrere 7y ago
  Both Puma (https://github.com/puma/puma https://github.com/puma/puma, a popular server these days) and Passenger Enterprise (paid) provide multithreaded web support. Also on the background jobs side, Sidekiq https://sidekiq.org https://sidekiq.org is very popular.
  - deleted 7y ago
    [deleted]
- ioquatix 7y ago
  Those are both solid choices for servers. However neither of them have a scalability model suitable for HTTP/2 or WebSockets. That's something I wanted to try and address.
  - nitrogen 7y ago
    I used EventMachine a lot about 8-10 years ago. I'm excited to see Ruby getting some concurrency love again. What are the goals and improvements of your underlying design in general, and especially those that make HTTP2 and WebSockets work?
    - ioquatix 7y ago
      Light weight event driven tasks are very important. I outline some of the key improvements in this talk: https://www.youtube.com/watch?v=Dtn9Uudw4Mo https://www.youtube.com/watch?v=Dtn9Uudw4Mo
  - timdorr 7y ago
    And thank you for addressing it, Samuel! I converted the streaming and Autopark clients in the Tesla API gem from EM to Async and has been great: https://github.com/timdorr/tesla-api/blob/master/lib/tesla_api/stream.rb https://github.com/timdorr/tesla-api/blob/master/lib/tesla_a... No more globals, which makes it much safer to use inside of threaded job runners like Sidekiq.
mrtweetyhack 7y ago
So nobody is talking about this? https://www.zdnet.com/article/clipboard-hijacking-malware-found-in-725-ruby-libraries/ https://www.zdnet.com/article/clipboard-hijacking-malware-fo... https://arstechnica.com/information-technology/2020/04/725-bitcoin-stealing-apps-snuck-into-ruby-repository/ https://arstechnica.com/information-technology/2020/04/725-b...

229 points by pvsukale3 7y ago

This is a great read and also provides useful links to get up to date to the state of MT and concurrency in Ruby-world.

I've run into an issue where a Ruby library is technically thread-safe but there is some C library code that blocks (i.e. connect with TinyTds). This makes it practically unusable with threading under special scenarios where your SQL servers are not guaranteed to be available.

matharmin 7y ago
Thread-safe just means it won't result in data corruption, race conditions or other similar issues when used from multiple threads. That's still quite different from being able to execute concurrently.
- k__ 7y ago
  haha, in fact "not being able to execute concurrently" is the easiest way "to ensure thread-safety" :D
  - ioquatix 7y ago
    Actually, completely sequential programs can have parallelism bugs due to the underlying hardware being out of order and other such features. I know, maybe a bit of a stretch by your definitions, but I'm also not sure that "not being able to execute concurrently" implies "thread safety" in practice (even though theoretically pretty sound).
    - BubRoss 7y ago
      That would not be a bug in the software, that would be a bug in the CPU. Still, I haven't ever heard of this actually happening, do you have a link?
      - ioquatix 7y ago
        Yes, agreed, all I'm trying to say is, as software engineers we have some assumptions about processors (and compilers/interpreters) that they do things in the order we specify... but that's not always the case, and sometimes that introduces non-determinism. Processors (and compilers) are not supposed to leak the implications of their non-determinism into user code... but unfortunately it does happen. Rewriting the order of instructions sometimes happens incorrectly (which is why we need memory barriers). It's not my area, but two things that come to mind, are out-of-order execution where the CPU will re-order micro-ops, and speculative execution, where the CPU will execute one or more branches before the result of a conditional is known. Both of these issues have lead to bugs in the past and will continue to be a source of bugs in the future. https://en.wikipedia.org/wiki/Spectre_(security_vulnerability) https://en.wikipedia.org/wiki/Spectre_(security_vulnerabilit...
        BubRoss 7y ago
        None of that has anything to do with software bugs, it doesn't change the behavior of a program. People can understand that or not, it doesn't affect the correctness of the software.
thibaut_barrere 7y ago
Maybe you are referring to https://github.com/rails-sqlserver/tiny_tds/issues/117 https://github.com/rails-sqlserver/tiny_tds/issues/117 ? If that's the case, it was fixed seven years ago :-)
- ketamine__ 7y ago
  I am referring to that project but the issue I'm referring to is an issue with connecting to multiple servers concurrently. If the servers have poor connectivity then you can only connect to one at a time despite using threads.
  - thibaut_barrere 7y ago
    Ok! If you ever find the issue back, I'm definitely interested!
    - ketamine__ 7y ago
      You can replicate the issue by doing this: 1) Setup 100-500 sql servers. 2) Artificially increase the amount of time needed to finish connect sequence (> 10 seconds). I have no idea how to do this but it's a likely scenario in my use case of servers on cellular connections. 2) Setup X number of threads to make a connection with TinyTds. 3) Those threads should return immediately but with my testing they did not. Only one connection could be in progress at any given time.
      - thibaut_barrere 7y ago
        Thanks! I'll keep this handy in case I multiply the number of connections in my cases.
ioquatix 7y ago
If you checked my background section, you'll see even the sqlite3 gem for Ruby blocks and doesn't release the GVL which is a big issue (https://github.com/sparklemotion/sqlite3-ruby/issues/287 https://github.com/sparklemotion/sqlite3-ruby/issues/287). So, completely agree that it's really tricky for users to understand what is going on and how to actually build scalable systems.
- aardvark179 7y ago
  It's hard even for developers to fully grasp where and how execution might switch. Yes, releasing the GVL will allow other threads to run, but so could other calls which end up calling Ruby code for reasons you would never consider, and that code, or other threads may have effects you didn't consider. I sometimes imagine a warning poster saying, "Do you know whether your RARRAY_PTR is still valid?" I've been thinking for a while how we could allow individual C extensions to opt in to greater concurrency in TruffleRuby. At the moment we have a single mutex for all C extensions, but maybe we could allow an opt out at compile time so an extension could be built that won't claim that mutex. There's some fiddly details to get right round this though, because I wouldn't want the existence of the opt out to have a negative impact on performance.
  - ioquatix 7y ago
    I completely agree it is difficult and that's also my point - that shared multi-thread is very tricky to get right, and there is a significant benefit to isolated event loops (even if they are running on threads, they should be isolated). I like this article by Herb Sutter "The Trouble With Locks". http://gotw.ca/publications/mill36.htm http://gotw.ca/publications/mill36.htm I have used multi-(fiber/thread/process) extensively, and at the right place they are useful tools. But by far, threads which share data (especially opaque) are the most tricky to get right. The combinatorial explosion of program states is very hard to deal with in practice and "it works for me" is a very common testing strategy.
pmontra 7y ago
Yep, I almost inherited a Rails application which uses SQL Server. It's the first time I saw it paired with Ruby and I found many deadlocks in the logs, which are basically not existent with PostgreSQL and MySQL. I didn't investigate it deeply because hopefully other people will take care of that project (not a nice technical setup, not because of SQL Server) but I immediately thought of the database driver.

vadasambar 7y ago

I get connection timeout when I try to load the page :(

ioquatix 7y ago
As you can imagine, deploying a new technology to production can have issues :p I've been dogfooding falcon (https://github.com/socketry/falcon https://github.com/socketry/falcon) for the past week, which is built on top of async/Ruby. The HN hug of death + Reddit hug of death is a really great traffic test. I think actually it's been pretty solid, but something caused the instance to run out of swap space, even though it had plenty of free memory. It's something I'll have to try and reproduce so I can understand how it's happening.

nickbauman 7y ago

Working complex systems develop from working simple systems. This is trying to get a working complex system by creating another initial working complex system. If only ruby didn't have that darned assignment operator...

ioquatix 7y ago
I definitely agree with your first statement. However, I think the Fiber + Reactor approach is about as simple as it gets, taking into consideration actual practical, scalable, concurrency. Every approach has trade offs, but I think this design is pretty good. My goal was to build enough of the stack to prove that.

thibaut_barrere 7y ago

ioquatix (Samuel Williams, the author) is doing tons of great work for Ruby at the moment. You'll want to check out, for instance: - https://github.com/socketry/async https://github.com/socketry/async "An awesome asynchronous event-driven reactor for Ruby" - https://github.com/socketry/falcon https://github.com/socketry/falcon "A high-performance web server for Ruby, supporting HTTP/1, HTTP/2 and TLS" I can recommend following him on twitter too: http://twitter.com/ioquatix http://twitter.com/ioquatix

giu 7y ago
In the linked article, the author also included some videos of himself implementing some of the presented parts e.g. exclusive Fibers [0], including writing the necessary C code using the Ruby C API. Recording yourself writing code and then publishing it takes quite a lot of courage, even moreso if it's a complex topic such as the one presented. Additionally, it's such a valuable resource: You can follow the whole implementation step by step. That's just awesome. [0] https://www.youtube.com/watch?v=cNaqbeagqUw https://www.youtube.com/watch?v=cNaqbeagqUw
- tylerscott 7y ago
  Thanks for the link. This is great!
- djohnston 7y ago
  This is really cool. Do you have any recs for similar vid of someone hacking on c python API? I want to gain a better understanding of how these langs interface w their implementation, but Ruby isn't really in my wheelhouse.
  - ioquatix 7y ago
    The internals aren't THAT different, so while I can't really answer your question (I'm not working on CPython), you might still find it interesting from the POV as to how interpreters work.
- psychoslave 7y ago

bepvte 7y ago

This thin font is very difficult to read

ioquatix 7y ago
Welcome suggestions for alternatives.
- bloudermilk 7y ago
  Bootstrap adopted a native font stack that looks great and doesn't require any web fonts. https://getbootstrap.com/docs/4.0/content/reboot/#native-font-stack https://getbootstrap.com/docs/4.0/content/reboot/#native-fon...
  - ioquatix 7y ago
    Okay, I will check it out, thanks!
  - ptx 7y ago
    Is "adopting a native font stack" different in some way from simply not using web fonts and doing it the old-fashioned way instead? Edit: Other than using the magic font names "-apple-system" and "BlinkMacSystemFont" it looks like it's just specifying the Windows, Android and Mac fonts in order. This is not going to use the native system font on other platforms or if the user happens to have Segoe UI installed on Android.
  - saagarjha 7y ago
    That stack should lead with system-ui.
psadauskas 7y ago
I noticed this, too. On Fedora KDE, it looks fine in Chromium, but poor in Firefox. It seems Firefox can't find the "PT Sans" font, even though I have it installed, and Chromium uses it correctly.

NickGerleman 7y ago

I haven't touched Ruby in a while, but are there any common multithreaded use cases? It seemed like the direction was to go multi-process for web workloads (E.g. with Unicorn).

thibaut_barrere 7y ago
Both Puma (https://github.com/puma/puma https://github.com/puma/puma, a popular server these days) and Passenger Enterprise (paid) provide multithreaded web support. Also on the background jobs side, Sidekiq https://sidekiq.org https://sidekiq.org is very popular.
- deleted 7y ago
  [deleted]
ioquatix 7y ago
Those are both solid choices for servers. However neither of them have a scalability model suitable for HTTP/2 or WebSockets. That's something I wanted to try and address.
- nitrogen 7y ago
  I used EventMachine a lot about 8-10 years ago. I'm excited to see Ruby getting some concurrency love again. What are the goals and improvements of your underlying design in general, and especially those that make HTTP2 and WebSockets work?
  - ioquatix 7y ago
    Light weight event driven tasks are very important. I outline some of the key improvements in this talk: https://www.youtube.com/watch?v=Dtn9Uudw4Mo https://www.youtube.com/watch?v=Dtn9Uudw4Mo
- timdorr 7y ago
  And thank you for addressing it, Samuel! I converted the streaming and Autopark clients in the Tesla API gem from EM to Async and has been great: https://github.com/timdorr/tesla-api/blob/master/lib/tesla_api/stream.rb https://github.com/timdorr/tesla-api/blob/master/lib/tesla_a... No more globals, which makes it much safer to use inside of threaded job runners like Sidekiq.

mrtweetyhack 7y ago

So nobody is talking about this? https://www.zdnet.com/article/clipboard-hijacking-malware-found-in-725-ruby-libraries/ https://www.zdnet.com/article/clipboard-hijacking-malware-fo... https://arstechnica.com/information-technology/2020/04/725-bitcoin-stealing-apps-snuck-into-ruby-repository/ https://arstechnica.com/information-technology/2020/04/725-b...