8 ms·

I've run into an issue where a Ruby library is technically thread-safe but there is some C library code that blocks (i.e. connect with TinyTds). This makes it p

I've run into an issue where a Ruby library is technically thread-safe but there is some C library code that blocks (i.e. connect with TinyTds). This makes it practically unusable with threading under special scenarios where your SQL servers are not guaranteed to be available.

matharmin 7y ago
Thread-safe just means it won't result in data corruption, race conditions or other similar issues when used from multiple threads. That's still quite different from being able to execute concurrently.
- k__ 7y ago
  haha, in fact "not being able to execute concurrently" is the easiest way "to ensure thread-safety" :D
  - ioquatix 7y ago
    Actually, completely sequential programs can have parallelism bugs due to the underlying hardware being out of order and other such features. I know, maybe a bit of a stretch by your definitions, but I'm also not sure that "not being able to execute concurrently" implies "thread safety" in practice (even though theoretically pretty sound).
    - BubRoss 7y ago
      That would not be a bug in the software, that would be a bug in the CPU. Still, I haven't ever heard of this actually happening, do you have a link?
      - ioquatix 7y ago
        Yes, agreed, all I'm trying to say is, as software engineers we have some assumptions about processors (and compilers/interpreters) that they do things in the order we specify... but that's not always the case, and sometimes that introduces non-determinism. Processors (and compilers) are not supposed to leak the implications of their non-determinism into user code... but unfortunately it does happen. Rewriting the order of instructions sometimes happens incorrectly (which is why we need memory barriers). It's not my area, but two things that come to mind, are out-of-order execution where the CPU will re-order micro-ops, and speculative execution, where the CPU will execute one or more branches before the result of a conditional is known. Both of these issues have lead to bugs in the past and will continue to be a source of bugs in the future. https://en.wikipedia.org/wiki/Spectre_(security_vulnerability) https://en.wikipedia.org/wiki/Spectre_(security_vulnerabilit...
        BubRoss 7y ago
        None of that has anything to do with software bugs, it doesn't change the behavior of a program. People can understand that or not, it doesn't affect the correctness of the software.
thibaut_barrere 7y ago
Maybe you are referring to https://github.com/rails-sqlserver/tiny_tds/issues/117 https://github.com/rails-sqlserver/tiny_tds/issues/117 ? If that's the case, it was fixed seven years ago :-)
- ketamine__ 7y ago
  I am referring to that project but the issue I'm referring to is an issue with connecting to multiple servers concurrently. If the servers have poor connectivity then you can only connect to one at a time despite using threads.
  - thibaut_barrere 7y ago
    Ok! If you ever find the issue back, I'm definitely interested!
    - ketamine__ 7y ago
      You can replicate the issue by doing this: 1) Setup 100-500 sql servers. 2) Artificially increase the amount of time needed to finish connect sequence (> 10 seconds). I have no idea how to do this but it's a likely scenario in my use case of servers on cellular connections. 2) Setup X number of threads to make a connection with TinyTds. 3) Those threads should return immediately but with my testing they did not. Only one connection could be in progress at any given time.
      - thibaut_barrere 7y ago
        Thanks! I'll keep this handy in case I multiply the number of connections in my cases.
ioquatix 7y ago
If you checked my background section, you'll see even the sqlite3 gem for Ruby blocks and doesn't release the GVL which is a big issue (https://github.com/sparklemotion/sqlite3-ruby/issues/287 https://github.com/sparklemotion/sqlite3-ruby/issues/287). So, completely agree that it's really tricky for users to understand what is going on and how to actually build scalable systems.
- aardvark179 7y ago
  It's hard even for developers to fully grasp where and how execution might switch. Yes, releasing the GVL will allow other threads to run, but so could other calls which end up calling Ruby code for reasons you would never consider, and that code, or other threads may have effects you didn't consider. I sometimes imagine a warning poster saying, "Do you know whether your RARRAY_PTR is still valid?" I've been thinking for a while how we could allow individual C extensions to opt in to greater concurrency in TruffleRuby. At the moment we have a single mutex for all C extensions, but maybe we could allow an opt out at compile time so an extension could be built that won't claim that mutex. There's some fiddly details to get right round this though, because I wouldn't want the existence of the opt out to have a negative impact on performance.
  - ioquatix 7y ago
    I completely agree it is difficult and that's also my point - that shared multi-thread is very tricky to get right, and there is a significant benefit to isolated event loops (even if they are running on threads, they should be isolated). I like this article by Herb Sutter "The Trouble With Locks". http://gotw.ca/publications/mill36.htm http://gotw.ca/publications/mill36.htm I have used multi-(fiber/thread/process) extensively, and at the right place they are useful tools. But by far, threads which share data (especially opaque) are the most tricky to get right. The combinatorial explosion of program states is very hard to deal with in practice and "it works for me" is a very common testing strategy.
pmontra 7y ago
Yep, I almost inherited a Rails application which uses SQL Server. It's the first time I saw it paired with Ruby and I found many deadlocks in the logs, which are basically not existent with PostgreSQL and MySQL. I didn't investigate it deeply because hopefully other people will take care of that project (not a nice technical setup, not because of SQL Server) but I immediately thought of the database driver.

matharmin 7y ago

Thread-safe just means it won't result in data corruption, race conditions or other similar issues when used from multiple threads. That's still quite different from being able to execute concurrently.

k__ 7y ago
haha, in fact "not being able to execute concurrently" is the easiest way "to ensure thread-safety" :D
- ioquatix 7y ago
  Actually, completely sequential programs can have parallelism bugs due to the underlying hardware being out of order and other such features. I know, maybe a bit of a stretch by your definitions, but I'm also not sure that "not being able to execute concurrently" implies "thread safety" in practice (even though theoretically pretty sound).
  - BubRoss 7y ago
    That would not be a bug in the software, that would be a bug in the CPU. Still, I haven't ever heard of this actually happening, do you have a link?
    - ioquatix 7y ago
      Yes, agreed, all I'm trying to say is, as software engineers we have some assumptions about processors (and compilers/interpreters) that they do things in the order we specify... but that's not always the case, and sometimes that introduces non-determinism. Processors (and compilers) are not supposed to leak the implications of their non-determinism into user code... but unfortunately it does happen. Rewriting the order of instructions sometimes happens incorrectly (which is why we need memory barriers). It's not my area, but two things that come to mind, are out-of-order execution where the CPU will re-order micro-ops, and speculative execution, where the CPU will execute one or more branches before the result of a conditional is known. Both of these issues have lead to bugs in the past and will continue to be a source of bugs in the future. https://en.wikipedia.org/wiki/Spectre_(security_vulnerability) https://en.wikipedia.org/wiki/Spectre_(security_vulnerabilit...
      - BubRoss 7y ago
        None of that has anything to do with software bugs, it doesn't change the behavior of a program. People can understand that or not, it doesn't affect the correctness of the software.

thibaut_barrere 7y ago

Maybe you are referring to https://github.com/rails-sqlserver/tiny_tds/issues/117 https://github.com/rails-sqlserver/tiny_tds/issues/117 ? If that's the case, it was fixed seven years ago :-)

ketamine__ 7y ago
I am referring to that project but the issue I'm referring to is an issue with connecting to multiple servers concurrently. If the servers have poor connectivity then you can only connect to one at a time despite using threads.
- thibaut_barrere 7y ago
  Ok! If you ever find the issue back, I'm definitely interested!
  - ketamine__ 7y ago
    You can replicate the issue by doing this: 1) Setup 100-500 sql servers. 2) Artificially increase the amount of time needed to finish connect sequence (> 10 seconds). I have no idea how to do this but it's a likely scenario in my use case of servers on cellular connections. 2) Setup X number of threads to make a connection with TinyTds. 3) Those threads should return immediately but with my testing they did not. Only one connection could be in progress at any given time.
    - thibaut_barrere 7y ago
      Thanks! I'll keep this handy in case I multiply the number of connections in my cases.

ioquatix 7y ago

If you checked my background section, you'll see even the sqlite3 gem for Ruby blocks and doesn't release the GVL which is a big issue (https://github.com/sparklemotion/sqlite3-ruby/issues/287 https://github.com/sparklemotion/sqlite3-ruby/issues/287). So, completely agree that it's really tricky for users to understand what is going on and how to actually build scalable systems.

aardvark179 7y ago
It's hard even for developers to fully grasp where and how execution might switch. Yes, releasing the GVL will allow other threads to run, but so could other calls which end up calling Ruby code for reasons you would never consider, and that code, or other threads may have effects you didn't consider. I sometimes imagine a warning poster saying, "Do you know whether your RARRAY_PTR is still valid?" I've been thinking for a while how we could allow individual C extensions to opt in to greater concurrency in TruffleRuby. At the moment we have a single mutex for all C extensions, but maybe we could allow an opt out at compile time so an extension could be built that won't claim that mutex. There's some fiddly details to get right round this though, because I wouldn't want the existence of the opt out to have a negative impact on performance.
- ioquatix 7y ago
  I completely agree it is difficult and that's also my point - that shared multi-thread is very tricky to get right, and there is a significant benefit to isolated event loops (even if they are running on threads, they should be isolated). I like this article by Herb Sutter "The Trouble With Locks". http://gotw.ca/publications/mill36.htm http://gotw.ca/publications/mill36.htm I have used multi-(fiber/thread/process) extensively, and at the right place they are useful tools. But by far, threads which share data (especially opaque) are the most tricky to get right. The combinatorial explosion of program states is very hard to deal with in practice and "it works for me" is a very common testing strategy.

pmontra 7y ago

Yep, I almost inherited a Rails application which uses SQL Server. It's the first time I saw it paired with Ruby and I found many deadlocks in the logs, which are basically not existent with PostgreSQL and MySQL. I didn't investigate it deeply because hopefully other people will take care of that project (not a nice technical setup, not because of SQL Server) but I immediately thought of the database driver.