3 ms·

If you checked my background section, you'll see even the sqlite3 gem for Ruby blocks and doesn't release the GVL which is a big issue (https://github.com/spark

If you checked my background section, you'll see even the sqlite3 gem for Ruby blocks and doesn't release the GVL which is a big issue (https://github.com/sparklemotion/sqlite3-ruby/issues/287 https://github.com/sparklemotion/sqlite3-ruby/issues/287). So, completely agree that it's really tricky for users to understand what is going on and how to actually build scalable systems.

aardvark179 6y ago
It's hard even for developers to fully grasp where and how execution might switch. Yes, releasing the GVL will allow other threads to run, but so could other calls which end up calling Ruby code for reasons you would never consider, and that code, or other threads may have effects you didn't consider. I sometimes imagine a warning poster saying, "Do you know whether your RARRAY_PTR is still valid?" I've been thinking for a while how we could allow individual C extensions to opt in to greater concurrency in TruffleRuby. At the moment we have a single mutex for all C extensions, but maybe we could allow an opt out at compile time so an extension could be built that won't claim that mutex. There's some fiddly details to get right round this though, because I wouldn't want the existence of the opt out to have a negative impact on performance.
- ioquatix 6y ago
  I completely agree it is difficult and that's also my point - that shared multi-thread is very tricky to get right, and there is a significant benefit to isolated event loops (even if they are running on threads, they should be isolated). I like this article by Herb Sutter "The Trouble With Locks". http://gotw.ca/publications/mill36.htm http://gotw.ca/publications/mill36.htm I have used multi-(fiber/thread/process) extensively, and at the right place they are useful tools. But by far, threads which share data (especially opaque) are the most tricky to get right. The combinatorial explosion of program states is very hard to deal with in practice and "it works for me" is a very common testing strategy.

aardvark179 6y ago

It's hard even for developers to fully grasp where and how execution might switch. Yes, releasing the GVL will allow other threads to run, but so could other calls which end up calling Ruby code for reasons you would never consider, and that code, or other threads may have effects you didn't consider. I sometimes imagine a warning poster saying, "Do you know whether your RARRAY_PTR is still valid?" I've been thinking for a while how we could allow individual C extensions to opt in to greater concurrency in TruffleRuby. At the moment we have a single mutex for all C extensions, but maybe we could allow an opt out at compile time so an extension could be built that won't claim that mutex. There's some fiddly details to get right round this though, because I wouldn't want the existence of the opt out to have a negative impact on performance.

ioquatix 6y ago
I completely agree it is difficult and that's also my point - that shared multi-thread is very tricky to get right, and there is a significant benefit to isolated event loops (even if they are running on threads, they should be isolated). I like this article by Herb Sutter "The Trouble With Locks". http://gotw.ca/publications/mill36.htm http://gotw.ca/publications/mill36.htm I have used multi-(fiber/thread/process) extensively, and at the right place they are useful tools. But by far, threads which share data (especially opaque) are the most tricky to get right. The combinatorial explosion of program states is very hard to deal with in practice and "it works for me" is a very common testing strategy.