16 ms·
Ruby 3.0 and the new FiberScheduler interface
- deleted 6y ago[deleted]
- elcritch 6y agoAs an outsider, why doesn't/can't Ruby implement a full actor system like Erlang/BEAM? Ruby is already based on message passing so it seems like it should be possible. Granted it'd likely induce a large performance hit since presumably every object would need to be locked or have a message queue.
- holstvoogd 6y agoThere are different fully concurrent ruby implementations: truffleruby, rubinius, jruby.. they all suck at what ruby is most used for, running rails. Last time i tried cruby was 10x faster still. So in practice we glady throw more memory and processes at the problem.
- cortesoft 6y agoI wonder what percentage of Ruby is rails... I have been using Ruby pretty much every day for work since 2005, and haven't done any rails since 2009. Wonder if I am that much of an anomaly.
- freedomben 6y agoWhat are you using Ruby for? I have used ruby since 2013 and started on rails but quit using rails in 2015. I still use ruby all the time though, mostly for scripts/automation that violate my rule on bash v. other lang for scripts (basically do I need arrays, maps, or to parse json beyond simple extractions that jq is great at). I do have a couple sinatra services now though that I maintain. Sinatra is wonderful with simple needs like mine. I don't know how much of an anomaly I am though, interested in hearing from others as well. Edit: not looking to debate bash vs other langs here. Of course you're free to do such below, but I won't be engaging (got to focus on work and have had the debate many times and I don't think any of us will ever convince the other. It's become religion at this point). Also I use Elixir/Phoenix these days for use cases that used to be rails
- cortesoft 6y agoA lot of services written with Sinatra, some developer tooling, scripts to maintain our systems.
- mediaman 6y agoAnything you miss from Rails now that you're primarily using Phoenix for those use cases? Elixir/Phoenix does look pretty intriguing to me, though the support universe is smaller. What held me back so far is that a lot of simple B2B, low volume stuff doesn't seem to benefit from the high parallelism that BEAM brings. But I'm wondering if it brings enough other advantages that I shouldn't view it that simplistically.
- realusername 6y ago> But I'm wondering if it brings enough other advantages that I shouldn't view it that simplistically. So to start with, Phoenix LiveView really is a game changer, you should have a look at Elixir just for that. This is the talk which really bluffed me when I first saw it: https://www.youtube.com/watch?v=MZvmYaFkNJI https://www.youtube.com/watch?v=MZvmYaFkNJI. For the other upsides, I like a little bit better the way everything is architectured in Phoenix, there's much less magic and it's easier to follow the data flow and what is going on.
- lawik 6y agoNot the previous commenter. But enthusiastic :) I would recommend trying it and seeing how you like it. I've basically dropped Python which was my daily language in favor of Elixir. I find I get a higher top bound to what I can do, high-level expressive but less magical code and a bunch of capabilities that aren't typically feasible with other runtimes (state handling, resiliency and stuff). The parallelism I get for free with Phoenix whether I try or not. I'd watch Sasa Jurics talk on the heart of elixir and erlang to get more of the technical advantages laid out quite well.
- cpuguy83 6y agoWhat are you doing that cruby is faster? I've run both Rubinius and jruby (with rails), both gave me significant performance gains.
- rubyn00bie 6y agoWell... the big reason almost no runtime/vm provides what erlang does is because BEAM will preempt a running actor if it takes too long (uses up its operation budget more or less). Ruby only provides mechanisms for cooperatively scheduled execution. That is to say, and this is the big problem with async most everywhere except Erlang/BEAM, a task will block everything else from executing until its finished and yields to another fiber/thread. Shrug Having tried like every possible method of async I/O with Ruby to eek out moar good perfz back in the day... including various actor implementations (sup, JRuby and Akka)... nothing compares to just using BEAM if you want the actor paradigm. The cooperative scheduling problem is a really enormous pain in the ass most of the time.
- elcritch 6y agoThat's part of my question (to me at least) is why they couldn't implement preemptive actors? At least at the granularity of individuals methods which in Ruby is pretty much any action, right? Async and the whole red/blue function problem seems like a pain.
- ioquatix 6y agoWe can do the same model for Ruby (preemption of event driven fibers) and may choose to do so in the future. We are just taking one step at a time.
- semiquaver 6y agoRactors, also introduced in ruby 3.0, are basically this. Ractors work by passing messages or immutable objects. https://github.com/ruby/ruby/blob/master/doc/ractor.md https://github.com/ruby/ruby/blob/master/doc/ractor.md
- chrisseaton 6y ago> As an outsider, why doesn't/can't Ruby implement a full actor system like Erlang/BEAM? Do you really want an actor system? Actors are non-determinsitic, extremely prone to difficult-to-debug race conditions, and inherently stateful. They're a classic concurrency foot-gun from the dark old ways of doing things. Don't we want to be moving away from these models that we know trip people up? Can't we do better than this for Ruby?
- dnautics 6y agoYou shouldn't use actor systems as a code organization pattern, only in places where async is necessary and in those cases you're going to have to debug race conditions anyways. When that happens you will want an actor system because it will make it easier to understand the data flowing through your system, and especially, effortlessly (0 loc) clean up dangling resources from error conditions and prevent leaks. Moreover Elixir provides you fantastic tools to drive unit and integration testing around your async systems.
- elcritch 6y ago> They're a classic concurrency foot-gun from the dark old ways of doing things. Sure you're not thinking of threads? Actor's have only really been done in Erlang, and more recenetly in Akka and Orlean's. AFAIK, there aren't any other paradigms other than CSP or threads for concurrency. Actor's can have race conditions, same as any other concurrent system but I've personally never actually had any given the design patterns in OTP and Elixir. It's really helpful (to me) that each actor is always deterministic in itself and doesn't share data, only messages. It also maps nicely onto multicore.
- chrisseaton 6y ago> Sure you're not thinking of threads? No I'm thinking of actors - stateful, non-determinstic, racey actors. A minefield of classic concurrency bugs! > Actor's can have race conditions, same as any other concurrent system Deterministic systems like fork-join don't ever have race-conditions. > is always deterministic in itself and doesn't share data But the global state of all the actors is shared. If an actor you send a message to responds differently due to its state then let's be honest you're implicitly sharing that state. > It also maps nicely onto multicore. It maps directly to multicore, right. Don't we want something higher level and safer than directly mapping to our hardware?
- claudiug 6y agowill this fiber scheduler + ractor will probably make ruby faster?
- nateberkopec 6y agoBoth of those changes increase parallelism and concurrency, they do not decrease latency by themselves.
- cplanas 6y agoNot directly, but they will facilitate using more performant paradigms.
- faebi 6y agoAre there any good libraries for fiber pools, so that not to many fibers run at once? I am interested in something like the concurrent-ruby threadpools.
- WJW 6y agoOnly one fiber per thread runs at a time, so that would naturally limit the concurrency of fibers to the number of threads?
- faebi 6y agoAs I understand it will yield back once it hits IO. But if you have something like a web server with a timeout of 60s you won't be able to open 10'000 connections at once to it. Some or all will be probably run into the timeout before the ruby could process the whole request. Therefore I would try to use some form of rate limiting via a FiberPool with a backlog.
- cogman10 6y agoUse a semaphore instead. That's what it is built for. That way you don't need to make a complex fiber scheme just for resource management. Spin up as many as you need an the runtime will do the task pool for you. https://ruby-concurrency.github.io/concurrent-ruby/1.1.4/Concurrent/Semaphore.html https://ruby-concurrency.github.io/concurrent-ruby/1.1.4/Con...
- ioquatix 6y agohttps://github.com/socketry/async-pool https://github.com/socketry/async-pool
- freedomben 6y agoI'm excited to see where this goes, but I must admit I'm conflicted on the idea of seeing async Ruby code. Async is great technically but one of the things I love about Ruby is the elegance and clarity. That tends to go out the window when async is introduced in many languages. async/await has helped a ton though, so hopefully Ruby will get something like that as we go. I would just suggest that sometimes performance isn't worth sacrificing clarity. Obviously sometimes you have to (Big O can be an unforgiving beast) but not always.
- Lio 6y agoIt worth remembering that ruby code already handles async, and threads. For example ActiveRecord database requests have been non-blocking for many years. This new scheduler interface just gives us a nicer and lighter weight abstraction for handling it rather than, for example, using OS thread.
- jashmatthews 6y ago> ActiveRecord database requests have been non-blocking for many years. That's blocking IO with multi-threading. The executing thread stops and waits.
- Lio 6y agoYes, poor choice of words on my part. The IO calls themselves still block but each is in its own OS thread and GVL releases when the thread blocks. If you can only do parallel IO in a thread why not use a light weight fibre (green thread) instead of an OS thread? Hence this new scheduler and fibres.
- jashmatthews 6y ago> each is in its own OS thread and GVL releases when the thread blocks. The GVL isn't automatically released when a thread blocks on IO. Each bit of native code performing IO has to explicitly release it like in the pg gem here: https://github.com/ged/ruby-pg/blob/fb465855ce1dd12cf7eb69c92af222052aa956ce/ext/gvl_wrappers.h#L4 https://github.com/ged/ruby-pg/blob/fb465855ce1dd12cf7eb69c9... If you don't need high concurrency then using threads is simpler.
- rdw 6y ago> all relevant standard library methods have been patched to yield to the scheduler whenever they encounter a situation where they will block the current fiber This is huge. They solved the function-coloring problem by deciding that all functions are now potentially async. It makes it more likely that the ecosystem as a whole actually becomes async-compatible. I wish Python had taken this approach, though I understand why they didn't.
- cogman10 6y agoThis is essentially the approach Java is taking with the upcoming loom project.
- Rapzid 6y agoI don't see how this solves the "coloring" problem. "Colored" functions have a different return type; a future of some sort. If you want to return a future to a caller, or have a method that blocks until the work is done, you still need a way to differentiate that. Golang does essentially the same thing and uses channels instead of "futures". However, some APIs will have functions that return a channel that you receive a message on later.. Which is a lot like a future. And if you provide a more simple non-channel API along side(instead of making the consumer wrap your calls in a Go routine or always use a channel), you now have "colored" functions. I believe this is just a solution to a scheduling problem(pre-emptive vs cooperative).
- chrisseaton 6y ago> "Colored" functions have a different return type; a future of some sort. This is an aside and possibly pedantic, but everyone seems to have forgotten that the whole idea of futures was originally that they weren't some different type. They were the same type, but blocked transparently on demand and had no `get` operation. The construct that came before futures, called eventual-values, had the different-type thing. The only difference of futures, the big idea, was they said 'maybe we can get rid of that part'. > These “futures” greatly resemble the “eventual values” of Hibbard’s Algol 68 [39]. The principal difference is that eventual values are declared as a separate data type, distinct from the type of the value they take on, whereas futures cannot be distinguished from the type of the value they take on. (Halstead, 1985) But now people write types like `Future[T]`... which completely misses the point! If you start to write `Future[...` then think to yourself... this isn't a future. And that's how you get this colouring problem you describe... it was already fixed and people have forgotten all about it.
- kyledrake 6y agoFYI, this domain is flagged as "dangerous" by mcafee's malicious site detection right now. Centurylink's nameservers are blocking it. You might want to talk with them about that.
- dochtman 6y agoHey WJW, nice blog you have there! Have been following along, good stuff.
- WJW 6y agoHey dochtman, thanks! Long time no see, hope everything is good in Rust-land. Gelukkig nieuwjaar!
- mojuba 6y agoCould someone explain if FiberScheduler can use all the available CPU cores?
- ioquatix 6y agoYes, you run one scheduler per core.
- dorianmariefr 6y agoCould somebody tell me what's wrong with threads, e.g. require "open-uri" require "json" results = [] (0..100).each_slice(10) do |slice| slice.map do |i| Thread.new do results << JSON.parse(URI.open("https://httpbin.org/get?i=#{i}").read)["args"] end end.each(&:join) end p results
- gray_-_wolf 6y agoWell the idea is that thread creation itself is not really cheap. For most software I would say it is not really a problem, but if the "work" to be done is short enough, that creation of thread becomes the most costly factor. True, that can be solved by thread pool usually. So I guess as with most things in software engineering, if there is something wrong with threads or not depends on your use case.
- tmpz22 6y agoYou can’t have a thread per http request.
- gmfawcett 6y agoOf course you can. In the days of CGI scripts, we forked a full process per request, never mind a thread. I'd wager that the great majority of Web apps (i.e., the not-terribly-busy ones) would scale reasonably well on thread-per-request.
- dperfect 6y agoCorrect me if I'm wrong, but this code would block if (for example) one single request in a batch of 10 takes significantly longer to complete. The 9 other threads are then doing nothing while they wait. Of course, you could implement a more robust algorithm for scheduling threads, but it sounds like an implementation of FiberScheduler would make the best use of a single thread's resources (and you could scale that to multiple threads as needed using something like a shared queue). Since fibers are lighter-weight, it's presumably more efficient to run parallel IO operations in a single thread, and only use more threads if your work requires it.
- smetj 6y agoInteresting! I'm happy to see Ruby isn't going the Python route ... It seems to look a lot how https://www.gevent.org/ https://www.gevent.org/ achieves async on Python, which I can only applaud ...