11 ms·
Fibers are the right solution to improve Ruby performance
- coder543 8y ago> fibers can context switch billions of times per second on a modern processor. On a 3GHz (3 billion hertz) processor, you expect to be able to context switch billions of times per second? I would probably accept millions without question, even though that might be pushing it for a GIL-ed runtime like Ruby has. But, unless your definition of "context switch" counts every blocked fiber that's passed over for a context switch as being implicitly context switched to and away from in the act of ignoring it, I find this hard to believe. It takes more than one clock cycle to determine the next task that we're going to resume execution for and then actually resume it.
- devxpy 8y agoWould love to see some numbers supporting this.. He might have mis-phrased it though. Maybe he meant to say that it can handle billions of concurrent tasks maybe not that they can switch rapidly in just a second xD
- fake-name 8y agoI can only assume they're talking about the entire cpu (as in, all 4+ cores in a single die) rather then a single logical core in the hardware. It's sloppy wording either way.
- ioquatix 8y agoHi, I'm the author, and I agree, so since everyone is so interested I'll check my original benchmark and fix the article. Update: I wrote an addendum https://www.codeotaku.com/journal/2018-11/fibers-are-the-right-solution/context-switching https://www.codeotaku.com/journal/2018-11/fibers-are-the-rig...
- erdewit 8y agoIt must be off by 3 of 4 orders of magnitude. Python with asyncio does about 120_000 switches per second with a trivial fibre/coroutine here: import time import asyncio REPS = 1_000_000 async def coro(): for i in range(REPS): await asyncio.sleep(0) t0 = time.time() asyncio.run(coro()) dt = time.time() - t0 print(REPS / dt, 'reps/s')
- blattimwind 8y agoasyncio eats about one order of magnitude. A more minimalistic async-approach gives something on the order of ~900000 resumptions per second. (Single-thread hardware performance differences would not account for a factor 10)
- blattimwind 8y agoSlightly modified "benchmark" from above: asyncio: import time import asyncio REPS = 1_000_000 async def coro(): for i in range(REPS): await asyncio.sleep(0) t0 = time.perf_counter() asyncio.run(coro()) dt = time.perf_counter() - t0 print(REPS / dt, 'reps/s') ~180k asynker (https://github.com/enkore/asynker https://github.com/enkore/asynker): import time import asynker REPS = 1_000_000 async def coro(): for i in range(REPS): await asynker.suspend() t0 = time.perf_counter() sched = asynker.Scheduler() sched.run_until_complete(coro()) dt = time.perf_counter() - t0 print(REPS / dt, 'reps/s') ~900k The excellent curio (https://github.com/dabeaz/curio https://github.com/dabeaz/curio): import time import curio REPS = 1_000_000 async def coro(): for i in range(REPS): await curio.sleep(0) t0 = time.perf_counter() curio.run(coro()) dt = time.perf_counter() - t0 print(REPS / dt, 'reps/s') ~180k Not that this number is hugely important.
- wenc 8y agoWould uvloop change these benchmarks? https://github.com/MagicStack/uvloop https://github.com/MagicStack/uvloop
- tomp 8y ago> It takes more than one clock cycle to determine the next task that we're going to resume execution for and then actually resume it. Not really, if you just implement a round-robin scheduler and if none of the fibres are actually blocked (i.e. if they just yield). Artificial benchmark? Sure. But there's no way you could ever approximate this with OS threads, even if the user-space code/threads themselves do no useful work.
- wahern 8y agoThe difference between stackful coroutines (i.e. fibers in this case) and Python-style coroutines is that Python's coroutines need to rebuild the stack on every resume, whereas resuming a fiber is basically a goto. The cost of yielding and resuming a coroutine in Python is O(N) for the depth of the call chain, but O(1) for fibers. So as you say, a simple scheduler that just walks a linked-list could resume a stackful coroutine in a few instructions, plus the cost of restoring hardware registers (which is the same cost as a non-inlined function call), and the latter is easily pipelined so would take fewer cycles than instructions.
- tomp 8y agoIndeed, a highly-optimized scheduler is basically the same as looping through an array and calling virtual methods of each object. Extremely cheap.
- snek 8y agotl;dr context switching gives good perf
- devxpy 8y agoFibers are basically just async tasks with multicore support, right?
- coder543 8y agoFibers are basically just async tasks where each task looks and feels like a normal, synchronous thread to the programmer. It doesn't really imply anything about how parallel (or not) the fibers execute.
- devxpy 8y agoSo it is preemptively switching then? > It doesn't really imply anything about how parallel (or not) the fibers execute What's all the fuss about paralllelism in the article about then?
- rubyn00bie 8y agoNo they aren't preemptively switching/scheduling, you have to schedule it yourself... I could be wicked wrong though, it's been a long while since I futzed with fibers.
- coder543 8y ago> So it is preemptively switching then? Any time you call a blocking function that the system provides, it should immediately yield the fiber, which makes it look preemptive. If you write a loop that just spins forever, that could block the whole system, potentially, making the abstraction leaky. In a language like Ruby, they could definitely add some true preemption, but I don't know if that's what they plan to do. From the article: "Fibers, on the other hand, are semantically similarly to threads (excepting parallelism), but with less overhead." So, the author definitely isn't implying parallelism of the fibers. > What's all the fuss about paralllelism in the article about then? The author was talking about different methods for handling more than one request at a given time, which include forking and threads. With Ruby's GIL, threads are a lot less attractive than they could be. A good fiber implementation can handle tons of network requests concurrently and very efficiently even on a single core, which is the case being discussed here. At the end, the author discusses a hybrid approach of forking and fibers, where each processor core would have a fork of the Ruby program running, and each fork would have its own fiber pool, running many tasks concurrently. In languages that don't have a GIL, forking is rarely a tool that I reach for. It really hurts your database pooling and all sorts of other small problems, but it's a common trade-off when using Ruby, Python, and Node.
- mfourcade 8y agobeware before using fibers because it won't access Thread.current[:vars]. So if you plan on going with fiber on an existing app, double check first if Thread.current[:vars] is a must have for you & you dependencies (ex: the I18n gem
- spariev 8y agoAre you sure about that ? If you search `Ruby Thread Locals are also Fiber-Local` there will be a blog post from 2012 about that, and the code sample works fine for me on ruby 2.5
- mfourcade 8y agoI'm sorry, my previous comment was not clear. I should have written beware before using fibers because it won't access Thread.current[:vars] _as you might expect_. Here is an example: fiber = Fiber.new do # Thread.current locals are copied to the Fiber when fiber it is built puts "1. #{Thread.current[:test]}" Thread.current[:test] = false # the fiber has it's own stack, won't leak away puts "2. #{Thread.current[:test]}" end Thread.current[:test] = true fiber.resume puts "3. #{Thread.current[:test]}" Output: 1. 2. false 3. true So fibers comes with their _own stack_, including threads locals, yes, but from _when_ you instantiated them. Not from Thread.current :/ Also writing Thread.current[] won't apply outside the Fiber. I was confused, and not alone: https://github.com/rails/rails/pull/30361 https://github.com/rails/rails/pull/30361. So my conclusion about Fiber is always be careful with Fiber / Thread.current. edit: style/explanation
- Tharkun 8y agoAre these similar in implementation to the upcoming Java Fibers? Those basically yield automatically whenever they perform a blocking operation, so another fiber can be executed.
- aardvark179 8y agoRuby fibers yield control to the fiber that spawned them or transfer control to another specific fiber explicitly. That's close to the Continuation model which Loom's fibers are built on top of, but Loom's fibers hide that process of yielding from the user, instead concentrating on providing a more general scheduling API and yielding at certain points inside the core library. There are quite a lot of other small differences as well since Ruby fibers are always run on the same thread which originally spawned them, whereas Loom's continuations and fibers can be resumed on a different thread. It is quite possible to implement Ruby fibers in terms of Loom's continuations, and we've done a prototype of this in TruffleRuby (I think Charles Nutter has done one in JRuby as well), and it certainly allows us to support very large numbers of active fibers.
- adamnemecek 8y agoCooperative switching hasn’t been this cool since 1985! /s
- mikekchar 8y agoI've been trying hard not to be snarky :-) It's pretty amazing how old things become new as a combination of 1) people not knowing about it and 2) it being given a new name. The first one is a bit sad and speaks volumes about our collective lack of education as an industry. I gladly include my self in that indictment by the way. I only console myself with the fact that the things I don't know (but should) is probably greater than what could fill my (mostly useless) 4 year CS degree. Of course I graduated a long time ago... I'm a bit worried that as people get into this industry in less traditional ways we are gradually losing the sight of the fact that there really is something important to learn.
- pjmlp 8y agoModula-2 co-routines FTW! :)
- srean 8y agoI am glad that fibers and coroutines are going mainstream the way it is. With sincere and profuse apologies to Ruby fans (lest they think this is an attempt at hijacking the discussion) let me share this https://felix.readthedocs.io/en/latest/fibres.html https://felix.readthedocs.io/en/latest/fibres.html I feel some HN'ers would be curious as there arent many languages that offer fibers as a first class citizen. BTW I am in no way involved in the development of the Felix. Its not a new language, its more than 15 years old and has had fibers from the start.
- deleted 8y ago[deleted]
- ridiculous_fish 8y agoHow does context switching occur wrt the native stack? Does ruby avoid using the native stack, or does this save and restore it?
- exabrial 8y agoTypes are the right solution to fix Ruby performance. Crystal is showing how a language can be simple and performant!
- theredbox 8y agoCrystal suffers from other problems. They diverged too much from type annotations and use macros / metaprogramming too much which immensely increases the compile times.
- bjz_ 8y agoI don't know about Crystal's preformance problems specifically, but I would be surprised if macros would take up much compile time. I would have guessed it was more to do with the fiendishly tricky type inference problem that they set for themselves (ie. global inference + subtyping + union types + inheritance + overloaded methods).
- chrisseaton 8y agoThe problem isn’t tricky it just gets very large very quickly.
- alehander42 8y agoMetaprogramming is very fast, this is not a problem: very rarely macros can be the bottleneck
- Maledictus 8y agoRuby has types. Not everyone enjoys working with statically typed languages.
- weberc2 8y agoYou’re still working with types, you just don’t have anything checking your work (except the tests you write). Controversial opinion, but I think many of the folks who don’t like types really just don’t want to be bothered about the bugs they’ve introduced off the happy path. This is based on my experience in a Python shop.
- vemv 8y agoI'm a bit hesitant about this article's proposition, although I'm open to be convinced. In the end there's a GIL. For pure CPU-bound workloads, your only true source of parallelism will be putting more processes into the mix, be it via forking or simply spawning more processes from scratch. But inside a given a process, it seems to me that no matter what you do (fibers/threads), only 1 CPU can do CPU-bound work at a time. If I was to design a high-performance Ruby solution, I'd ditch forking, threading and fibers. I'd focus instead in creating low-memory-footprint, fast-startup processes (read: no Rails!) that one can comfortably spawn in parallel, be it in a single server, or in a cluster (think k8s). Max Ruby processes count: 1 per core.
- weberc2 8y agoPresumably fibers address IO bound performance.
- repsilat 8y agoMostly Ruby web app workloads aren't CPU-bound. And when they are, you're using a language that is 100 to 1000 times slower than C. That is, for a CPU-bound task, you might be able to replace your thousand node k8s cluster with one machine running code written in a faster language (before even getting into communication and load balancing and HA and all that crap.)
- timdorr 8y agoBut abandoning everything you've written in one language for the sake of a single hot path isn't really a good idea either. It's probably a more sensible idea to use a native extension to optimize just that hot path. For instance, the Rust bindings are really good: https://github.com/tildeio/helix https://github.com/tildeio/helix
- nine_k 8y agoCrystal or Elixir allow for relatively less painful migration to high(er) performance overall. But this is beyond the scope of the article.
- magwa101 8y agoI've coded multiprocess, multithread, callback and context switching on multiple OSes and languages. Golang nails it, a bed of green threads scheduled onto CPU threads with isolation. The foundation is correct.
- ioquatix 8y agoIt's the perfect approach except users must deal with mutual exclusion which is tricky for most developers.
- evantahler 8y agoAnd ruby become node!
- bhuga 8y ago> We show that fibers require minimal changes to existing application code and are thus a good approach for retrofitting existing systems. I disagree that 'minimal changes to existing code' is a good goal for a Ruby parallelism solution. The large Ruby codebases I have dealt with have gigantic hacks to get around the GIL: complex fork management systems, subprocess management, crazy load-this-but-not-that systems for prefork memory management to optimize copy on write, and probably more. Parallel tasks in Ruby are a nightmare to work with. Changing existing code _should_ be a goal of any Ruby parallelism solution. If we can't get rid of this kind of cruft, what are we even doing? I still love Ruby, but I want go-style channels, not apologies for the GIL.
- olivierlacan 8y agoThat’s the plan with Guilds: http://www.atdot.net/~ko1/activities/2018_rubyconf2018.pdf http://www.atdot.net/~ko1/activities/2018_rubyconf2018.pdf These are the slides for Koichi Sasada’s RubyConf 2018 (last week) talk updating the community on his progress in the design and implementation of Guilds. Surprising that the author isn’t aware of this planned CRuby feature announced in 2016. I wrote about it then: https://olivierlacan.com/posts/concurrency-in-ruby-3-with-guilds/ https://olivierlacan.com/posts/concurrency-in-ruby-3-with-gu... Take this with a grain of salt of course because the design has clearly changed since then and I need to upgrade my post or do a follow-up.
- ioquatix 8y agoHi, I'm the author, I'm also a Ruby core committer, and yes I'm aware of Guilds. The are a model for parallelism and don't really affect concurrency at all. At best, they provide a 3rd option on top of processes/threads/guilds.
- olivierlacan 8y agoThat makes sense. Sorry for the assumption. Guilds felt omitted in your post since they (should) address one of the points you make about the usability/ergonomics of existing Ruby APIs for managing non-sequential execution. But it’s definitely a bit early to tell what Guilds will actually look like as a final product.
- karmakaze 8y agoTitle is actually "Fibers are the Right Solution" and the article ends with "Fibers are the best solution for composable, scalable, non-blocking clients and servers." The title as posted is most definitely not true. I have worked on high volume Ruby applications where the problem with using async I/O combined with poor raw execution of Ruby code resulted in excess garbage to be collected. Ruby performance should mean the performance of executing Ruby code. Perhaps the title could have been 'Ruby webapp performance'.
- ioquatix 8y ago> I have worked on high volume Ruby applications where the problem with using async I/O combined with poor raw execution of Ruby code resulted in excess garbage to be collected. What async I/O approach did you use and where did the garbage come from? I/O buffers?
- karmakaze 8y agoIt was an api endpoint making a number of db and other network requests. The garbage was the accumulation of temporary and response objects that were held for the long duration of processing, up to half of which could be in the ruby code not waiting on i/o. If the code ran faster there would be more time for gc and handling the incoming request volume.
- xutopia 8y agoI've been using Ruby and Rails for about 12 years now and I see all the solutions people come up with as bandages over a wound. A normal Rails app starts off with Node and Redis installed. If like me you hop onto the Elixir world it becomes refreshing to see the language you are using do all the stuff multiple tools did and doing it faster.
- mamcx 8y agoBTW, a easy explanation in how implement this for a language? I asked in https://www.reddit.com/r/ProgrammingLanguages/comments/9xsyad/how_provide_in_an_interpreter_iteratosgenerators/ https://www.reddit.com/r/ProgrammingLanguages/comments/9xsya... and get a nice answer but is also of the style "just read this code and get it".
- zeveb 8y agoSeems to me that one interesting approach would be to automatically rewrite a program in continuation-passing style, and then use callbacks (the lowest-overhead approach) where the callback is the continuation. It ought to compile pretty cleanly, since a continuation can end up being compiled into an address.
- ioquatix 8y agoI've thought about this too, and it might be the route C++ takes. That being said, to resume you need to wind through all your function calls executing parts of the state machine, it might actually be slower. That being said, I'm sure there are different ways to implement it and I look forward to proposed faster/efficient implementations.
- kljdlwjfkewe 8y agoI'm not trying to be snarky but writing a performant server or web server in userland Ruby doesn't seem to be good engineering. We already have nginx & apache. If you want to write performant servers you probably want something like Go, Rust, C++, C, D, JVM, BEAM.