28 ms·

Async Ruby

• Toutouxc 5y ago
  There may be some huge piece I'm missing, but how exactly is the "starting multiple async tasks wrapped in blocks and waiting for them to finish at the end of the main Async block" different from "starting multiple threads wrapped in blocks and manually collecting them at some point"? I thought Ruby did release the GIL when a thread is blocked (waiting for IO etc).
  • rajangdavis 5y ago
    It looks like from a brief glimpse into the source code that the library is using Ruby Fiber's under the hood instead of Threads. From my limited understanding, the programmer has to be explicit about starting and resuming the work within a Fiber as opposed to the Ruby VM.
    • brunosutic 5y ago
      Yes, async is using fibers as a concurrency primitive. Async gem is starting fibers automatically. Fiber pausing and resuming is handled by event reactor (also called "event loop") from 'nio4r' gem. io_uring support with asynchronous File IO is also on the way.
      • rajangdavis 5y ago
        Awesome! Thanks for the article, can't wait to see this library progress!
      • Toutouxc 5y ago
        So the Async gem allows the programmer to fire "tasks" and wait for them to finish (the same way we can fire threads), but instead of OS-level threads (which is what MRI uses for Threads) it uses a new kind of Fibers, called "non-blocking Fibers", that are lightweight and don't use OS threads, like normal Fibers, but unlike normal Fibers they yield automatically to the scheduler when blocked (sort of like threads). Is this a correct-ish way to describe the current state of affairs?
        • brunosutic 5y ago
          I think you're generally on the right track. Small correction: fibers (blocking or non-blocking) never used threads in any way. Fibers are "stackful coroutines", another concurrency primitive. They're pretty hard to understand, and unfortunately I can't find a good explanation of "Ruby Fibers" that I can link to.
    • brunosutic 5y ago
      Ruby's Global Interpreter Lock (GIL) is applicable, and not "sidestepped" with Async.
  • brunosutic 5y ago
    - Async Ruby is much more performant than threads. There are less context switches, enabled by the event reactor. The performance benefits are visible in simple scenarios like making a thousand HTTP requests. - Async is more scalable, can handle millions concurrent tasks (like HTTP connections). There can only be a couple thousand threads at the same time. - Threads are really hard to work with - race conditions everywhere. Async doesn't have this.
    • Toutouxc 5y ago
      > race conditions everywhere. Async doesn't have this What does Async (Fibers underneath) do differently than normal Threads? Using threads to handle concurrent work doesn't immediately bring race conditions, unless the programmer explicitly creates them (accessing the same stuff from different threads). Fibers themselves AFAIK don't stop you from accessing the same stuff, aside from the obvious "the fiber code runs 'atomically' until it hits the next yield (which non-blocking Fibers take away anyway).
      • BiteCode_dev 5y ago
        I assumee unless you have a GIL, most operations on collections are not thread safe, but they are async safe, since context switching can only happen where you await.
        • gpderetta 5y ago
          That's true for stackless coroutines where you can only yield from the top, but I understand that ruby has full coroutines/fibers
      • brunosutic 5y ago
        > What does Async (Fibers underneath) do differently than normal Threads? There's a lot to say on this topic. - Threads implement "preemptive scheduling". A scheduler switches control from one thread to another every 10ms or so. A thread running the code may be ready for the switch or not. The ensuing race conditions are nasty. - Async + Fibers implement "cooperative scheduling". The currently running fiber (voluntarily) yields control to another fiber when it's ready. The result is there are no race conditions. There's so much to say about this, I'll blog about this in the future.
  • sizediterable 5y ago
    Yeah, it's unclear to me whether you'd have to "join" to get the result from an operation, or if you can have some form of "await" expression
    • Toutouxc 5y ago
      Apparently the tasks are "joined" automatically before the enclosing "Async do" block is allowed to finish.
  • Ginden 5y ago
    Threads use quite much more memory than coroutines. Spawning eg. 3 threads for each request, x 1000 requests per second would probably eat a ton of memory.
    • liuliu 5y ago
      Stackful coroutines (often used to implement colorless async) is the same except you can specify stack size. You can specify thread stack size yourselves as well, though no one does that. OTOH, growable stack is useful, as Go demonstrated.
  • Smaug123 5y ago
    Please, someone, correct me if I've misunderstood. The big difference appears to be that async Ruby does not merely give you an easy sugar to perform the sync-over-async antipattern you have described. The real innovation is that, as far as the user is concerned, Ruby is magically turning blocking methods into non-blocking ones.
    • ioquatix 5y ago
      That's pretty accurate.
    • nixpulvis 5y ago
      That's basically how I'm thinking of things as well. To illustrate a bit further, consider the following: Given a blocking method call `foo(x)`, I can make it non-blocking by wrapping it in a "thunk" as `λx.foo(x)`. Where things start to get interesting is when I add another method call `foo(x) + bar(x)`. Now to keep things "async" I need to transform the abstraction into something more like `λx.foo(x) + λx.bar(x)`, and have the `+` call dispatch both fibers and wait for them before performing its operation. Doing this automatically seems pretty cool, I'll have to think about this a bit more sometime.
  • burlesona 5y ago
    The difference is just that fiber overhead is lower so you can run more fibers than threads on a given system. Even though fibers have been around a while, people rarely used them because they are cooperative rather than preemptive, so you had to manually write the scheduling logic. Much easier to just use threads. I think the big breakthrough for Ruby Async is that fiber scheduler in Ruby 3.0 now makes it possible for the runtime to manage fibers in a less manual way, so you now get the lightweight option more easily. The Async gem seems to be wrapping all that up in a very nice interface so you can write simple code and get good concurrency without much effort.
  • DarkWiiPlayer 5y ago
    The difference only shows itself in the real world, when you do a bit more per thread/coroutine and end up mutating shared state. This is where threads can lead to race conditions, whereas coroutines will not (unless you're basically ask for it)
• veesahni 5y ago
  Does Falcon + Async allow a second request to be processed while the first is blocked on a network call? For example: * HTTP request 1 requires information for which we can't respond immediately.. so we hold the connection open and respond in a few seconds. * Can HTTP request 2 come in at the same time? EventMachine enabled the above with what they called "streaming responses"
  • brunosutic 5y ago
    Yes, this is possible! In fact this is a perfect use case for Falcon + Async.
    • veesahni 5y ago
      I just tried this out. Falcon with count=1 & sinatra. Worked perfectly. It seems every request is processed in an async block so literally no special code is required. A request waiting on network will allow others to go through. This is awesome! :)
  • Matthias247 5y ago
    That seems mostly like the question of "Does this HTTP framework support HTTP pipelining". While I don't know the answer, it doesn't seem highly relevant. Most clients went away from using pipelining, since follow-up requests on the same connection are subject to unknown latency (stuck behind the first request) and a connection failure can impact all of those requests. The better approach is to use either more connections, or proper request multiplexing via HTTP/2 or /3. In the latter case a server framework would just see multiple request invocations in parallel.
    • polynox 5y ago
      > subject to unknown latency (stuck behind the first request) Also known as "head of line blocking" https://en.wikipedia.org/wiki/Head-of-line_blocking https://en.wikipedia.org/wiki/Head-of-line_blocking
    • jrochkind1 5y ago
• rememberlenny 5y ago
  This seems huge. Does anyone know plans around how this will be implemented into Rails?
  • hiyer 5y ago
    From reading the article it seems to me that you could just wrap any Rails method (say a DB read) in an Async block to make it run asynchronously.
    • rajangdavis 5y ago
      Also says at the end: The only caveat is that it doesn't work with Ruby on Rails, because ActiveRecord doesn't support async gem. You can still use it with Rails if ActiveRecord is not involved.
      • hiyer 5y ago
        Oh, sorry I missed that. Hopefully it will show up in Rails 7 then.
        • rajangdavis 5y ago
          You're not wrong. If you have external services that are accessible via HTTP, there is some potential to get some performance gains there. You're just limited in terms of some of the core Rails utilities.
      • DangitBobby 5y ago
        I don't understand. Is this misleading then? Did HTTParty have to implement Async for their example to work? > You probably have your preferred HTTP gem, and you may be asking "will it work with Async"? To find out, here's an example using HTTParty, a well-known HTTP client. They then go to show that your "preferred HTTP gem" will just work. How is the ActiveRecord situation different?
        • 5y ago
• sizediterable 5y ago
  > any blocking operation (a method where Ruby interpreter waits) is compatible with Async and will work asynchronously within Async code block with Ruby 3.0 and later. That's pretty magical. Does this mean it would be possible to implement structured concurrency [0], but without the function coloring problem? Regardless, I think I prefer how this looks from a code read/writeability perspective compared to Zig's or Swift's (and potentially future Rust's [2]) approaches. [0] https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful https://vorpus.org/blog/notes-on-structured-concurrency-or-g... [1] https://kristoff.it/blog/zig-colorblind-async-await/ https://kristoff.it/blog/zig-colorblind-async-await/ [2] https://blog.yoshuawuyts.com/async-overloading/ https://blog.yoshuawuyts.com/async-overloading/
  • oezi 5y ago
    I am actually not a 100% how far the compatibility shows. When I test Async gem with httparty and open-uri, there is no speed-up when comparing with threads: https://gist.github.com/coezbek/07061fb7de2b036bf5c69f8a8b8cc3e5 https://gist.github.com/coezbek/07061fb7de2b036bf5c69f8a8b8c... I might be using it wrong...
    • oezi 5y ago
      Ha! This only works with Ruby 3.0, because only since then are standard library function using async-compatible methods internally.
  • brunosutic 5y ago
    Async Ruby is colorless! It's obvious from the examples provided in the post. For example, you can use the method `URI.open` both synchronously and asynchronously. It's something I didn't want to mention in the article because it's a relatively advanced async concept, but yea - Async Ruby is already colorless and it's great.
    • gpderetta 5y ago
      IIRC ruby has first class continuations, so it doesn't have to deal with any async/await nonsense.
• Mikeb85 5y ago
  This gem is almost a decade old (although looks like it's been updated to work with fibers?). There's a bunch of gems that allow for asynchronous/concurrent and/or parallel execution for Ruby. It looks like a nice enough gem but it's not particularly novel. Seems like it's just trying to ride JS' popularity with the name. Ruby threads also allow for non-blocking operations. Edit - this isn't some new paradigm for Ruby. It's a gem... A bunch of others ("Parallel", "Concurrent-Ruby", "Eventmachine", etc...) have been around forever and do similar things. Edit2 - https://rubygems.org/gems/async https://rubygems.org/gems/async
  • brunosutic 5y ago
    > This gem is almost a decade old Hm, the first commit to the gem was in 2017. The gem was not "advertised" so that the interfaces can be polished and done right. Async gem is hugely improved with Ruby 3.0 release (from December 2020) when Ruby language added "fiber scheduler" feature just to integrate better with Async gem.
    • Mikeb85 5y ago
      Rubygems.org has the first commit at 2013. Unless the gem changed ownership.
      • brunosutic 5y ago
        > the gem changed ownership I think the gem did change ownership at some point. The first couple gem releases from 2013 were yanked.
        • Mikeb85 5y ago
          Different author credited but the same owner the whole time. So might be different. The library that async is based on is the same owner as well. Edit - the owner in 2013 is the same as the owner of the current gem and main contributor on Github. So yes it seems like it's been around since 2013.
• claudiug 5y ago
  another cool ruby library for async is https://github.com/digital-fabric/polyphony https://github.com/digital-fabric/polyphony
  • brunosutic 5y ago
    Polyphony monkey-patches Ruby core methods... this has always prevented me from experimenting more with it. Source link: https://digital-fabric.github.io/polyphony/faq/#why-does-polyphony-reimplements-core-apis-such-as-ioread-and-kernelsleep https://digital-fabric.github.io/polyphony/faq/#why-does-pol... > Polyphony “patches” some Ruby core and stdlib APIs, providing behavioraly compatible fiber-aware implementations
  • ksec 5y ago
    I think this deserves more attention, especially it is coming from the original author of sequel. >Polyphony is a library for writing highly concurrent Ruby apps. Polyphony harnesses Ruby fibers and a powerful io_uring-based I/O runtime to provide a solid foundation for building high-performance concurrent Ruby apps. https://noteflakes.com https://noteflakes.com
• continuational 5y ago
  That seems kinda verbose to use. It's also a bit suspicious that all the examples discard the results; is it hard to get the results back out?
  • Spivak 5y ago
    Just tried it out. Async blocks evaluate to Task objects which have a wait method and a result attribute which evaluates to the value of the block. require 'async' res = Async do |task| name_task = task.async do sleep 2 "Jenny" end task.async do sleep 5 9 end "Hello #{name_task.wait}" end puts res # => "Hello Jenny" after 5 seconds. After using Python's and JS's async implementations this seems beautiful by comparison. Here's the a rough Python equivalent. import asyncio async def get_name(): await asyncio.sleep(2) return "Jenny" async def get_number(): await asyncio.sleep(5) return 7 async def main(): number_task = get_number() name_task = get_name() name, _ = await asyncio.gather(name_task, number_task) return f"Hello {name}" print(asyncio.run(main()))
    • derefr 5y ago
      Also, it looks like there's a getter for the "current" task (https://socketry.github.io/async/guides/getting-started/index.html#nested-tasks https://socketry.github.io/async/guides/getting-started/inde...), sort of like Thread.current. Which means that you can just write a whole normal Ruby program, that just uses Task::Async.current.async(...) wherever it likes to schedule subtasks (sort of like calling spawn/3 in Erlang), and then treat them as regular futures, even returning the future out of the current lexical scope without thunking it; and then have exactly one Async block at the toplevel that kicks off your main driver logic and then #wait s on it. All without having to pass the current task down the call stack everywhere. (And if you want to schedule a bunch of stuff to happen in parallel and then wait for it all to be done, but you're below the toplevel Async block, you'd do that by scheduling the subtasks against an Async::Barrier: https://socketry.github.io/async/guides/getting-started/index.html#waiting-for-multiple-tasks https://socketry.github.io/async/guides/getting-started/inde...)
• michauu 5y ago
  Could anyone comment on the thread safety? What's shared between async blocks and what's safe to modify?
  • veesahni 5y ago
    Multiple async blocks aren't really running concurrently because GIL is still in play. Control of what's running only switches if one async block yields... manually, or automatically. What's new is Ruby is auto-yielding when waiting on network or other blocking actions.
• jmarchello 5y ago
  I can't wait to play with this! Having worked with Python's Asyncio a lot, this looks SO MUCH easier to work with.
• BMorearty 5y ago
  This gem had a huge impact in May of this year on one of Airbnb’s Ruby services. Last year when I was still at Airbnb I did the research and foundational work to make Airbnb’s HTTP client gem compatible with the Async gem and Falcon server. I’m no longer there but this year, one of my friends who’s still at Airbnb put that work to use on Airbnb’s Ruby service that talks to a Genesys API. Most Genesys API calls have reasonable response times but there are these occasional, weird, unpredictable 30-second response times. The slow responses can come in bursts, where you can a bunch of them at once. This makes it extremely hard to predictably scale the Ruby service that calls it, because in normal synchronous mode, every thread that calls Genesys is blocked and on longer available to take more client requests. You could scale up the service to assume these 30-second response times will happen all the time, but then you’ve got huge server costs just for the 1% case. But using Async and Falcon (an Async-compatible HTTP server), Airbnb was able to safely scale down the service by around 85% and turn on autoscaling, and it’s working beautifully.
  • maxpert 5y ago
    I would love to learn a little more here. So for 30 second tail latency cases, doesn't Ruby HTTP client support a timeout with cancellation? The only useful case where I believe Async Ruby can help is the parallel requests cases. Disclaimer: I am not a full time Ruby person, so if it sounds naive please feel free to point it out.
    • kayodelycaon 5y ago
      Ruby’s built in timeout library creates another thread with a sleep call. If something is blocking on a socket in the C code, the timeout doesn’t work.
      • insensible 5y ago
        Right, don’t use that, but there are well-implemented timeouts in network libraries, etc.
      • ioquatix 5y ago
• faebi 5y ago
  How will this deal with temporary CPU starvation in long running connections? I am currently using concurrent-ruby's thread pool to run a few hundred Net-SSH Connections per process at the same time. This helped me to avoid most of these kind of issues.
• matheusmoreira 5y ago
  > you can think of it as "threads with none of the downsides" And likely no actual parallel execution of Ruby code. I suppose fibers are scheduled for later when they perform I/O. Just like how C extensions release the interpreter lock before some expensive function, allowing other Ruby code to run in concurrently.
  • ric2b 5y ago
    Ruby threads don't execute in parallel either, because of the GIL.
  • dragonwriter 5y ago
    > And likely no actual parallel execution of Ruby code. Async mentions “Multi-thread/process containers for parallelism.” https://socketry.github.io/async/index.html https://socketry.github.io/async/index.html These seem to be provided by a companion project: https://socketry.github.io/async-container/ https://socketry.github.io/async-container/
    • matheusmoreira 5y ago
      > Multi-thread/process I thought so... Threads and processes are still in use. While yielding control on system calls is a great idea, it's a bit innacurate to say it's just like threads with none of the downsides. The upsides are missing too. Only the system calls are running in parallel here. The Ruby code remains single-threaded. Still, much better than languages that lack schedulers where people are tricked into writing cooperatively scheduled code without even realizing it.
      • jrochkind1 5y ago
        In MRI (aka "CRuby", standard ruby), you don't get parallelism with threads either, due to the GIL. (Similar to CPython). A lot of people are used to thinking mostly of that scenario, where ruby doesn't currently give you parallism for threads either. JRuby does give you parellism for threads -- but I believe currently doesn't actually implement true coroutines (Fibers are just implemented on top of threads), so there are now two reasons this wouldn't be as useful on JRuby, true.
• jasonhansel 5y ago
  No function colors! Is this like Go, where each fiber maintains a separate stack at runtime, or is this like Rust, where each task is effectively transformed into a state machine?
  • pansa2 5y ago
    Like Go, except there's no parallelism (and no preemption). I think Rust's approach requires colored functions.
    • deleted 5y ago
      [deleted]
• midrus 5y ago
  I stopped using python long ago, out of the frustration with the breakage caused in the ecosystem first due to the 2/3 transition, and then by splitting again the ecosystem with all the async stuff they introduced, when having instead gone the way of gevent would have ended up in something like this. The company I was at the time tried really hard to move to python 3. It was a huge codebase with thousands of engineers. After more than a year of work they cancelled the project and said they'll never upgrade to python 3, and instead started writing smaller "microservices" around the core product just to calm engineers anxiety to use python3. Now Django is suffering kind of the same problem with all the async stuff being bolted on, adding a lot of gotchas and side effects and incompatibilities around every corner. It is a mess. I left that company and I will never use python again. I love how Ruby seems to care about the ecosystem, and both ruby and rails keep innovating and providing awesome full stack solutions.
  • lpapez 5y ago
    Was it really Python's fault? Version 2 was deprecated for a decade basically, and any sensible company invested the time to migrate.
    • midrus 5y ago
      I'm talking about many years ago (6 more or less). And this company has a HUGE codebase. It has its managements problems too, but the situation I'm talking about didn't make it any easier.
    • rictic 5y ago
      Yes. The 2->3 change was not engineered to make migration easy, and as a maintainer of a large and project that people have built on top of this is something that both current and potential users will judge you on. Not speaking from a place of scorn, I and my team have made this same mistake, we lost both users and momentum. Big learning experience.
  • vbg 5y ago
    Whether or not a software project succeeds is substantially influenced by politics. When I read your post a strong anti python 3 feeling is conveyed. It may well be that it’s impossible to upgrade such a company/code base when the engineers have such a negative attitude. The barrier was perhaps not technical but about the attitude of the people. There were lots of python 2 developers who were rabidly anti python 3. Some of the most well known python developers were aggressively very publicly anti python 3. Imagine a python 3 upgrade project with that attitude prevailing. No chance of success. Today python 3 is more popular than ever before and is arguably the most popular language in the world after javascript. Lots of people love python 3.
• pinzonjulian 5y ago
  Great article! I was wondering if you could add an extra section on why Async will make the @counter error go away. I'm new to Async and Threads in general and the rest of the article was super easy to follow but I didn't quite understand why or how that error would disappear.
• pinzonjulian 5y ago
  Great article! I was wondering if you could add an extra section on why Async will make the @counter error go away. I'm new to Async and Threads in general and the rest of the article was super easy to follow but I didn't quite understand why or how that error would disappear.