4 ms·
Polyphony: Fine-Grained Concurrency for Ruby
- artyom 4y agoI'd love a performance comparison against concurrent-ruby on the Readme. That's assuming Polyphony is aiming to replace most or all of it.
- ciconia 4y ago> That's assuming Polyphony is aiming to replace most or all of it. Not really. Concurrent-Ruby implements a lot of different concurrency patterns. Polyphony has a much smaller scope, though I imagine you could reimplement a lot of concurrent-ruby on top of Polyphony. I think there's a place for the two in the Ruby ecosystem.
- jupp0r 4y agoWithout support from Rails, all of these libraries unfortunately can't be used by 99% of Ruby code out there and using Ruby in any sort of fan out architecture remains a pain (ie almost impossible in practice).
- zeckalpha 4y agoI don't see why you couldn't use Polyphony inside a Rails app.
- rafamvc 4y agoBecause eventually a dependency will make it non concurrent. Rails is the driving force for adoption in the ruby ecosystem. If it isn't a rails requirement to be concurrent friendly, it will be broadly ignored.
- jupp0r 4y agoThis is where the "in practice" part comes in. Ruby culture thinks it's a great idea to use global mutable state in lieu of request local state all over the place because the code looks so beautiful if you don't have all of these parameters and just grab stuff from the ether. So unless all of your dependency tree (three quarters of which probably had its last release in 2017) fixes their usage of global variables, you won't have much fun even if Rails itself got rid of their global variables (to be fair, they've done a great job over the last year or so).
- lloeki 4y ago> Ruby culture thinks it's a great idea to use global mutable state in lieu of request local state all over the place because the code looks so beautiful if you don't have all of these parameters and just grab stuff from the ether That is utterly incorrect. Make it s/Ruby/Rails and you'd be somewhat more correct. Cue Rack, Sinatra, Hanami, Roda, and that's just web "frameworks". > all of your dependency tree (three quarters of which probably had its last release in 2017) I don't see how that could make sense. Some libraries can be mature and thus not warrant much, if any, update. Also, the aforementioned pattern is only present on a small subset of Ruby gems, usually ones that are close in mindset to Rails. Nothing prevents anyone to code in Ruby in a functional-like style, nor pick dependencies that are written with such sensibility. But if you wilfully pick Rails you can't really expect to veer too far from the philosophy that comes with it.
- ciconia 4y agoI haven't really had time to explore the usage of Polyphony in Rails apps. Personally, I don't use Rails in my work, and instead develop custom Ruby apps. I believe there's a place for Ruby development work without any connection to Rails.
- lloeki 4y agoThere is a lot more to Ruby than Rails. I can see it being useful for many projects, web or non-web.
- rgbrgb 4y agoVery cool! Small rec would be to pull up a few things from the linked docs into the main readme: - code examples - motivation for building - what I should try this for vs the alternatives
- ciconia 4y agoFurther work on the Polyphony docs is in the pipeline.
- lukegru 4y agoDoes this use the Fiber Scheduler API to figure out which fibers are blocking? If so, why does this need libev - I'm guessing it's to do with performance?
- rubiquity 4y agoFiber::Scheduler is only an interface. You still need an IO multiplexer underneath it.
- Alifatisk 4y agoThis is not what I don’t understand with the Ruby Team, why did they not ship Fiber Scheduler with an implementation and just an interface? Dis they expect everyone to implement the scheduler om their own? I am aware of the existing fiber scheduler implementations you can get on Ruby Gems but there should be official implementation.
- ciconia 4y agoNo, Polyphony has a design that diverges from the model proposed by the FibeScheduler API. Note that the FiberScheduler API included with Ruby is only an interface, not an implementation. Polyphony uses io_uring on recent Linux kernels, or libev otherwise to perform I/O operations and implement timeouts.
- jeffreygoesto 4y agoExcellent explanation and very well thought out. Can we have something similar for C++ and Rust please?
- samsquire 4y agoThank you for this. I am interested in how concurrency can be represented elegantly and efficiently, so I am interested in how libraries can simplify async and make it easier to reason about and write What kind of code do you want to be capable of writing? Any imaginary syntax you can think of? The libev and ioring support is great for IO scalability (https://github.com/enki/libev https://github.com/enki/libev not sure if this is the official repo) In Python I use the "select" module and use epoll on Linux. I am currently thinking of designing an API that allows the registration of epoll-like listeners to arbitrary objects, including business objects, so you can efficiently register a listener on multiple behaviours of multiple arbitrary objects. I wrote an async/await simulation in Java and my scheduler is really simple, it's just a for loop that checks to see if there are any tasks that can progress. I notice the switch_fiber in polyphony must do something similar. This is similar to a yield in a coroutine. My async/await simulation takes the following program: task1: while True: handle1 = async task2(); handle2 = async task3(); print(await handle1) print(await handle2) task2: n = 0 while True: yield n++ task3: n = 0 while True: yield n++ I had someone on HN explain to me how async works in Rust, but I've not written enough Rust to truly understand async in rust. https://news.ycombinator.com/item?id=33592334 https://news.ycombinator.com/item?id=33592334
- ciconia 4y ago> I am interested in how concurrency can be represented elegantly and efficiently This was the main driving force behind the development of Polyphony. I've been exploring all the different ways of expressing concurrent operations: callbacks, promises, async/await. Each of these solutions has its drawbacks: callbacks make you split your app logic across different functions; promises are a somewhat better API but do not solve the split logic problem; async/await potentially leads to two types of functions (the "what color is your function" problem). All three kind of break Ruby's exception handling model. In contrast, Polyphony relies on Ruby fibers, which provide a base for implementing cooperative concurrency, where each fiber does its work, yielding execution whenever it performs a blocking operation, such as reading from a socket, or waiting for a timeout. In addition, fibers are arranged into a tree where each fiber is considered the child of the fiber from which it was spawned, and is guaranteed to stop executing before its immediate parent stops executing. Any uncaught exception raised by a fiber will be propagated up the fiber tree. This implements a pattern known as Structured Concurrency [1]. Fibers can communicate by sending each other messages, in a similar fashion to Erlang/Elixir processes. There's no direct equivalent in Polyphony to the program you included, which seems to implement two generators running asynchronously. But the following does something similar. def number_generator n = 0 loop do client = receive client << (n += 1) end end task1 = spin { number_generator } task2 = spin { number_generator } 10.times do task1 << Fiber.current puts "next number from task1: #{receive}" task2 << Fiber.current puts "next number from task2: #{receive}" end So in the above program, two fibers are spun, each running the same number generator, which repeatedly waits for a message which is actually the fiber of the "client", and sends the next number to the client. In Python there's the Trio library [2], which implements structured concurrency, and which was actually one of the main inspirations for developing Polyphony. [1] https://en.wikipedia.org/wiki/Structured_concurrency https://en.wikipedia.org/wiki/Structured_concurrency [2] https://github.com/python-trio/trio https://github.com/python-trio/trio
- ciconia 4y agoI'm the author of Polyphony. I'd be happy to answer any questions you have.