10 ms·
Ruby concurrency is hard: how I became a Ruby on Rails contributor
- igravious 4y agoReally interesting and in depth article on concurrency in Ruby. Thank you, author, for improving the Ruby ecosystem. Pity hardly any of the 129 comments before me bothered to address it.
- naikrovek 4y agoRuby on Rails contributors should be called "conductors".
- mikedelago 4y agoI'm pretty sure that's where Elixir got its start, too. Jose Valim is a member of the RoR core team and he wanted better concurrency for Ruby.
- cpursley 4y agoYeah. Unless you're already running a big Rails app that's not a good candidate for a rewrite, what's the point? Nothing against Ruby, but just use Elixir/Erlang already.
- valenterry 4y agoThe BEAM is awesome, but both Elixir and Erlang are not really great for concurrency. They lack support for pure functional programming, which is really one of the most useful things in this area.
- brightball 4y agoHow are Elixir / Erlang anything but great for concurrency?
- valenterry 4y agoUsing Elixir / Erlang on the BEAM is great for concurrency - but the programming languages themselves are not. It's an important distinction, because I believe that there is still a lot of potential for those languages.
- ramchip 4y agoWhat is it about pure functional that makes concurrency much easier? Is there a code sample that would illustrate it? I've done both Haskell and Erlang/Elixir and yet I don't really see what you're referring to concretely. There was hope that automatic parallelization would be a big win for pure functional, but I don't think it's really worked out in practice, because of the overhead and difficulty of predicting if parallelizing something would make it faster or slower.
- valenterry 4y agoIt's not about performance as much as correctness and easiness to understand code. Functional programming removes the need to simulate (local) state in your head when trying to understand what code does. And in the places where it is inevitable, it makes it explicit and leads to a design where it's easy to track why and how things got changed. For example, .map and .filter are now in almost every language, because they are much easier than for-loops. You see them and you know "aha, this collection will be transformed, the number of elements stays the same" or "this collection will be filtered and the elements will look the same, but some might be gone and no new ones will have been added". Pure functional programming is similar, just that the scope is suddenly "the whole runtime/machine" instead of "a small piece of code that somehow does a loop".
- mrdoops 4y agoRunning into concurrency woes of that whole generation of languages (Ruby/Python/PHP/Java/JS) is how most Elixir developers happen I think ;)
- letmeinhere 4y agoIsnt the php concurrency story distinctly good, with every request spawning its own process?
- colonwqbang 4y agoAnyone can do "concurrency" by starting from scratch with a fresh process each time. That isn't "having a concurrency story" it's rather the opposite I think. It's very wasteful of system resources, energy, etc.
- stevewatson301 4y agoPHPs model would be okay if they used green threads/goroutines. That way, you can have a far larger number of workers, as most of them are blocked on database connections or HTTP requests to other services.
- heurisko 4y agoPHP's model is different, not a mistake. It has proved successful so far. The shared-nothing, short-lived process avoids classes of issues such as with slow memory leaks, accidentally blocking event loops, and shared memory threading issues. Most PHP sites will utilise php-fpm, so it isn't really true that each request will spawn another PHP process. The language historically hasn't been that great, but its shared-nothing architecture has always been the good part.
- weatherlite 4y agoOn the other hand its simple and works for many many people and apps. I wouldn't just discount it. The fact that "Anyone can do" it is a pro.
- Gigachad 4y ago[flagged]
- mountainriver 4y agoMaybe he didn’t get the memo that we’ve tried dynamic types and it’s not that great of an idea outside of some edge cases
- phyrex 4y agoYes that’s why everybody collectively moved on to static types exclusively 30 years ago and we all lived happily ever after because there was never a point where dynamic typing made more sense or was anything but inferior. The end.
- Gigachad 4y agoMy theory is that types were the baby thrown out with the bathwater. Dynamic languages changed a whole lot more than just types. The older typed languages were a huge pain to work with, dynamic languages fixed a lot of that and added standard libraries that do a huge amount of work for you. Now TypeScript and Rust are showing typed languages can be just as productive, if not more productive than Ruby/Python/JS.
- phyrex 4y agoI don’t buy it. Static languages have certainly become more ergonomic, but if the evidence was really so overwhelmingly clear, everybody would have switched by now. To wit, regarding your comment about Rust, I just read this article earlier today: https://mdwdotla.medium.com/using-rust-at-a-startup-a-cautionary-tale-42ab823d9454 https://mdwdotla.medium.com/using-rust-at-a-startup-a-cautio...
- Gigachad 4y agoJust about everyone has switched from JS to TS. When given the choice of the exact same language with and without types, it seems pretty clear that typed languages win.
- hit8run 4y agoI haven’t found anything like Rails when it comes to developer ergonomics. I like typed languages and I feel safer with them but there are things in Rails I haven’t seen in other Frameworks before like the webconsole that instantiates a Repl in the browser with the context of the loaded page.
- smohare 4y ago
- r-s 4y agoThread safety in Rails is a total pain in the ass. So many libraries out there are not thread safe. I have worked with many large rails codebases and there always are gems that somehow work their way in with major threading issues. I think its a legitimate strategy to run rails apps single threaded and multi process. Not necessarily for all apps or workloads of course.
- Rapzid 4y agoMultiprocess is more often than not better for latency anyway. Otherwise a big serialization or other cpu clog will block up the other, potentially quick, requests on the same process. Much has been written about this.
- revskill 4y agoCould JRuby help ?
- mperham 4y agoThis is utterly false in my experience. The popularity of Puma and Sidekiq mean that threading issues are quickly found and fixed.
- dalyons 4y agoYeah I agree. Thread safety in ruby is a non issue these days. Maybe this is 10 years out of date experience?
- pdhborges 4y agoHow does Puma help with the libraries that are not thread safe?
- sparker72678 4y agoI think he's saying threading issues are more likely to be found and fixed these days given the context in which most ruby libraries are running.
- _odey 4y agoI've written some threaded ruby code in the past and in addition to what was said in this article I would like to emphasize 2 more points: 1. Every time you have a shared data structure, put a Mutex around it's usage (even in POC examples so people learn the properly). In the reproduction example line 4 you have `ids = Set.new` (the shared data structure) and on line 9 `ids << semaphores['test'].object_id`, since this is within a thread you need to put a Mutex around the push to `ids`. You can still have `semaphores['test'].object_id` outside the Mutex, assigning that to a variable and then push that to `ids` in the synchronize block to achieve the same result. If you have both inside the Mutex it would hide the presented issue. 2. Write stress tests that loop through your multithreaded code multiple times with trivial workloads; 100 times might not be enough to trigger a concurrency issue, you might need tens of thousands, or millions of iterations, or more. So don't be afraid to have the loop size configurable and just run it for a longer period of time, just to make sure. In CI you can tweak the loop size to have it running for let's say 3-5 minutes. That should make it relevant and still keep it fast/cheap enough. Additionally, ruby now has Ractors, but still marked as experimental. These are amazing as you not only get safe parallelism (not just concurrency, true parallelism), but also data safety as a piece of data is either copied deeply, or ownership is passed to one single Ractor. Fingers crossed we see it stable soon.
- POPOSYS 4y agoWhy are people still using programming languages where concurrency is hard?
- eduge4u 4y agoAccording to the MATZ: Concurrency is not the problem if you ask me, We can handle it. The issue is in the read and write of database, the time spend there is the bottleneck.
- jononomo 4y agoI honestly don't understand why Elixir and the Erlang/BEAM ecosystem haven't taken over the software world more completely. It is inconceivable to me to begin a new web app using Ruby on Rails when I could use Elixir and Phoenix.
- jononomo 4y ago1. Once I got the hang of the functional programming paradigm it suddenly felt like a huge weight off my shoulders. I love never thinking about objects and the OO approach. 2. Concurrency is built in so thoroughly that there is an entire framework called OTP that provides powerful tools and supervisor processes and so forth to really take concurrency to the next level. 3. Once I got the hang of OTP, similar to when I got the hang of functional programming, it felt like a huge weight off my shoulders again. OTP makes concurrency so easy and lightweight that you think nothing of spinning up three dozen processes just to handle some trivial calculation or data transformation -- processes are like objects that are actually alive and useful and doing things, and OTP is like an orchestrator. It just feels to me like everything is clearer, simpler, and more robust when I use Elixir and the BEAM/OTP ecosystem. Also, writing a web application in Elixir means writing a program that you then fire up and which stays up until you bring it down or it crashes for some reason. In Rails on the other hand, you write a script that is fired up every time an http request arrives, builds it's entire context for everything, handles the request, and then dies split second later, only to start the whole process over again a split second after that. There is a certain simplicity in only firing things up once and then just keeping your context alive in perpetuity.
- nickjj 4y ago> I honestly don't understand why Elixir and the Erlang/BEAM ecosystem haven't taken over the software world more completely I thought similar things in 2016 when I started learning Elixir and Phoenix but after 2 years, building multiple apps, dabbling with it for years afterwards (up until about last year) and giving it everything I had (and more). It wasn't for me. The conclusion I came to was Elixir makes certain things that are hard in other languages easier and there's certain things in other languages that are simple but much different in Elixir. For the apps I build (typical web apps like GitHub, etc.) I found that I ran into more scenarios where Elixir as a language made it harder for me to build the things I wanted. For things that Elixir made easier, other languages have "good enough" solutions. If the goal of all of this is to build things to provide value for others, choosing a stack with a larger ecosystem of tools and community support wins in my book when you've reached the point of "good enough" in other things. Responding back in <= 100ms for the p95 case is "good enough" and most web tech stacks can do this without a huge amount of server costs or extra effort if you're talking about million dollar business sized apps where you might be dealing with a few hundred thousand visitors a month for a typical SAAS app. Things like Hotwire[0] are also tech stack agnostic. You can build really nice feeling apps with minimal effort. I spent about a week upgrading one of my apps to use it while learning as I went and it made a massive difference. Conceptually it was low effort to understand and didn't require rewriting everything. That has been the polar opposite experience I've had when I used LiveView. [0]: https://hotwired.dev/ https://hotwired.dev/
- deleted 4y ago[deleted]