10 ms·

On the last section, where he says people "weren't avoiding Ruby for speed reasons", that's only true if you take speed very literally. Ruby has concurrency pro

• ketamine__ 6y ago
  Can you really blame Ruby when the project didn't have the right resources? The team could have booted up more processes. Isn't that the answer? Or have I misinterpreted your comment?
  • Railsify 6y ago
    This is the right answer. The demo failed because the dev team failed to understand the performance and scaling characteristics of the code they wrote. This was a failure of people, not technology.
    • josephg 6y ago
      Eh - it was a failure of the (people, technology) tuple. Different choices by the engineers on the team could have fixed that problem. And different technology choices by the engineers behind ruby (/rails) could also have fixed the problem. (Eg async io, much larger thread pools, etc). Everyone dropped the concurrency ball hoping that was someone else's problem. Its the dev team's fault because everything is ultimately the dev team's fault. But it would also be great if it was fixed in ruby + ruby's ecosystem.
      • Railsify 6y ago
        If something is a known-known, you have to take personal responsibility when you don't take that thing into account and get bit. It's not appropriate to even mention some person you have not met as part of the reason a demo failed. Personal accountability it key in this industry, do your due diligence and events like were described will not happen.
        • dwrowe 6y ago
          Easier to blame the seemingly nameless technology and completely change the conversation by using some other stack entirely that accounted for your lack of due diligence.
    • sanderjd 6y ago
      Yes but the point is that lots of people became disenchanted with ruby because it was notably harder to get this sort of thing right than it was for the alternatives with more built in support for in-process concurrency. Maybe it's not the case anymore, but back around 2010, the only people who knew what they needed to do to configure a rails app to scale decently were those who had battle scars of the kind in the parent comment. And nobody came out of those battles feeling all sunshine-and-roses about the platform.
      • jashmatthews 6y ago
        Ruby had support for in-process concurrency via threads for the last 14 years.
        • sanderjd 6y ago
          And yet, see the comment that started this thread. It is simply true that concurrency was a pain point for rails for a lot of people for a long time. (I don't know if it still is, I've been out of that game for five years.)
  • ReactiveJelly 6y ago
    By resources do you mean hardware resources or expertise or what? Cause running more than 2 processes on a 2-core server just sounds weird. I thought Node's doctrine was "One process per core", and (async) Rust and Go seem to do fine with managing their own thread pools. I haven't _tried_ to do a realistic benchmark of Hyper, but I've never seen it struggle with concurrency either, even on long-polling applications where concurrency was the norm for all clients. More processes would work, but it doesn't seem like the right layer of abstraction for 2015, now that Nginx and Node have made the event model so popular.
    • ketamine__ 6y ago
      More hardware resources. The example given was a Rails app. I don't think one process per core is common for Rails deployments but I'm not sure.
      • jashmatthews 6y ago
        Process per core with threading for parallel IO has been the default for almost a decade now.
        • ketamine__ 6y ago
          Default in what app servers?
          • jashmatthews 6y ago
            Puma is part of the default Rails Gems and was first released all the way back in 2011.
  • strken 6y ago
    You're absolutely right! The biggest things I learnt from that were: - overprovision resources for big demos by a significant amount - test your demo before you give it - deploying pets (servers you ssh into and lovingly configure) instead of cattle (instance abc123 that's created automatically) means you can't scale them as easily - if you have concerns about a big demo, particularly ones that could be fixed by spending $10 on EC2 instances, you should fix them yourself instead of emailing the tech lead and assuming he'll handle it - spinning up processes with phusion passenger is awful[0] and we should have been using something threaded like puma That being said, the same footguns apply at various points as you scale, because Ruby's GVL still exists, and because even though threads are lighter weight than processes, they're still relatively expensive to switch between and to maintain in memory compared to e.g. Go/Elixir lightweight threads or evented IO. Unfortunately (or perhaps fortunately) the problems with threaded IO start applying well above 20 concurrent users, so I don't have any fun stories about failed demos involving Puma instead of Passenger. [0] Specifically, Rails processes take up a lot of memory, none of it is shared, context switching between processes is pretty expensive, and so you're limited to a smaller number of processes that can become IO-bound very easily
    • ketamine__ 6y ago
      Passenger can be configured to restart processes after they reach a memory threshold. Another optimization is using jemalloc. Going to try this soon with Sidekiq workers.
    • jashmatthews 6y ago
      > Threads are lighter weight than processes they're still relatively expensive to switch between and to maintain in memory ... context switching between processes is pretty expensive This is a really common misconception. Both switching between threads and processes is _extremely_ fast compared to what web applications do. An exceptionally fast web response is generally <1ms. Linux thread context switching takes ~2 microseconds. Forking a new processes takes ~20 microseconds.
      • strken 6y ago
        Relatively expensive was probably not phrased well, but a goroutine context switch is ~200 nanoseconds and instantiating a new one takes ~1 microsecond, from memory and a quick google.
  • chmod775 6y ago
    I think you have missed where he said it had 20 people using it. Not 200,000. A single process wasn't able to handle 20. You'll be hard pressed intentionally writing code that performs that slowly in a framework in another language. Ruby by default has an incredibly long list of "you can but you shouldn't" and things you need to be aware of, especially around concurrency. Other languages are mostly fast by default. When you follow some nodejs/express/whatever introduction tutorial and implement your first application in the most naive way possible, you'll end up with something that will easily scale to hundreds if not thousands of users out of the box. Do the same with ruby (or even with some python frameworks, to be fair), and stuff can break at less than a hundred users. And that's why yes, you can blame that on ruby. Or at least ruby how it is commonly taught.
    • ketamine__ 6y ago
      There is a lot we don't know with the example given. It's unfair to blame Ruby.
      • lostcolony 6y ago
        But it's not unique to this example. It's common; if you don't start out concerned about concurrency in Ruby, and just do the naive thing, you are going to fall over at -extremely- low numbers compared with most other modern languages.
        • debaserab2 6y ago
          The "naive" thing in Ruby, if you're serving HTTP requests, is to use some kind of app gateway that allows concurrency, such as puma or unicorn. It's just not all that viable to do something more basic than that unless you're willing to write all the request handling stuff by hand. That aside, a ruby script that can't handle 20 req/s on 2 vcpus seems rather suspect. I'd be really curious what that code looked like. I'm doubtful that it has much to do with the language and not the code design.
        • the-dude 6y ago
          If it is so common as you claim, could you point us to some public examples of this? I have never ever been concerned about concurrency in Ruby, or Python or PHP. Just deployed the number of workers which would be able to serve our peaks. I do write high concurrency application though, but those are not web apps and are not written in these languages.
• ioquatix 6y ago
  This problem is being addressed in Ruby 3.
  • cutler 6y ago
    Ruby 3 seems to be going the way of Perl 6, ie. always about to be more efficient/faster sometime in the future. I hope not but isn't there a limit to how much you can re-jig an interpreted language designed in the early 90s? The same applies to Python.
    • bdcravens 6y ago
      The same also applies to Javascript.
      • petre 6y ago
        Yup, only the Node runtime is very fast, pretty much 2x-3x times faster than Python 3.6 and Perl 5.26 and 4-5x faster than Ruby 2.4 for non I/O operations (loop, push to array, set hash keys). Only Lua and Luajit beats it. Raku/Perl6 was still slower than Ruby last time I checked. My tests were crunching primes and the Raku foreach loop had worse performance than the loop statement (ugly C-style loop). For number crunching Raku is slower probably because of the rational math. Anyway, I like Ruby more than any of those languages, so hopefully speed in 3.0 is at least on par with Python and Perl now.
        • bdcravens 6y ago
          Now Ruby just needs someone putting Google-level resources into optimizing the runtime (since most of Node's performance comes from v8)
          • SahAssar 6y ago
            I don't think anyone put google-level cash into LuaJIT but it still manages to be pretty fast.
    • 6y ago
• tempest_ 6y ago
  This is the same reason I have been shifting away from python for anything more than a quick script. Processors have only been going sideways for a while and python makes this a pain. Elixir and Rust are a little less painful in that regard.
• dragonwriter 6y ago
  > Ruby has concurrency problems [...] standard libraries don't default to evented IO, and the GIL messes up threading, both of which can be footguns for anyone trying to build a resilient and scalable service. Yes, but this article is about speed, not concurrency, because speed is a thing 3.0 didn't do a major improvement to from the last 2.x release. There's no need for a “Why didn't Ruby 3 improve concurrency/parallelism” article, because it did. Fiber#scheduler addresses your IO issue, and Ractors (while still experimental) are aimed at the parallelism issue.
• protomikron 6y ago
  There is something seriously wrong with your number (struggling to serve 20 people). I do not have experience with Ruby, but I think it's similar to Python (in performance) and even if it's let's say 5 times slower (which I don't think it is), a webservice on a simple machine (workstation) in Ruby can easily serve north to 1k (or more) requests per seconds (behind nginx or apache or whatever), and in no way would 20 users be a problem. It doesn't matter if that's async or not (note, not being async does not rule out concurrency or parallelism), but there was something seriously wrong with your application (if you have to serve long-running tasks, you have to make a msg queue anyway and shouldn't do that in your HTTP response directly). You may gain from async if you are IO bound, but with 20 users you are not IO bound, believe me.
  • strken 6y ago
    We were "IO-bound" using a setup that executed a maximum of two requests at a time, completely blocked processing new requests until earlier ones fully completed, and took a long time to execute requests because of unexpected latency in a dependency, while 20 frustrated users sat in a room hammering the refresh button and dumping more tasks onto the queue. You're absolutely right when you say there was something seriously wrong with the setup! The problem is that it was more or less what you'd get if you set up a default Rails app behind Passenger, and Passenger did this because some of the major Ruby gems weren't threadsafe (e.g. Rails, until 2008), which comes back to concurrency problems with the ecosystem.
    • darwinwhy 6y ago
      What is Passenger exactly? Never heard of it before now. A quick Google brings up a marketing page that doesn't explain much more.
      • johnmcauley 6y ago
        It’s a Ruby web server that can we bed used with nginx or Apache.
      • universa1 6y ago
• pmontra 6y ago
  What year was that? I had Rails applications running on a dual processor Pentium 3 in 2007. An online sailing game, no problems with performances. It's hard to remember what we had to do back then, but Passenger was probably there. My first though: did you use a connection pool and a C driver to make parallel requests to the database?
• vidarh 6y ago
  So you hadn't load-tested, and had misconfigured Rails. I really, really dislike Rails for any number of reasons, but not having spent the time to understand how to set it up is not Rails fault.
  • bzb6 6y ago
    Defaults matter.
    • WJW 6y ago
      So does knowledge of your stack. Rails default settings are for development on one box, it expects you to look over the settings at least once before you push to prod. I can't blame Rails for not having "client demo for 20 people" as the default settings (and I also can't really blame GP for not thinking about Rails configuration too deeply as a junior engineer, that is on the more senior engineers on that team who apparently completely forgot to verify it worked before doing the demo).
• throwaway189262 6y ago
  Java's concurrency support is quite good. One of the best language level implementations I've used. It supports OS level threads and async. And they're adding support for fibers. At that point it will be the only mainstream language that supports all three popular models JS has an equally bad concurrency story as Ruby. Maybe even worse. There are no threads, you can only use a single core per instance. In practice the problem is ram. JS already uses a lot of ram compared to Java and running an instance for each core makes it way worse.IMO only crazy people run JS backends when there's so many other similar options
  • jiofih 6y ago
    > JS has an equally bad concurrency story as Ruby. Maybe even worse. This is absolutely false. Yes, JS is single threaded, but all IO is async and happens via a worker pool. On top of that it is 3-10x faster than Ruby for most tasks due to the insane amount of development effort that went into the browser runtimes, it’s often within 50% of C speeds. And then the entire ecosystem is async. This is why you’ll see JS frameworks near the top of TechEmpower benchmarks achieving massive concurrency numbers, and also why nobody recommends any kind of CPU-heavy workload in node. Imagine Ruby came with Puma built-in, and all libraries only exported async interfaces.
    • throwaway189262 6y ago
      Makes sense. Better than Ruby I guess. It's still not really concurrency, it just handles being single threaded better. JS is strange because of all the optimization. It's technically multi threaded with IO but in a way that you only control the "main thread". Reminds me of old UI work
      • coldtea 6y ago
        >It's still not really concurrency Well, it also has Workers, which is.
      • jiofih 6y ago
        Concurrency != parallelism. You can of course run a multi-process cluster, but that’s besides the point (and these days it’s best to run a single process inside orchestration like k8s). Basically almost every high performing server atm is running on some form of event loop / cooperative multi-tasking. Ruby Puma, Gunicorn, rust’s tokio and async/await, goroutines... Java is probably the big exception here (if you’re not using VertX et al) and we all know the pain of thread safety.
• jiofih 6y ago
  The first enterprise Java app I was a part of, failed out of the gate because some external requests were too long and exhausted the thread pool (100 by default).
• gray_-_wolf 6y ago
  > When I was a junior engineer, one of our demos to a client failed because we had a group of 20 people try to use a server that had a highish latency connection to a database, and using synchronous IO meant that a 50ms db round trip for each executed query, with each request running maybe 5 or 10 queries, on a VM with 2 cores and one Rails process per core, would queue up incoming requests until they hit the 20 second mark and started timing out. I'm fairly sure GIL is released when waiting for external I/O, at least all the decent C database libraries do so. So in your case threads would have help quite a bit. In general yes, the GIL is imho a problem and I do not like it, but at the same time there are ways to configure applications so that they do not mind that much. For non-thread safe things (are there still any?) there is always the goold old way of just running multiple processes (with preloading the memory usage of even 10, 15 puma workers is often acceptable).
• Person5478 6y ago
  yeah, Heroku infamously had a really hard time getting Ruby to scale well. Ruby traded safety (GIL) for absolutely speed, although it should be noted that Ruby's C interfaces do allow you to op out of the GIL, which means if speed is absolutely paramount you can write a Ruby C extension to opt out of it. I've done it in the past, but OTOH if you're to that point maybe Ruby no longer makes sense.
• ufmace 6y ago
  Probably a little late to be second-guessing exactly what happened there, but a few comments anyways: It's almost always worth doing some concurrency/scaling sanity checks on anything you're preparing to demo IME. There's plenty of ways any particular app, no matter what language or framework, could unexpectedly fall down when facing 20x traffic. If not from the language itself, perhaps from various external dependencies. It may be true, particularly at some previous times, that doing the most naive thing in Rails hosting was significantly worse than doing the most naive thing in Java hosting, but still, always worth checking. The 50ms DB round-trip 10x per request smells funny. Most queries to a closely located DB with demo app data should be more like 5ms or faster. Maybe there's good reason for it to take so long, I'm not sure. Perhaps the DB also had some slowdown from so many of those queries being executed concurrently, or maybe it would have if you had had a faster Rails setup or used Java instead. All the more reason to always test it and not just assume it'll be fine because Java should be fast. Each individual web request is still gonna be rather slow if it's gotta do multiple 50ms DB round trips. I don't know Java that well, but C# and Ruby both have pretty decent tools for making that IO concurrent in the scope of an individual request. You have to deliberately use it though, since only you know which queries actually depend on data from other queries. I've been working professionally with Rails for several years. If I was deploying such an app now, it'd be simple and fast to say, I suppose I'll serve it with Puma. 2 cores, so 2 processes. 5-10 threads per process is pretty normal, maybe make it 20 here since I expect a lot of time waiting on DB IO. Now we've got 40 workers and it ought to serve traffic from 20 users fine. Assuming some other part doesn't fall down first.