4 ms·

Twitter's issue was more with concurrent IO than anything. This problem was solved about 10 years ago with the release of Ruby 1.9 and since then only one Ruby

Twitter's issue was more with concurrent IO than anything. This problem was solved about 10 years ago with the release of Ruby 1.9 and since then only one Ruby process per core is required, just like NodeJS. In that time most CPU intensive parts of the web stack have also been re-written as native extensions. Recent benchmarks of a properly setup Ruby environment vs Go/Gin are showing Ruby/Sinatra as having 50% of the throughput https://www.techempower.com/benchmarks/#section=data-r14&hw=ph&test=json&l=8vmkfz https://www.techempower.com/benchmarks/#section=data-r14&hw=... You can also just use JRuby and use a single JVM process. For small CLI apps, MRuby can even be compiled to C, then compiled as an executable.

hasenj 9y ago
Contrived benchmarks are not useful. Specially when you have tiny datasets. If you want to do anything interesting, Python/Ruby are slow as hell, which is why you cannot do anything interesting in them. For example, while in Go, you can load say 1000 rows from the db and perform some data manipulation on it in the code to get a desired result, you cannot do this in Python because it will very very slow. So what you do is you write complicated sql queries and essentially offload all your work from the application server(s) on to the database server. Now imagine that these rows on the database don't actually change very often. You could just load them once, keep them in memory (in a global object), and only update them once in a while (when needed). You can always do whatever search/manipulation operation directly on the data that is readily available and always respond very quickly. This would be _unthinkable_ if you are using Ruby or Python, so instead you keep hammering your database with the same query, over and over and over again.
- tychver 9y ago
  Simple benchmarks are a useful yardstick. I recently wrote a service in Rust/Iron which only has 4.7x the throughput of the same Ruby/Rails service. That was rather disappointing considering how much more effort is required to do it in a lower level language. Is Python/Django performance significantly worse than Ruby/Rails? The situations you describe are things I do every day in Ruby. Getting 1000 rows from the DB and performing some operation only takes a couple of milliseconds in Ruby. Ruby/Python are meant to be glue, and you can most certainly use them to glue together "interesting things", like image processing or audio processing in a web layer. Memory caching rarely changed, often accessed, but ultimately persisted in a DB things like exchange rates in a global object is exactly what you do in Rails. There's a specific helper for doing it. http://api.rubyonrails.org/classes/ActiveSupport/Cache/MemoryStore.html http://api.rubyonrails.org/classes/ActiveSupport/Cache/Memor...
  - steveklabnik 9y ago
    Iron is still using sync IO, and while Ruby isn't great at parallel stuff, it at least does async IO, IIRC. That's going to be a huge difference.
    - viraptor 9y ago
      It depends what you're doing. If you're running a ruby web app server and talking to the database, all the io you're doing is most likely synchronous. In the one-server-per-cpu model, anyway.
      - steveklabnik 9y ago
        It's been a while, but I thought that MRI basically slept during IO and released the GIL so that other threads could do work. In that case, you'd still be handling more requests while the IO was performed. I could be wrong. This kind of thing: http://ablogaboutcode.com/2012/02/06/the-ruby-global-interpreter-lock http://ablogaboutcode.com/2012/02/06/the-ruby-global-interpr...
        viraptor 9y ago
        You're right that it can do async io. But you compared a framework to a language. Rust has Tokio for async - Iron is just not using it. It's similar with Ruby/RoR - yeah, they can do async. But not on their own. With unicorn server, you still get no threading and just a bunch of processes. With puma you can do threading (async cooperative really) - as long as you keep the configuration/code within the limits of what's allowed. And due to the extra care needed whenever you do caching/storage things, I expect unicorn is still the king in RoR deployments. (GH uses that for example)
        steveklabnik 9y ago
        Yeah, I guess I moved to Puma long enough ago that I forgot about this. Good points, thank you.
        9y ago
- viraptor 9y ago
  Every bigger web application will start caching at some point. This is nothing new in either python or ruby. It's even well integrated into SQL access libs: python sqlalchemy http://docs.sqlalchemy.org/en/latest/orm/examples.html#module-examples.dogpile_caching http://docs.sqlalchemy.org/en/latest/orm/examples.html#modul... or just using the in memory memcache.
slackingoff2017 9y ago
Twitter has moved key parts of the system to Java. The search functionality was redone on top of Netty which they had some articles about.

hasenj 9y ago

Contrived benchmarks are not useful. Specially when you have tiny datasets. If you want to do anything interesting, Python/Ruby are slow as hell, which is why you cannot do anything interesting in them. For example, while in Go, you can load say 1000 rows from the db and perform some data manipulation on it in the code to get a desired result, you cannot do this in Python because it will very very slow. So what you do is you write complicated sql queries and essentially offload all your work from the application server(s) on to the database server. Now imagine that these rows on the database don't actually change very often. You could just load them once, keep them in memory (in a global object), and only update them once in a while (when needed). You can always do whatever search/manipulation operation directly on the data that is readily available and always respond very quickly. This would be _unthinkable_ if you are using Ruby or Python, so instead you keep hammering your database with the same query, over and over and over again.

tychver 9y ago
Simple benchmarks are a useful yardstick. I recently wrote a service in Rust/Iron which only has 4.7x the throughput of the same Ruby/Rails service. That was rather disappointing considering how much more effort is required to do it in a lower level language. Is Python/Django performance significantly worse than Ruby/Rails? The situations you describe are things I do every day in Ruby. Getting 1000 rows from the DB and performing some operation only takes a couple of milliseconds in Ruby. Ruby/Python are meant to be glue, and you can most certainly use them to glue together "interesting things", like image processing or audio processing in a web layer. Memory caching rarely changed, often accessed, but ultimately persisted in a DB things like exchange rates in a global object is exactly what you do in Rails. There's a specific helper for doing it. http://api.rubyonrails.org/classes/ActiveSupport/Cache/MemoryStore.html http://api.rubyonrails.org/classes/ActiveSupport/Cache/Memor...
- steveklabnik 9y ago
  Iron is still using sync IO, and while Ruby isn't great at parallel stuff, it at least does async IO, IIRC. That's going to be a huge difference.
  - viraptor 9y ago
    It depends what you're doing. If you're running a ruby web app server and talking to the database, all the io you're doing is most likely synchronous. In the one-server-per-cpu model, anyway.
    - steveklabnik 9y ago
      It's been a while, but I thought that MRI basically slept during IO and released the GIL so that other threads could do work. In that case, you'd still be handling more requests while the IO was performed. I could be wrong. This kind of thing: http://ablogaboutcode.com/2012/02/06/the-ruby-global-interpreter-lock http://ablogaboutcode.com/2012/02/06/the-ruby-global-interpr...
      - viraptor 9y ago
        You're right that it can do async io. But you compared a framework to a language. Rust has Tokio for async - Iron is just not using it. It's similar with Ruby/RoR - yeah, they can do async. But not on their own. With unicorn server, you still get no threading and just a bunch of processes. With puma you can do threading (async cooperative really) - as long as you keep the configuration/code within the limits of what's allowed. And due to the extra care needed whenever you do caching/storage things, I expect unicorn is still the king in RoR deployments. (GH uses that for example)
        steveklabnik 9y ago
        Yeah, I guess I moved to Puma long enough ago that I forgot about this. Good points, thank you.
        9y ago
viraptor 9y ago
Every bigger web application will start caching at some point. This is nothing new in either python or ruby. It's even well integrated into SQL access libs: python sqlalchemy http://docs.sqlalchemy.org/en/latest/orm/examples.html#module-examples.dogpile_caching http://docs.sqlalchemy.org/en/latest/orm/examples.html#modul... or just using the in memory memcache.

slackingoff2017 9y ago

Twitter has moved key parts of the system to Java. The search functionality was redone on top of Netty which they had some articles about.