4 ms·

Well, we've seen for example, Twitter do that. I saw some slides[0] that other day from sometime in 2007[1] where they said they were using Ruby and spawning 18

by hasenj 9y ago

Well, we've seen for example, Twitter do that. I saw some slides[0] that other day from sometime in 2007[1] where they said they were using Ruby and spawning 180 rails instances to handle 600 requests per second. Like, that's insane. A compiled language can handle that load with just one instance. It's nothing. [0]: https://www.slideshare.net/Blaine/scaling-twitter/3-First_Some_Facts_600_requests https://www.slideshare.net/Blaine/scaling-twitter/3-First_So... Also, if your audience is small enough that a slow python can suffice, why even bother using a database server such as postgresql or mysql when you can just use sqlite and simplify the architecture? EDIT: [1] It's been 10 years since 2007 but people still do this more or less all the time. Write a webapp in js/ruby/python and spawn 20 instances of the application server

tychver 9y ago
Twitter's issue was more with concurrent IO than anything. This problem was solved about 10 years ago with the release of Ruby 1.9 and since then only one Ruby process per core is required, just like NodeJS. In that time most CPU intensive parts of the web stack have also been re-written as native extensions. Recent benchmarks of a properly setup Ruby environment vs Go/Gin are showing Ruby/Sinatra as having 50% of the throughput https://www.techempower.com/benchmarks/#section=data-r14&hw=ph&test=json&l=8vmkfz https://www.techempower.com/benchmarks/#section=data-r14&hw=... You can also just use JRuby and use a single JVM process. For small CLI apps, MRuby can even be compiled to C, then compiled as an executable.
- hasenj 9y ago
  Contrived benchmarks are not useful. Specially when you have tiny datasets. If you want to do anything interesting, Python/Ruby are slow as hell, which is why you cannot do anything interesting in them. For example, while in Go, you can load say 1000 rows from the db and perform some data manipulation on it in the code to get a desired result, you cannot do this in Python because it will very very slow. So what you do is you write complicated sql queries and essentially offload all your work from the application server(s) on to the database server. Now imagine that these rows on the database don't actually change very often. You could just load them once, keep them in memory (in a global object), and only update them once in a while (when needed). You can always do whatever search/manipulation operation directly on the data that is readily available and always respond very quickly. This would be _unthinkable_ if you are using Ruby or Python, so instead you keep hammering your database with the same query, over and over and over again.
  - tychver 9y ago
    Simple benchmarks are a useful yardstick. I recently wrote a service in Rust/Iron which only has 4.7x the throughput of the same Ruby/Rails service. That was rather disappointing considering how much more effort is required to do it in a lower level language. Is Python/Django performance significantly worse than Ruby/Rails? The situations you describe are things I do every day in Ruby. Getting 1000 rows from the DB and performing some operation only takes a couple of milliseconds in Ruby. Ruby/Python are meant to be glue, and you can most certainly use them to glue together "interesting things", like image processing or audio processing in a web layer. Memory caching rarely changed, often accessed, but ultimately persisted in a DB things like exchange rates in a global object is exactly what you do in Rails. There's a specific helper for doing it. http://api.rubyonrails.org/classes/ActiveSupport/Cache/MemoryStore.html http://api.rubyonrails.org/classes/ActiveSupport/Cache/Memor...
    - steveklabnik 9y ago
      Iron is still using sync IO, and while Ruby isn't great at parallel stuff, it at least does async IO, IIRC. That's going to be a huge difference.
      - viraptor 9y ago
        It depends what you're doing. If you're running a ruby web app server and talking to the database, all the io you're doing is most likely synchronous. In the one-server-per-cpu model, anyway.
        steveklabnik 9y ago
        It's been a while, but I thought that MRI basically slept during IO and released the GIL so that other threads could do work. In that case, you'd still be handling more requests while the IO was performed. I could be wrong. This kind of thing: http://ablogaboutcode.com/2012/02/06/the-ruby-global-interpreter-lock http://ablogaboutcode.com/2012/02/06/the-ruby-global-interpr...
- slackingoff2017 9y ago
  Twitter has moved key parts of the system to Java. The search functionality was redone on top of Netty which they had some articles about.

by hasenj 9y ago

tychver 9y ago

Twitter's issue was more with concurrent IO than anything. This problem was solved about 10 years ago with the release of Ruby 1.9 and since then only one Ruby process per core is required, just like NodeJS. In that time most CPU intensive parts of the web stack have also been re-written as native extensions. Recent benchmarks of a properly setup Ruby environment vs Go/Gin are showing Ruby/Sinatra as having 50% of the throughput https://www.techempower.com/benchmarks/#section=data-r14&hw=ph&test=json&l=8vmkfz https://www.techempower.com/benchmarks/#section=data-r14&hw=... You can also just use JRuby and use a single JVM process. For small CLI apps, MRuby can even be compiled to C, then compiled as an executable.

hasenj 9y ago
Contrived benchmarks are not useful. Specially when you have tiny datasets. If you want to do anything interesting, Python/Ruby are slow as hell, which is why you cannot do anything interesting in them. For example, while in Go, you can load say 1000 rows from the db and perform some data manipulation on it in the code to get a desired result, you cannot do this in Python because it will very very slow. So what you do is you write complicated sql queries and essentially offload all your work from the application server(s) on to the database server. Now imagine that these rows on the database don't actually change very often. You could just load them once, keep them in memory (in a global object), and only update them once in a while (when needed). You can always do whatever search/manipulation operation directly on the data that is readily available and always respond very quickly. This would be _unthinkable_ if you are using Ruby or Python, so instead you keep hammering your database with the same query, over and over and over again.
- tychver 9y ago
  Simple benchmarks are a useful yardstick. I recently wrote a service in Rust/Iron which only has 4.7x the throughput of the same Ruby/Rails service. That was rather disappointing considering how much more effort is required to do it in a lower level language. Is Python/Django performance significantly worse than Ruby/Rails? The situations you describe are things I do every day in Ruby. Getting 1000 rows from the DB and performing some operation only takes a couple of milliseconds in Ruby. Ruby/Python are meant to be glue, and you can most certainly use them to glue together "interesting things", like image processing or audio processing in a web layer. Memory caching rarely changed, often accessed, but ultimately persisted in a DB things like exchange rates in a global object is exactly what you do in Rails. There's a specific helper for doing it. http://api.rubyonrails.org/classes/ActiveSupport/Cache/MemoryStore.html http://api.rubyonrails.org/classes/ActiveSupport/Cache/Memor...
  - steveklabnik 9y ago
    Iron is still using sync IO, and while Ruby isn't great at parallel stuff, it at least does async IO, IIRC. That's going to be a huge difference.
    - viraptor 9y ago
      It depends what you're doing. If you're running a ruby web app server and talking to the database, all the io you're doing is most likely synchronous. In the one-server-per-cpu model, anyway.
      - steveklabnik 9y ago
        It's been a while, but I thought that MRI basically slept during IO and released the GIL so that other threads could do work. In that case, you'd still be handling more requests while the IO was performed. I could be wrong. This kind of thing: http://ablogaboutcode.com/2012/02/06/the-ruby-global-interpreter-lock http://ablogaboutcode.com/2012/02/06/the-ruby-global-interpr...
slackingoff2017 9y ago
Twitter has moved key parts of the system to Java. The search functionality was redone on top of Netty which they had some articles about.