4 ms·

> I recently wrote a project that needs to do 100's or 1000's of possibly slow network requests per second. The first try was Ruby threads. That was a disaster

> I recently wrote a project that needs to do 100's or 1000's of possibly slow network requests per second. The first try was Ruby threads. That was a disaster (as I should have predicted). I had an entire 8-core server swamped and wasn't getting near the performance I needed. I don't mean to be offensive, but welcome to at least the 1980s. We've known this doesn't scale for ages. The fact that you even tried it and thought it might be a viable solution just shows your education has failed you. I am highly biased against Node, I think it is a giant step backwards. Every blog post I have read that says the opposite admits they have no experience in anything else so they just default to Node being good. I only hope Node is a fad.

icebraining 15y ago
So you hold unfair prejudices against Node?
- sausagefeet 15y ago
  What prejudices do I have against Node that are unfair?
  - gcr 15y ago
    Citing your sources would be a great way of asserting your credibility. "Every blog post I have read that says the opposite" doesn't count. I'd love to read some well-thought-out arguments against node from people who've seriously given node a shot, but I haven't seen any. Granted, I haven't been looking for them, so please prove us wrong.
    - sausagefeet 15y ago
      I don't log the numerous NodeJS blogs I have come across over the year, my opinions on if the authors of Node blogs are qualified isn't really relevant to my point though: NodeJS is a step backwards. I wrote a comment on reddit that expands on my reasons more, although the second point is less of a problem if people use something like TameJS with Node (which I don't think most people are doing). http://www.reddit.com/r/programming/comments/ilols/the_node_beginner_book_an_introduction_to_nodejs/c24ugto http://www.reddit.com/r/programming/comments/ilols/the_node_...
  - icebraining 15y ago
    I don't know, you said you were biased against it. That's what biased means. Maybe your education has failed you?
    - sausagefeet 15y ago
      Being bias doesn't necessitate being unfair, we all have biases. It just means, all things being equal I'm not pro Node.
FooBarWidget 15y ago
> The fact that you even tried it and thought it might be a viable solution just shows your education has failed you This holier-than-thou attitude is exactly the thing that prevents more people from becoming educated on these kind of subjects. Knowledge and experience on these kinds of subjects are _not_ trivial and are _not_ easy to obtain! Information about what scales, what does not, and why, are scattered all over the place and difficult to find. It may be very obvious to you after you already know it but it's really not. If, instead of spending so much time on declaring other people as dumb or uneducated, people would spend more time on educating other people, then the world would be much better off.
- sausagefeet 15y ago
  If I told you that the Sun revolved around the earth, would you consider it holier-than-thou to tell me my education has failed me? The knowledge that the original solution (threads in Ruby for handling large numbers of concurrent connections) does not work is not a secret. Even with a peripheral following of Ruby it is well known that this solution would not work, not only is the base Ruby implementation slow but threads have been a known issue in it from day one. The solution is so misaligned with the problem that it is really difficult to validly argue this knowledge is too difficult to find. Yes, scaling to Twitter is another issue, but we're talking about the basics here. But don't put words in my mouth, I didn't call anyone dumb, I said his education has failed him. This could be himself failing to properly research the problem space, it could be his school for not properly introducing him to the subject, it could be a whole host of things. I never argued he was incapable of learning (clearly he did). And I do spend a lot of time educating people, don't take a singular snapshot of a comment on HN as indication of how my entire life is spent.
  - FooBarWidget 15y ago
    If 99% of the world doesn't know that the Sun revolved around the earth, and knowledge of that fact is scattered across a handful of monks in obscure monasteries, then yes that would indeed be a holier-than-thou attitude. And that's exactly what's going on with scaling knowledge. I would even argue that the Ruby threading problems he's experiencing may not necessarily because Ruby threads don't scale, but possibly because he's using them wrong or because he's not using the right version of Ruby. Ruby 1.8 uses select() to schedule I/O between threads so the more threads and the more sockets you have, the slower things become because select() is linear time. The use of select() also results in a hard limit of about 1024 file descriptors per Ruby 1.8 process. Also, context switching in Ruby 1.8 requires copying the stack. Ruby 1.9 is much better in this regard since it uses native threads and no longer uses select() to schedule threads that are blocked on I/O. I'm running a multithreaded, multiprocess Ruby (1.8!) analytics daemon that generates 12 GB of data per day. It flies. VMWare CloudFoundry's router is written in Ruby + EventMachine. That thing has to process tons and tons of requests and they've found Ruby + EventMachine to be fast enough. To simply say "Ruby doesn't scale and is slow" is too simplistic, and ignoring the underlying more complex reasons would result in one bumping against the same problems in a different context. So no, it isn't so obvious from day 1 that using Ruby would be a problem.
pkulak 15y ago
Known that what doesn't scale? Threads? Sure they do. I could have made Ruby threads work, I guarantee you. Move the network requests into a giant thread pool, and then do everything else synchronously. It was the context switching that was killing me, but you can use threads in such a way that that's not a problem. It's just a lot of work. More work than just using Node, which pretty much does exactly that for me for free. And on a side note "your education has failed you"? Seriously? You can't just preface something with "I don't mean to be offensive" and then say whatever you like. I don't mean to be offensive, but get yourself some social skills.
- sausagefeet 15y ago
  I don't understand why saying your education has failed you is considered offensive. In the US we say that about high schoolers all the time but it's a knock on the education system not the students.
  - pkulak 15y ago
    You're just using five words to say "ignorant".
    - sausagefeet 15y ago
      And being ignorant is an insult?

icebraining 15y ago

So you hold unfair prejudices against Node?

sausagefeet 15y ago
What prejudices do I have against Node that are unfair?
- gcr 15y ago
  Citing your sources would be a great way of asserting your credibility. "Every blog post I have read that says the opposite" doesn't count. I'd love to read some well-thought-out arguments against node from people who've seriously given node a shot, but I haven't seen any. Granted, I haven't been looking for them, so please prove us wrong.
  - sausagefeet 15y ago
    I don't log the numerous NodeJS blogs I have come across over the year, my opinions on if the authors of Node blogs are qualified isn't really relevant to my point though: NodeJS is a step backwards. I wrote a comment on reddit that expands on my reasons more, although the second point is less of a problem if people use something like TameJS with Node (which I don't think most people are doing). http://www.reddit.com/r/programming/comments/ilols/the_node_beginner_book_an_introduction_to_nodejs/c24ugto http://www.reddit.com/r/programming/comments/ilols/the_node_...
- icebraining 15y ago
  I don't know, you said you were biased against it. That's what biased means. Maybe your education has failed you?
  - sausagefeet 15y ago
    Being bias doesn't necessitate being unfair, we all have biases. It just means, all things being equal I'm not pro Node.

FooBarWidget 15y ago

> The fact that you even tried it and thought it might be a viable solution just shows your education has failed you This holier-than-thou attitude is exactly the thing that prevents more people from becoming educated on these kind of subjects. Knowledge and experience on these kinds of subjects are _not_ trivial and are _not_ easy to obtain! Information about what scales, what does not, and why, are scattered all over the place and difficult to find. It may be very obvious to you after you already know it but it's really not. If, instead of spending so much time on declaring other people as dumb or uneducated, people would spend more time on educating other people, then the world would be much better off.

sausagefeet 15y ago
If I told you that the Sun revolved around the earth, would you consider it holier-than-thou to tell me my education has failed me? The knowledge that the original solution (threads in Ruby for handling large numbers of concurrent connections) does not work is not a secret. Even with a peripheral following of Ruby it is well known that this solution would not work, not only is the base Ruby implementation slow but threads have been a known issue in it from day one. The solution is so misaligned with the problem that it is really difficult to validly argue this knowledge is too difficult to find. Yes, scaling to Twitter is another issue, but we're talking about the basics here. But don't put words in my mouth, I didn't call anyone dumb, I said his education has failed him. This could be himself failing to properly research the problem space, it could be his school for not properly introducing him to the subject, it could be a whole host of things. I never argued he was incapable of learning (clearly he did). And I do spend a lot of time educating people, don't take a singular snapshot of a comment on HN as indication of how my entire life is spent.
- FooBarWidget 15y ago
  If 99% of the world doesn't know that the Sun revolved around the earth, and knowledge of that fact is scattered across a handful of monks in obscure monasteries, then yes that would indeed be a holier-than-thou attitude. And that's exactly what's going on with scaling knowledge. I would even argue that the Ruby threading problems he's experiencing may not necessarily because Ruby threads don't scale, but possibly because he's using them wrong or because he's not using the right version of Ruby. Ruby 1.8 uses select() to schedule I/O between threads so the more threads and the more sockets you have, the slower things become because select() is linear time. The use of select() also results in a hard limit of about 1024 file descriptors per Ruby 1.8 process. Also, context switching in Ruby 1.8 requires copying the stack. Ruby 1.9 is much better in this regard since it uses native threads and no longer uses select() to schedule threads that are blocked on I/O. I'm running a multithreaded, multiprocess Ruby (1.8!) analytics daemon that generates 12 GB of data per day. It flies. VMWare CloudFoundry's router is written in Ruby + EventMachine. That thing has to process tons and tons of requests and they've found Ruby + EventMachine to be fast enough. To simply say "Ruby doesn't scale and is slow" is too simplistic, and ignoring the underlying more complex reasons would result in one bumping against the same problems in a different context. So no, it isn't so obvious from day 1 that using Ruby would be a problem.

pkulak 15y ago

Known that what doesn't scale? Threads? Sure they do. I could have made Ruby threads work, I guarantee you. Move the network requests into a giant thread pool, and then do everything else synchronously. It was the context switching that was killing me, but you can use threads in such a way that that's not a problem. It's just a lot of work. More work than just using Node, which pretty much does exactly that for me for free. And on a side note "your education has failed you"? Seriously? You can't just preface something with "I don't mean to be offensive" and then say whatever you like. I don't mean to be offensive, but get yourself some social skills.

sausagefeet 15y ago
I don't understand why saying your education has failed you is considered offensive. In the US we say that about high schoolers all the time but it's a knock on the education system not the students.
- pkulak 15y ago
  You're just using five words to say "ignorant".
  - sausagefeet 15y ago
    And being ignorant is an insult?