3 ms·

I'm not trying to knock on anyone's development platform of choice, but how can anyone expect unproven software to perform at the kind of scale that some of the

I'm not trying to knock on anyone's development platform of choice, but how can anyone expect unproven software to perform at the kind of scale that some of these web apps are being asked to perform at? Lisp is a rather old and finely tuned language, but in Ansi Common Lisp, Paul Graham even suggests re-writing frequently-run sections of code in C or assembler. If a highly-optimized language needs to be profiled and tuned, how can that be avoided with a language like Python or Ruby? Of course, there are issues such as SQL queries and system/hardware configurations that come into effect as well. The point is this: at some point, when you are trying to grow a product to such a massive scale, you need to expend some energy on optimization. If you just continue growing things in the same way you built your initial prototype, you're doomed to failure.

staunch 19y ago
I think you're making the mistake of thinking just because they couldn't make Rails scale that no one could have. They probably have guys working on the problem that are early employees -- not necessarily the best of the best. That said, I don't really know too many details about who they have working on it.
- BrandonM 19y ago
  That was the purpose of my second paragraph, but maybe I didn't get the point across enough. I don't think Rails is bad, not at all. I think Rails is great for getting projects up and running quickly, and for easily adding to the projects later on. You simply need to expend some energy on optimization (and reliability) once you find yourself growing to sizes that test your current code base. This could involve writing some frequently used code in a low-level language, rewriting a Rails function you're using which does more than you really need into straight Ruby, or reconsidering some of your SQL queries and handwriting some of those that are automatically handled. All I'm trying to say is that no matter who you are, at some point in the scaling process, you are going to have to revisit your code and make improvements. The great thing about Ruby, Python, Lisp, etc is the ability to throw a bunch of code down and get something running quickly. This code is inherently not optimal, and this is not a bad thing. Once you start to see problems, however, it's time to profile and re-think some of the things you are doing in order to improve your software overall.
  - russ 19y ago
    Adding to the point you made about rewriting code - it's really easy to write inline C with Ruby. DHH has spoken about how 37signals found some performance "hotspots" with Basecamp (?) and simply dropped down into C and did a line-by-line translation to fix their issues.
    - BrandonM 19y ago
      This is exactly the kind of thing I was talking about. Thanks for the good concrete example (most of what I said was hand-waving :-).
davidw 19y ago
There's a difference between optimization and reliability. For instance, Erlang isn't the fastest thing out there, but you can make some extremely reliable systems with it. I still think Rails is a winner because of how fast it lets you develop the application. Most of us will probably never see the massive popularity spike that something like twitter has. So, better to optimize for development time, and put code speed optimization off until it's necessary.
- SwellJoe 19y ago
  True...Optimizing for getting to that popularity level fast is the wise thing to do, and so the most productive tool to develop in is the best one. But...there are major costs to a lack of reliability, and they can rear their head before you reach that level of popularity. If you're not able to stay up during that all important first crunch or slashdotting or digg...you won't ever make it to the sustained popularity and growth that requires scaling up. RoR does seem to have some problems with that. Ruby is the slowest of the most popular scripting languages, by most measures. RoR is pathological with databases in some circumstances (not common cases, admittedly). And, from what I can tell most RoR stacks (Apache+fcgi, Mongrel, ngenix, Lighty, etc.) have one or more pretty serious flaws when scaling up. I'd still consider using it, since it's so much damned fun to write Ruby code.
- BrandonM 19y ago
  Ahh, you're right. I should have said, "you need to expend some energy on optimization and ensuring reliability." I guess I was making the assumption that the software was basically working, but failing under heavy load, which seems to be an optimization problem. I let it be given that the software is correct, but perhaps that is not a safe thing to do. And I totally agree that Rails et al are great development platforms to get up and running quickly, but when that software begins to fail with increasing traffic, it's time to first reconsider your own code (algorithms and bulletproofiness (as Colbert might say)), and if that doesn't help, to consider rewriting at least parts in something else. I can't imagine the call stack for a single Rails function call (I'm actually not all that familiar with Rails). At the very least, it might be good to migrate some code from Rails to straight Ruby, and some of the auto-generated stuff to hand-tuned SQL. That's all I was trying to say.
papersmith 19y ago
One of the nice features in Common Lisp is the ability to optionally declare types on objects to make them static. This is very convenient when you want to optimize something on-the-fly; just find the bottleneck and add a type declarator to it. This can go a long way before you drop into C.
- BrandonM 19y ago
  I completely agree. This is a feature I would love to see in more languages, as it's such a no-brainer optimization that just about any language could benefit from its use.

staunch 19y ago

I think you're making the mistake of thinking just because they couldn't make Rails scale that no one could have. They probably have guys working on the problem that are early employees -- not necessarily the best of the best. That said, I don't really know too many details about who they have working on it.

BrandonM 19y ago
That was the purpose of my second paragraph, but maybe I didn't get the point across enough. I don't think Rails is bad, not at all. I think Rails is great for getting projects up and running quickly, and for easily adding to the projects later on. You simply need to expend some energy on optimization (and reliability) once you find yourself growing to sizes that test your current code base. This could involve writing some frequently used code in a low-level language, rewriting a Rails function you're using which does more than you really need into straight Ruby, or reconsidering some of your SQL queries and handwriting some of those that are automatically handled. All I'm trying to say is that no matter who you are, at some point in the scaling process, you are going to have to revisit your code and make improvements. The great thing about Ruby, Python, Lisp, etc is the ability to throw a bunch of code down and get something running quickly. This code is inherently not optimal, and this is not a bad thing. Once you start to see problems, however, it's time to profile and re-think some of the things you are doing in order to improve your software overall.
- russ 19y ago
  Adding to the point you made about rewriting code - it's really easy to write inline C with Ruby. DHH has spoken about how 37signals found some performance "hotspots" with Basecamp (?) and simply dropped down into C and did a line-by-line translation to fix their issues.
  - BrandonM 19y ago
    This is exactly the kind of thing I was talking about. Thanks for the good concrete example (most of what I said was hand-waving :-).

davidw 19y ago

There's a difference between optimization and reliability. For instance, Erlang isn't the fastest thing out there, but you can make some extremely reliable systems with it. I still think Rails is a winner because of how fast it lets you develop the application. Most of us will probably never see the massive popularity spike that something like twitter has. So, better to optimize for development time, and put code speed optimization off until it's necessary.

SwellJoe 19y ago
True...Optimizing for getting to that popularity level fast is the wise thing to do, and so the most productive tool to develop in is the best one. But...there are major costs to a lack of reliability, and they can rear their head before you reach that level of popularity. If you're not able to stay up during that all important first crunch or slashdotting or digg...you won't ever make it to the sustained popularity and growth that requires scaling up. RoR does seem to have some problems with that. Ruby is the slowest of the most popular scripting languages, by most measures. RoR is pathological with databases in some circumstances (not common cases, admittedly). And, from what I can tell most RoR stacks (Apache+fcgi, Mongrel, ngenix, Lighty, etc.) have one or more pretty serious flaws when scaling up. I'd still consider using it, since it's so much damned fun to write Ruby code.
BrandonM 19y ago
Ahh, you're right. I should have said, "you need to expend some energy on optimization and ensuring reliability." I guess I was making the assumption that the software was basically working, but failing under heavy load, which seems to be an optimization problem. I let it be given that the software is correct, but perhaps that is not a safe thing to do. And I totally agree that Rails et al are great development platforms to get up and running quickly, but when that software begins to fail with increasing traffic, it's time to first reconsider your own code (algorithms and bulletproofiness (as Colbert might say)), and if that doesn't help, to consider rewriting at least parts in something else. I can't imagine the call stack for a single Rails function call (I'm actually not all that familiar with Rails). At the very least, it might be good to migrate some code from Rails to straight Ruby, and some of the auto-generated stuff to hand-tuned SQL. That's all I was trying to say.

papersmith 19y ago

One of the nice features in Common Lisp is the ability to optionally declare types on objects to make them static. This is very convenient when you want to optimize something on-the-fly; just find the bottleneck and add a type declarator to it. This can go a long way before you drop into C.

BrandonM 19y ago
I completely agree. This is a feature I would love to see in more languages, as it's such a no-brainer optimization that just about any language could benefit from its use.