10 ms·

The people who come along later to scale to those millions of users your site attracted might.

by noonespecial 16y ago

cfinke 16y ago
Is there any (standard) technology stack that has proven to be unscalable?
- j_baker 16y ago
  Yes. The one you started out with is always unscalable.
  - kondro 16y ago
    I wish I could give you a 1000 up-votes for this. I'm so sick of hearing the rhetoric about language X not being able to scale. Repeat after me, if your app doesn't scale it is YOUR fault, not a language, platform or framework. Scalability is about design and implementation, not what tools you use.
  - gsivil 16y ago
    <no-sarcasm>Can you please explain that?</no-sarcasm>
    - j_baker 16y ago
      When you start out, you're designing your site in hopes that someone other than your mom is going to use it. Usually, all you need is an app server and a database. I you get successful, that won't cut it. You usually need to start adding more and more pieces and rethink some technology decisions you made originally.
      - gsivil 16y ago
        Thanks! I think I understand why the initial design/technology decisions would rarely make it to a product that has to be largely scaled. This makes perfect sense for almost all cases. Do you think that there are technologies that are inherently more scalable than others, so if some body was initially choosing them would stand more chances to scale up in a non painful manner?
        j_baker 16y ago
        Bear in mind that developer experience and smarts count for more than anything else (if you don't have those yet, don't worry - they will come with time, patience, and effort). That said, I would say that languages and technologies that help you achieve a more modular design will help you scale more. This makes it easier to tear things out and move them around so you can alter your architecture much more easily than you would otherwise. And of course efficiency (in terms of code performance) doesn't hurt either, but I wouldn't choose something on that alone. If you can have efficiency and modularity, so much the better!
        sokoloff 16y ago
        Worry about the "nobody cares enough" problem long before worrying about the "oh crap, too many people care" problem. User apathy, aka "not enough traction" kills far more startups than inability to scale. Pick the ecosystem with which your company will be most efficient and agile, and cross the scaling bridge if you come to it. We've rewritten substantially all parts of our codebase and key infrastructure 2 or 3 times in the last 8 years. You'll get the pleasure of doing the same iff you have users...
        gsivil 16y ago
        I completely agree with you. I think that in the 99.99% of the cases the question of scalability is related with a pseudo-problem or at best a good problem to have.
- mkr-hn 16y ago
  I recall twitter originally being written in something that proved unscalable, so they had to switch. Can't remember what it was though.
  - kondro 16y ago
    It was Rails and, it wasn't that it wasn't unscalable but because they did it wrong then, they hired a bunch of developers that knew nothing about Rails, so of course they moved a lot of new and core development to other technologies.
    - jarin 16y ago
      It also had a lot to do with the fact that at the time Rails sites mostly ran on lighttpd or Mongrel, which was notoriously crashy (god was originally invented just to restart crashed Mongrels). From what I understand, at the time ActiveRecord hadn't yet been rewritten in C, which was also a big bottleneck for them (not to mention trying to scale MySQL to an insane level of traffic). Nowadays, we have fancy stuff like Resque for background jobs, Passenger (aka mod_rails) for serving up Rails content, and a plethora of NoSQL databases.
      - texel 16y ago
        ActiveRecord was never rewritten in C. Twitter's problems stemmed from the fact that they had written a realtime messaging system, but designed it like a microblogging platform. The design wouldn't have scaled well, regardless of the technology they used.
        jarin 16y ago
        Oh you're right, I was thinking of the MySQL bindings (don't know where the original ones are at) https://github.com/brianmario/mysql2/tree/master/ext/mysql2 https://github.com/brianmario/mysql2/tree/master/ext/mysql2
  - j_baker 16y ago
    They didn't actually switch. They still use Ruby on Rails. They do use more things like Scala now than they used to.
tejaswiy 16y ago
This is what I don't get. ASP.NET scales pretty well as Joel opines. Why do people think .NET is bad just because it feels enterprise-y? More: http://www.youtube.com/watch?v=NWHfY_lvKIQ http://www.youtube.com/watch?v=NWHfY_lvKIQ
- ChuckN 16y ago
  It scales well but the pool of people with the experience and inclination to run a major Internet site using .NET is small and more importantly, getting smaller. Nobody graduating with a CS degree in the States right now is excited to work with MS - it just isn't sexy right now. Sexy isn't everything, but it's not nothing either.
- kondro 16y ago
  You're right, .NET (even though I despise it) scales as well as anything else. Scalability is much more about design decisions than language/platform choice. However, Joel Spolsky is a tool. Everyone I know and respect thinks he's a tool. He is a prime example of being successful because of being lucky, rather than being good. Do not quote Joel, he can't form a valid argument for a topic without circling back on himself to save his life.
  - MortenK 16y ago
    Whether Joel Spolsky is or is not a tool, lucky, good etc is completely irrelevant. He and the rest of the SE/SO team has built a highly scalable solution based on .NET. So in a discussion about scaling on .NET, it absolutely makes sense to listen to his experiences.
    - kondro 16y ago
      Actually, FogBugz is trivial to scale - there is never any more than a few dozen, to hundreds of users for any one project. And Copilot wasn't even envisaged or created by Joel, but was rather a project created by some interns. FogCreek doesn't innovate (they have a bug tracker and a remote support application based on VNC). Joel just happened to be one of the first to blog constantly about technology. He doesn't even allow comments on his posts because he doesn't want to have to deal with people contradicting him (not that he doesn't do an excellent job at this himself). There are so many great developers and businesses out there that you could use as an example to model your business and software on. Use them, not Joel. Let Joel's business and opinions fade away as other smart, innovative companies run rings around his ageing product suite.
nostrademons 16y ago
You almost always have to rewrite your whole system anyway to scale to millions of users. It doesn't matter whether it's PHP, Python, C++, or ASP.NET; it'll have to be rewritten. The real problem is the flip-side of something that the OP mentioned as an advantage: ASP.NET coders tend to be cheaper, less experienced, and hence less likely to know about some critical detail that'll boost adoption, whether it's usability or latency or security. As someone relatively inexperienced himself, the OP can't know what those things are, but it's highly likely that one of them will bite him. In other words, it's correlation, not causation. ASP.NET won't make your site suck. However, ASP.NET biases your talent pool towards developers who suck, and that will make your site suck.
- mwdev 16y ago
  To me this sounds like my friends who didn't want to go to UT because it 'was too big'. The awesome thing about too big is that if you look hard enough you are almost assured of finding someone like you. Yes there are crappy c# developers. I've also seen some bad ass c#.
  - nostrademons 16y ago
    Right, "biasing your talent pool" doesn't mean that everyone in that talent pool sucks. There are certainly some very talented C#/.NET devs. The problem is that when you go to hire, you have very little information about just how badass a prospective dev is, unless you've worked with them before. So if you take a random dev out of the C# pool, chances are he'll be worse than a random dev in the Python pool. http://www.paulgraham.com/pypar.html http://www.paulgraham.com/pypar.html
    - kenjackson 16y ago
      So if you take a random dev out of the C# pool, chances are he'll be worse than a random dev in the Python pool. That was probably true in 1995. But not today. Python has become the new Pascal/Java. Commonly the intro language in CS, so you have a lot of students who graduate using Python regardless of skill. If you said Haskell, then probably. Actually C++ maybe more then either, surprisingly.
blake8086 16y ago
Perhaps you would be interested in purchasing my Scaling Insurance? It costs a ton upfront, and it's likely to kill your nascent company, but if you ever make it big, you'll save a tiny bit of money in the long run!
- arethuza 16y ago
  Call them Scaling Default Swaps - insurance is regulated!

The people who come along later to scale to those millions of users your site attracted might.

cfinke 16y ago

Is there any (standard) technology stack that has proven to be unscalable?

j_baker 16y ago
Yes. The one you started out with is always unscalable.
- kondro 16y ago
  I wish I could give you a 1000 up-votes for this. I'm so sick of hearing the rhetoric about language X not being able to scale. Repeat after me, if your app doesn't scale it is YOUR fault, not a language, platform or framework. Scalability is about design and implementation, not what tools you use.
- gsivil 16y ago
  <no-sarcasm>Can you please explain that?</no-sarcasm>
  - j_baker 16y ago
    When you start out, you're designing your site in hopes that someone other than your mom is going to use it. Usually, all you need is an app server and a database. I you get successful, that won't cut it. You usually need to start adding more and more pieces and rethink some technology decisions you made originally.
    - gsivil 16y ago
      Thanks! I think I understand why the initial design/technology decisions would rarely make it to a product that has to be largely scaled. This makes perfect sense for almost all cases. Do you think that there are technologies that are inherently more scalable than others, so if some body was initially choosing them would stand more chances to scale up in a non painful manner?
      - j_baker 16y ago
        Bear in mind that developer experience and smarts count for more than anything else (if you don't have those yet, don't worry - they will come with time, patience, and effort). That said, I would say that languages and technologies that help you achieve a more modular design will help you scale more. This makes it easier to tear things out and move them around so you can alter your architecture much more easily than you would otherwise. And of course efficiency (in terms of code performance) doesn't hurt either, but I wouldn't choose something on that alone. If you can have efficiency and modularity, so much the better!
      - sokoloff 16y ago
        Worry about the "nobody cares enough" problem long before worrying about the "oh crap, too many people care" problem. User apathy, aka "not enough traction" kills far more startups than inability to scale. Pick the ecosystem with which your company will be most efficient and agile, and cross the scaling bridge if you come to it. We've rewritten substantially all parts of our codebase and key infrastructure 2 or 3 times in the last 8 years. You'll get the pleasure of doing the same iff you have users...
        gsivil 16y ago
        I completely agree with you. I think that in the 99.99% of the cases the question of scalability is related with a pseudo-problem or at best a good problem to have.
mkr-hn 16y ago
I recall twitter originally being written in something that proved unscalable, so they had to switch. Can't remember what it was though.
- kondro 16y ago
  It was Rails and, it wasn't that it wasn't unscalable but because they did it wrong then, they hired a bunch of developers that knew nothing about Rails, so of course they moved a lot of new and core development to other technologies.
  - jarin 16y ago
    It also had a lot to do with the fact that at the time Rails sites mostly ran on lighttpd or Mongrel, which was notoriously crashy (god was originally invented just to restart crashed Mongrels). From what I understand, at the time ActiveRecord hadn't yet been rewritten in C, which was also a big bottleneck for them (not to mention trying to scale MySQL to an insane level of traffic). Nowadays, we have fancy stuff like Resque for background jobs, Passenger (aka mod_rails) for serving up Rails content, and a plethora of NoSQL databases.
    - texel 16y ago
      ActiveRecord was never rewritten in C. Twitter's problems stemmed from the fact that they had written a realtime messaging system, but designed it like a microblogging platform. The design wouldn't have scaled well, regardless of the technology they used.
      - jarin 16y ago
        Oh you're right, I was thinking of the MySQL bindings (don't know where the original ones are at) https://github.com/brianmario/mysql2/tree/master/ext/mysql2 https://github.com/brianmario/mysql2/tree/master/ext/mysql2
- j_baker 16y ago
  They didn't actually switch. They still use Ruby on Rails. They do use more things like Scala now than they used to.

tejaswiy 16y ago

This is what I don't get. ASP.NET scales pretty well as Joel opines. Why do people think .NET is bad just because it feels enterprise-y? More: http://www.youtube.com/watch?v=NWHfY_lvKIQ http://www.youtube.com/watch?v=NWHfY_lvKIQ

ChuckN 16y ago
It scales well but the pool of people with the experience and inclination to run a major Internet site using .NET is small and more importantly, getting smaller. Nobody graduating with a CS degree in the States right now is excited to work with MS - it just isn't sexy right now. Sexy isn't everything, but it's not nothing either.
kondro 16y ago
You're right, .NET (even though I despise it) scales as well as anything else. Scalability is much more about design decisions than language/platform choice. However, Joel Spolsky is a tool. Everyone I know and respect thinks he's a tool. He is a prime example of being successful because of being lucky, rather than being good. Do not quote Joel, he can't form a valid argument for a topic without circling back on himself to save his life.
- MortenK 16y ago
  Whether Joel Spolsky is or is not a tool, lucky, good etc is completely irrelevant. He and the rest of the SE/SO team has built a highly scalable solution based on .NET. So in a discussion about scaling on .NET, it absolutely makes sense to listen to his experiences.
  - kondro 16y ago
    Actually, FogBugz is trivial to scale - there is never any more than a few dozen, to hundreds of users for any one project. And Copilot wasn't even envisaged or created by Joel, but was rather a project created by some interns. FogCreek doesn't innovate (they have a bug tracker and a remote support application based on VNC). Joel just happened to be one of the first to blog constantly about technology. He doesn't even allow comments on his posts because he doesn't want to have to deal with people contradicting him (not that he doesn't do an excellent job at this himself). There are so many great developers and businesses out there that you could use as an example to model your business and software on. Use them, not Joel. Let Joel's business and opinions fade away as other smart, innovative companies run rings around his ageing product suite.

nostrademons 16y ago

You almost always have to rewrite your whole system anyway to scale to millions of users. It doesn't matter whether it's PHP, Python, C++, or ASP.NET; it'll have to be rewritten. The real problem is the flip-side of something that the OP mentioned as an advantage: ASP.NET coders tend to be cheaper, less experienced, and hence less likely to know about some critical detail that'll boost adoption, whether it's usability or latency or security. As someone relatively inexperienced himself, the OP can't know what those things are, but it's highly likely that one of them will bite him. In other words, it's correlation, not causation. ASP.NET won't make your site suck. However, ASP.NET biases your talent pool towards developers who suck, and that will make your site suck.

mwdev 16y ago
To me this sounds like my friends who didn't want to go to UT because it 'was too big'. The awesome thing about too big is that if you look hard enough you are almost assured of finding someone like you. Yes there are crappy c# developers. I've also seen some bad ass c#.
- nostrademons 16y ago
  Right, "biasing your talent pool" doesn't mean that everyone in that talent pool sucks. There are certainly some very talented C#/.NET devs. The problem is that when you go to hire, you have very little information about just how badass a prospective dev is, unless you've worked with them before. So if you take a random dev out of the C# pool, chances are he'll be worse than a random dev in the Python pool. http://www.paulgraham.com/pypar.html http://www.paulgraham.com/pypar.html
  - kenjackson 16y ago
    So if you take a random dev out of the C# pool, chances are he'll be worse than a random dev in the Python pool. That was probably true in 1995. But not today. Python has become the new Pascal/Java. Commonly the intro language in CS, so you have a lot of students who graduate using Python regardless of skill. If you said Haskell, then probably. Actually C++ maybe more then either, surprisingly.

blake8086 16y ago

Perhaps you would be interested in purchasing my Scaling Insurance? It costs a ton upfront, and it's likely to kill your nascent company, but if you ever make it big, you'll save a tiny bit of money in the long run!

arethuza 16y ago
Call them Scaling Default Swaps - insurance is regulated!