4 ms·

I've never understood the obsession with scale when there are so many challenging problems left in user interface technology are relatively ignored. The room fo

I've never understood the obsession with scale when there are so many challenging problems left in user interface technology are relatively ignored. The room for innovation is staggering, yet we obsess over nosql.

ehsanul 16y ago
Can you elaborate? Any particular problems you have in mind?
mechanical_fish 16y ago
Why is scaling such a popular obsession? Some hypotheses: It's survivor bias. When you go to found a venture, where do you look for inspiration? You find a very successful analogue and read all about them. For example, if you want to found a social network, you look to Facebook for inspiration. And what is something that many incredibly successful web properties are worried about? Scaling. It's objective, or it feels objective. You load the web page, it takes 20ms. You make a tweak and load the page again, it takes 15ms. You can measure benchmarks, and then you can point to the benchmarks getting better and claim that you were objectively successful. The fact that there's a scoring system lures people into the game. It's fun to think about. Nobody likes the guy who points out the danger that the product will flop. Everyone likes the guy who points out that the product might just be so successful that we'll be overwhelmed by success. You get to sound like an ambitious, expansive optimist when planning out all the fancy hardware you're going to need. You're focusing on the positive, just as all the self-help manuals tell you to do. It's Parkinson's Law, in its original form: By default, the size of a business unit grows until it is limited by resources, because the path to power in an organization is to be big. Lots of subordinates, lots of infrastructure, lots of budget. You might think that an organization would lavishly reward the person who could singlehandedly run the entire web infrastructure off of a single PC, but often the opposite is the case. So it pays to puff up the scalability problem as big as you can, in order to get the resources to build a big team, so that you gain political power from being the head of a big team...
- peripitea 16y ago
  >It's survivor bias. I really think it's as simple as that. When you read about a successful startup, they invariably mention the troubles they had scaling. So you immediately think "how can I avoid scaling troubles?", when you should be thinking "how can I build a site that will have scaling troubles?"
  - dice 16y ago
    >you should be thinking "how can I build a site that will have scaling troubles?" I work for a company that does consulting for startups, and occasionally one of our clients takes off like a rocket. Everyone has scaling problems, even the people who have thought about and worked on scaling. Load tests and projections are only so good: at the end of the day theory will meet practice and we all know who wins there. We always tell them the same thing, "these are good problems to have". Everyone has a sleepless week or so until the bottlenecks are cleared, and then life is back to normal (except now people have heard about that thing you've been working on for the past couple years).

ehsanul 16y ago

Can you elaborate? Any particular problems you have in mind?

mechanical_fish 16y ago

Why is scaling such a popular obsession? Some hypotheses: It's survivor bias. When you go to found a venture, where do you look for inspiration? You find a very successful analogue and read all about them. For example, if you want to found a social network, you look to Facebook for inspiration. And what is something that many incredibly successful web properties are worried about? Scaling. It's objective, or it feels objective. You load the web page, it takes 20ms. You make a tweak and load the page again, it takes 15ms. You can measure benchmarks, and then you can point to the benchmarks getting better and claim that you were objectively successful. The fact that there's a scoring system lures people into the game. It's fun to think about. Nobody likes the guy who points out the danger that the product will flop. Everyone likes the guy who points out that the product might just be so successful that we'll be overwhelmed by success. You get to sound like an ambitious, expansive optimist when planning out all the fancy hardware you're going to need. You're focusing on the positive, just as all the self-help manuals tell you to do. It's Parkinson's Law, in its original form: By default, the size of a business unit grows until it is limited by resources, because the path to power in an organization is to be big. Lots of subordinates, lots of infrastructure, lots of budget. You might think that an organization would lavishly reward the person who could singlehandedly run the entire web infrastructure off of a single PC, but often the opposite is the case. So it pays to puff up the scalability problem as big as you can, in order to get the resources to build a big team, so that you gain political power from being the head of a big team...

peripitea 16y ago
>It's survivor bias. I really think it's as simple as that. When you read about a successful startup, they invariably mention the troubles they had scaling. So you immediately think "how can I avoid scaling troubles?", when you should be thinking "how can I build a site that will have scaling troubles?"
- dice 16y ago
  >you should be thinking "how can I build a site that will have scaling troubles?" I work for a company that does consulting for startups, and occasionally one of our clients takes off like a rocket. Everyone has scaling problems, even the people who have thought about and worked on scaling. Load tests and projections are only so good: at the end of the day theory will meet practice and we all know who wins there. We always tell them the same thing, "these are good problems to have". Everyone has a sleepless week or so until the bottlenecks are cleared, and then life is back to normal (except now people have heard about that thing you've been working on for the past couple years).