4 ms·

Last time I built a server project from scratch it was on a 300MHz quad-core Xeon base, which hosted around 150 simultaneous web users. It took some effort to m

Last time I built a server project from scratch it was on a 300MHz quad-core Xeon base, which hosted around 150 simultaneous web users. It took some effort to make it scale like that but it was worth it. My hardware maintenance was low because we really maximized the software capacity. In modern times, RAM and CPUs are more than 10 times bigger and faster, but I am seeing people get around 25 times LESS out of them, because they choose terrible tools, don't benchmark, and generally don't care. Now (different company) our app server pool has 150 nodes and each serves 4 users. The application complexity is significantly smaller than what I did 13 years ago. I sincerely doubt any of my coworkers have read this document. It shows.

fiatmoney 13y ago
"It took some effort to make it scale" It's often a lot easier to rely on horizontal scaling, especially on a strategic level when you're deciding whether to trust your organization's technical ability (which really means management's hiring ability) or your ability to call AWS and spin up another hundred servers. "Easier" doesn't mean "cheaper" or "faster" in the long run, but it is lower-variance for sure.
- philwelch 13y ago
  Horizontal scaling isn't just about throwing hardware at the problem. It's about distributing your application over n number of nodes without single points of failure.
  - jcrites 13y ago
    I'd argue there's one more criterion for horizontal scalability: an increase in hardware capacity (number of nodes N) must yield a proportional increase in system throughput. In other words, horizontal scalability means being _able_ to throw hardware at the problem :-) Anyway, I agree with fiatmoney's sentiment that if you're in a corporate environment where your goal is growth, it's often best to focus on ensuring that your systems can grow along with demand, in a confident, easy, operations-free way -- as opposed to ensuring you've eked every bit of performance out of the hardware you have. In many business projects, the cost of human engineering time exceeds the cost of hardware. Even when considering large-scale systems, it makes sense to scrutinize optimizations. If you had time to implement only one of these two equal-effort projects, which would you choose? (a) An optimization project expected to reduce a recurring yearly cost by $1 million (b) A new feature development project expected to earn an additional $2 million yearly recurring revenue? I've found it effective to look at all engineering decisions as business decisions. From a business perspective, the mistake at the center of fallacies like premature optimization is the failure to consider opportunity cost. (On the other hand sometimes pushing a technical project as far as you can is its own reward. Not everything in life has to be about business. Whether or not the results are widely relevant to typical businesses, I look forward to seeing the research into the C10K and C10M problems!)
    - justincormack 13y ago
      People are often too optimistic about these additional $2m, while cutting yearly cost by $1m would allow you to hire several more developers, or increase your runway to profitability. There is a business case for both...
    - sokoloff 13y ago
      > If you had time to implement only one of these two equal-effort projects, which would you choose? Impossible to answer without knowing the gross margin of your business. For a big-box retailer (low margins), take the first. For a high-(gross)-margin business, take the second.
davidw 13y ago
A lot of the calculation is the $$$ you make per connection. For a SaaS kind of thing, you can generally keep that high enough to not worry about squeezing every last drop of efficiency out of things. A much bigger worry for most startup-ish companies is finding product market fit.
- DrJ 13y ago
  While I would agree that trying to squeeze every last drop of efficiency may not be worth the time/money for the company. I think the parent is implying that people are wasting a lot of extra resources due to poor tools, poor coding, poor profiling, poor maintenance, etc. I would agree though that for startups, the problem is finding the place to $$$, it is important to know what the cost per user/connection/instance is and being able to control or monitor it.

fiatmoney 13y ago

"It took some effort to make it scale" It's often a lot easier to rely on horizontal scaling, especially on a strategic level when you're deciding whether to trust your organization's technical ability (which really means management's hiring ability) or your ability to call AWS and spin up another hundred servers. "Easier" doesn't mean "cheaper" or "faster" in the long run, but it is lower-variance for sure.

philwelch 13y ago
Horizontal scaling isn't just about throwing hardware at the problem. It's about distributing your application over n number of nodes without single points of failure.
- jcrites 13y ago
  I'd argue there's one more criterion for horizontal scalability: an increase in hardware capacity (number of nodes N) must yield a proportional increase in system throughput. In other words, horizontal scalability means being _able_ to throw hardware at the problem :-) Anyway, I agree with fiatmoney's sentiment that if you're in a corporate environment where your goal is growth, it's often best to focus on ensuring that your systems can grow along with demand, in a confident, easy, operations-free way -- as opposed to ensuring you've eked every bit of performance out of the hardware you have. In many business projects, the cost of human engineering time exceeds the cost of hardware. Even when considering large-scale systems, it makes sense to scrutinize optimizations. If you had time to implement only one of these two equal-effort projects, which would you choose? (a) An optimization project expected to reduce a recurring yearly cost by $1 million (b) A new feature development project expected to earn an additional $2 million yearly recurring revenue? I've found it effective to look at all engineering decisions as business decisions. From a business perspective, the mistake at the center of fallacies like premature optimization is the failure to consider opportunity cost. (On the other hand sometimes pushing a technical project as far as you can is its own reward. Not everything in life has to be about business. Whether or not the results are widely relevant to typical businesses, I look forward to seeing the research into the C10K and C10M problems!)
  - justincormack 13y ago
    People are often too optimistic about these additional $2m, while cutting yearly cost by $1m would allow you to hire several more developers, or increase your runway to profitability. There is a business case for both...
  - sokoloff 13y ago
    > If you had time to implement only one of these two equal-effort projects, which would you choose? Impossible to answer without knowing the gross margin of your business. For a big-box retailer (low margins), take the first. For a high-(gross)-margin business, take the second.

davidw 13y ago

A lot of the calculation is the $$$ you make per connection. For a SaaS kind of thing, you can generally keep that high enough to not worry about squeezing every last drop of efficiency out of things. A much bigger worry for most startup-ish companies is finding product market fit.

DrJ 13y ago
While I would agree that trying to squeeze every last drop of efficiency may not be worth the time/money for the company. I think the parent is implying that people are wasting a lot of extra resources due to poor tools, poor coding, poor profiling, poor maintenance, etc. I would agree though that for startups, the problem is finding the place to $$$, it is important to know what the cost per user/connection/instance is and being able to control or monitor it.