3 ms·

Golden rule to follow: always optimize first before you attempt scaling. Working for large banks I see this error made every day. Nobody knows how to optimize,

Golden rule to follow: always optimize first before you attempt scaling. Working for large banks I see this error made every day. Nobody knows how to optimize, but everybody knows how to request for couple more servers. Then there is a talk of budgets, hiring more people for maintaining the behemoth, then introducing some pretty exotic and expensive technology to "optimize operations". I have seen yesterday a huge piece of infrastructure with literally terabytes of RAM and a number of GPUs costing at least 250k a year for a batch job that should fit a single server with no problems.

mattlondon 6y ago
But how many engineering-hours would be needed to optimise it to fit into a single server? And what would the risk be that there is a screw up? Sure there might be a few low hanging things to clean up ( i.e. a few places where a data structure could be better picked), but if it needs that much iron then it is probably not a trivial refactor of a few hotspots, but likely a more extensive rewrite. Probably cheaper to just pay 250k than for a couple of engineers taking risks modifying a battle tested bit of code... If it is important enough for the business to justify paying that much for then you'll need to "do things properly" and not do a hatchet-job of a cowboy refactor, but plan things out and test them and have rollout/rollback plans, project management etc.
- lmilcin 6y ago
  Optimisation is typically one time cost. You make a feature A and you spend a bit more time to ensure it works couple times faster or couple times less resources than initial/naive version. Maintaining larger infrastructure is an ongoing cost as you need to provide x times more resources throughout the life of the application. Not only that, but now future development will have to be done in context of large application. Maybe writing a process for a single node could be easy and now you have 100 nodes and it is complex to design your functionality to work correctly on multiple nodes, complex to deploy, complex to resolve issues. Maybe you decide to employ new technology to solve some of those problems but at the cost of creating other problems and also additional complexity. Then maybe you need more people -- and this comes with its own inefficiency. The point really is, most applications could be orders of magnitude more efficient if you just don't make stupid decisions.
metafunctor 6y ago
Agreed; but to clarify, the first thing to do is to scale vertically (get bigger servers). When you're starting hit limits with that, then optimise, within reason. When you're starting to hit limits there, then scale horizontally (add more servers). The point with optimizing before scaling horizontally is to make sure you understand what is going on. Scaling horizontally without a good understanding is going to be a fail-whale shitshow. I'd also note that high availability needs some horizontal scaling.
- greggyb 6y ago
  HA and scaling are not the same thing. High availability represents architectural decisions to maintain operation in the face of subsystem failures. With respect to high availability within a data center, this can be achieved with a passive secondary system and a mechanism to failover to this secondary in the face of a primary failure. Such an HA strategy is valid, but provides no horizontal scalability. Scalability is the ability to serve more traffic. Scaling horizontally implies adding hardware. Though the HA example given above includes additional hardware compared to a single primary node, it does not support any additional traffic. Scaling would imply that the addition of the secondary server allows the system to serve more traffic.
- lmilcin 6y ago
  No, don't scale first. First try to optimize your app to use as little resources as possible, then scale vertically, then horizontally.

mattlondon 6y ago

But how many engineering-hours would be needed to optimise it to fit into a single server? And what would the risk be that there is a screw up? Sure there might be a few low hanging things to clean up ( i.e. a few places where a data structure could be better picked), but if it needs that much iron then it is probably not a trivial refactor of a few hotspots, but likely a more extensive rewrite. Probably cheaper to just pay 250k than for a couple of engineers taking risks modifying a battle tested bit of code... If it is important enough for the business to justify paying that much for then you'll need to "do things properly" and not do a hatchet-job of a cowboy refactor, but plan things out and test them and have rollout/rollback plans, project management etc.

lmilcin 6y ago
Optimisation is typically one time cost. You make a feature A and you spend a bit more time to ensure it works couple times faster or couple times less resources than initial/naive version. Maintaining larger infrastructure is an ongoing cost as you need to provide x times more resources throughout the life of the application. Not only that, but now future development will have to be done in context of large application. Maybe writing a process for a single node could be easy and now you have 100 nodes and it is complex to design your functionality to work correctly on multiple nodes, complex to deploy, complex to resolve issues. Maybe you decide to employ new technology to solve some of those problems but at the cost of creating other problems and also additional complexity. Then maybe you need more people -- and this comes with its own inefficiency. The point really is, most applications could be orders of magnitude more efficient if you just don't make stupid decisions.

metafunctor 6y ago

Agreed; but to clarify, the first thing to do is to scale vertically (get bigger servers). When you're starting hit limits with that, then optimise, within reason. When you're starting to hit limits there, then scale horizontally (add more servers). The point with optimizing before scaling horizontally is to make sure you understand what is going on. Scaling horizontally without a good understanding is going to be a fail-whale shitshow. I'd also note that high availability needs some horizontal scaling.

greggyb 6y ago
HA and scaling are not the same thing. High availability represents architectural decisions to maintain operation in the face of subsystem failures. With respect to high availability within a data center, this can be achieved with a passive secondary system and a mechanism to failover to this secondary in the face of a primary failure. Such an HA strategy is valid, but provides no horizontal scalability. Scalability is the ability to serve more traffic. Scaling horizontally implies adding hardware. Though the HA example given above includes additional hardware compared to a single primary node, it does not support any additional traffic. Scaling would imply that the addition of the secondary server allows the system to serve more traffic.
lmilcin 6y ago
No, don't scale first. First try to optimize your app to use as little resources as possible, then scale vertically, then horizontally.