5 ms·

It’s interesting because while this is an extreme case of performance improvement, the ROI doesn’t seem amazing. "rewriting in another language reduced the num

by hcks 3y ago

It’s interesting because while this is an extreme case of performance improvement, the ROI doesn’t seem amazing. "rewriting in another language reduced the number of servers by 95%" is hard to beat, but at the same time, this saves "only" 2m a year, or about 0.3% of FY22 cost of revenue (per another post) Pinterest per employee revenue seems to be around 1m, which basically suggest that this could even be a worse than average allocation of resources. My takeaway would be "don’t bother with this kind of optimisation before you reach a scale where you can afford to do marginal improvements"

manmal 3y ago
Wouldn't you also need a smaller ops team that way, further reducing your costs?
- diarrhea 3y ago
  Managing 200 Python servers/nodes certainly sounds operationally challenging.
  - magicCat 3y ago
    i disagree, not much more then 2 servers.. you should automate it fully for documentation purposes alone at this scale & test failover regulary if you do this with 4 or 200 machines, does not make a big difference.
    - di4na 3y ago
      Actually it does because the chance of hitting strange edge cases grow significantly, as do the runtime of your deployment, plus networking problems risks.
    - manmal 3y ago
      Running 200 servers will force you to automate everything and also handle (relatively) rare edge cases, meaning added complexity, huge up front dev costs, and a continuous dev effort just to stay afloat. Work has to be balanced in a clever way, making tradeoffs that you otherwise wouldn't need to make. The mental model shifts from individual servers to a whole ecosystem that shows emergent behavior (= things break in a totally new way). Maybe some of that also applies to 4 servers, but to a way lesser extent.
boxed 3y ago
2 million dollars a year is "marginal improvement" just because the sales and customer support team is big? That's your logic?
impulser_ 3y ago
It more than just server costs. They reduced the complexity of their services by a lot. Having to maintain 95% less servers is worth it even if they didn't save any money IMO. This also could lead to them reducing their engineering team that maintains these services which would reduce costs even more.
- paxys 3y ago
  Can switching to an esoteric functional language from Python and Java really be considered reducing complexity? No matter how well it is written, I'm willing to bet that way fewer people in the company/industry understand the new codebase and can make changes to it.
  - weatherlight 3y ago
    > Can switching to an esoteric functional language from Python and Java really be considered reducing complexity? It's not esoteric. It's a modern programing language, with 35 years behind its Virtual machine. 1 in 3 phone calls you've ever made was routed through a telephon switch written in a lang implemented in BEAM virtual machine. WhatsApp was written in a lang implemented on the BEAM. > I'm willing to bet that way fewer people in the company/industry understand the new codebase and can make changes to it. It's really not hard to learn. It's not Haskell, It's not Coq, It's not Brainfuck. You could pick up the syntax and the major semantics in a weekend.
- robbintt 3y ago
  This isn't a huge difference most of the time. Once you have cluster management at scale, 3 vs 3000 are pretty similar.

by hcks 3y ago

manmal 3y ago

Wouldn't you also need a smaller ops team that way, further reducing your costs?

diarrhea 3y ago
Managing 200 Python servers/nodes certainly sounds operationally challenging.
- magicCat 3y ago
  i disagree, not much more then 2 servers.. you should automate it fully for documentation purposes alone at this scale & test failover regulary if you do this with 4 or 200 machines, does not make a big difference.
  - di4na 3y ago
    Actually it does because the chance of hitting strange edge cases grow significantly, as do the runtime of your deployment, plus networking problems risks.
  - manmal 3y ago
    Running 200 servers will force you to automate everything and also handle (relatively) rare edge cases, meaning added complexity, huge up front dev costs, and a continuous dev effort just to stay afloat. Work has to be balanced in a clever way, making tradeoffs that you otherwise wouldn't need to make. The mental model shifts from individual servers to a whole ecosystem that shows emergent behavior (= things break in a totally new way). Maybe some of that also applies to 4 servers, but to a way lesser extent.

boxed 3y ago

2 million dollars a year is "marginal improvement" just because the sales and customer support team is big? That's your logic?

impulser_ 3y ago

It more than just server costs. They reduced the complexity of their services by a lot. Having to maintain 95% less servers is worth it even if they didn't save any money IMO. This also could lead to them reducing their engineering team that maintains these services which would reduce costs even more.

paxys 3y ago
Can switching to an esoteric functional language from Python and Java really be considered reducing complexity? No matter how well it is written, I'm willing to bet that way fewer people in the company/industry understand the new codebase and can make changes to it.
- weatherlight 3y ago
  > Can switching to an esoteric functional language from Python and Java really be considered reducing complexity? It's not esoteric. It's a modern programing language, with 35 years behind its Virtual machine. 1 in 3 phone calls you've ever made was routed through a telephon switch written in a lang implemented in BEAM virtual machine. WhatsApp was written in a lang implemented on the BEAM. > I'm willing to bet that way fewer people in the company/industry understand the new codebase and can make changes to it. It's really not hard to learn. It's not Haskell, It's not Coq, It's not Brainfuck. You could pick up the syntax and the major semantics in a weekend.
robbintt 3y ago
This isn't a huge difference most of the time. Once you have cluster management at scale, 3 vs 3000 are pretty similar.