3 ms·

No doubt there are people who do it for cynical reasons. But at least some people do it sincerely thinking it’s the right choice. It’d be more interesting to ta

by tlb 7y ago

No doubt there are people who do it for cynical reasons. But at least some people do it sincerely thinking it’s the right choice. It’d be more interesting to talk about them, and how they came to make the wrong decision for what they thought were the right reasona.

asdfasgasdgasdg 7y ago
I think it's a bit presumptuous to say that the decision is "wrong". That paper demonstrated that a single server can outperform a small server farm on a toy problem. Nobody, not even google, solves pagerank in production as a batch job. Real problems are often more complex.
- tluyben2 7y ago
  For many workloads, companies have tech leads and CTOs absolutely overarchitecting their stack. If you can run your entire system off of 4 loadbalanced $200/mo servers, why am I seeing hadoop/kafka/kubernetes/etc running on $10k+/mo premoney, so paid with investor money? Sure there are a lot of cases where this is fine, but I would say (from real life) that there are far more cases where this was a pretty poor choice. Usually proven to be even poorer when the tech lead leaves and no-one seemingly having a clue how it all works together, despite the whole docker/kubernetes/ci stories peddled to management. That is usually when I get asked to take a look. In my experience these problems are definitely not always toy problems while some of them can, easily, be ran on 1 server for the expected lifespan of the company because the company will never get that much users/clients/data even though immensely profitable. Not everyone (almost no one) is a FAANG and I find it offensive to use the company money (which can be investor money) to realise the wet dreams of the cto while the case for that architecture and the cost of it, makes 0 sense to the business and bottomline. Obviously in life, things are nuanced and differ case by case but nah, real world problems are, generally, not more complex. They are more complex in some (rare) cases, like the one you named. But most companies are not doing anything like that, but they do have their cto’s gearing up for that incredibly unlikely future.
FpUser 7y ago
Talked to many people lit you mentioned. Large percentage repeats these points: 1) We always wait for database, so performance of our code does not matter. This comes from places where ironically the whole database can fit into RAM. 10+ gbps connectivity - well it is almost commodity for business so the latency is not much of a bottleneck. Fast IO to store data - well imagine array of Optane drives. Not very cheap but really peanuts for normal business. All this means that purposely built data server which is a heart of many type of business residing on decent computer, can be blindingly fast and most of the businesses will never ever outgrow it. I've written quite a few of such servers serving loads of businesses in NA so hopefully my experience is not irrelevant. 2) Scripting languages are so convenient to use and save so much time and this is what matters as the salary is a main expense. - Personally I use those mostly for management/deployment/etc.etc type of scripts and maybe to quickly test some small ideas. I clearly see their benefits there. Anything that resembles a product that would actually run business. Sorry but experienced developer can implement those just as fast as in any scripting language and it will save a ton on maintenance. 3) And finally do not do premature optimization mantra . - In my opinion switching from scripting language to something like C++ has nothing to do with premature optimization. Developer can be very productive with compiled languages as they also have megatons of libraries for any imaginable task.
- kortilla 7y ago
  > Sorry but experienced developer can implement those just as fast as in any scripting language and it will save a ton on maintenance. This is just a no true Scotsman argument. For 10 years I’ve watched python/ruby shops drastically outpace projects in C++/Java shops. What you’re failing to realize is how trivial most apps are and how fast fully functional back ends can be created with frameworks in those languages (django/rails). Your bit about maintenance is also bullshit. I’ve spent entire days peeling apart complex C++ code bases to make a small change to some core abstraction. Ability to maintain code is entirely up to how well organized and documented it is. It has nothing to do with “scripting”.
  - AmericanChopper 7y ago
    > What you’re failing to realize is how trivial most apps are and how fast fully functional back ends can be created with frameworks in those languages A vast majority of web dev is incredibly simple. People tend to easily convince themselves that their app is very special and requires a lot of complex solutions to make it work. There’s a lot of incentives that lead people to that conclusion, but most of the time following the golden path laid out for you by an existing framework is going to produce a better product with much less effort.
0x445442 7y ago
> It’d be more interesting to talk about them, and how they came to make the wrong decision for what they thought were the right reason My guess, essentially holding their finger to the wind. Most orgs can't afford to do real engineering.

by tlb 7y ago

asdfasgasdgasdg 7y ago

I think it's a bit presumptuous to say that the decision is "wrong". That paper demonstrated that a single server can outperform a small server farm on a toy problem. Nobody, not even google, solves pagerank in production as a batch job. Real problems are often more complex.

tluyben2 7y ago
For many workloads, companies have tech leads and CTOs absolutely overarchitecting their stack. If you can run your entire system off of 4 loadbalanced $200/mo servers, why am I seeing hadoop/kafka/kubernetes/etc running on $10k+/mo premoney, so paid with investor money? Sure there are a lot of cases where this is fine, but I would say (from real life) that there are far more cases where this was a pretty poor choice. Usually proven to be even poorer when the tech lead leaves and no-one seemingly having a clue how it all works together, despite the whole docker/kubernetes/ci stories peddled to management. That is usually when I get asked to take a look. In my experience these problems are definitely not always toy problems while some of them can, easily, be ran on 1 server for the expected lifespan of the company because the company will never get that much users/clients/data even though immensely profitable. Not everyone (almost no one) is a FAANG and I find it offensive to use the company money (which can be investor money) to realise the wet dreams of the cto while the case for that architecture and the cost of it, makes 0 sense to the business and bottomline. Obviously in life, things are nuanced and differ case by case but nah, real world problems are, generally, not more complex. They are more complex in some (rare) cases, like the one you named. But most companies are not doing anything like that, but they do have their cto’s gearing up for that incredibly unlikely future.

FpUser 7y ago

Talked to many people lit you mentioned. Large percentage repeats these points: 1) We always wait for database, so performance of our code does not matter. This comes from places where ironically the whole database can fit into RAM. 10+ gbps connectivity - well it is almost commodity for business so the latency is not much of a bottleneck. Fast IO to store data - well imagine array of Optane drives. Not very cheap but really peanuts for normal business. All this means that purposely built data server which is a heart of many type of business residing on decent computer, can be blindingly fast and most of the businesses will never ever outgrow it. I've written quite a few of such servers serving loads of businesses in NA so hopefully my experience is not irrelevant. 2) Scripting languages are so convenient to use and save so much time and this is what matters as the salary is a main expense. - Personally I use those mostly for management/deployment/etc.etc type of scripts and maybe to quickly test some small ideas. I clearly see their benefits there. Anything that resembles a product that would actually run business. Sorry but experienced developer can implement those just as fast as in any scripting language and it will save a ton on maintenance. 3) And finally do not do premature optimization mantra . - In my opinion switching from scripting language to something like C++ has nothing to do with premature optimization. Developer can be very productive with compiled languages as they also have megatons of libraries for any imaginable task.

kortilla 7y ago
> Sorry but experienced developer can implement those just as fast as in any scripting language and it will save a ton on maintenance. This is just a no true Scotsman argument. For 10 years I’ve watched python/ruby shops drastically outpace projects in C++/Java shops. What you’re failing to realize is how trivial most apps are and how fast fully functional back ends can be created with frameworks in those languages (django/rails). Your bit about maintenance is also bullshit. I’ve spent entire days peeling apart complex C++ code bases to make a small change to some core abstraction. Ability to maintain code is entirely up to how well organized and documented it is. It has nothing to do with “scripting”.
- AmericanChopper 7y ago
  > What you’re failing to realize is how trivial most apps are and how fast fully functional back ends can be created with frameworks in those languages A vast majority of web dev is incredibly simple. People tend to easily convince themselves that their app is very special and requires a lot of complex solutions to make it work. There’s a lot of incentives that lead people to that conclusion, but most of the time following the golden path laid out for you by an existing framework is going to produce a better product with much less effort.

0x445442 7y ago

> It’d be more interesting to talk about them, and how they came to make the wrong decision for what they thought were the right reason My guess, essentially holding their finger to the wind. Most orgs can't afford to do real engineering.