7 ms·

I like a lot of the answers, but something else I'd add: lots of "popular" architectures from the late 00s and early 2010s have fallen by the wayside because pe

I like a lot of the answers, but something else I'd add: lots of "popular" architectures from the late 00s and early 2010s have fallen by the wayside because people realized "You're not Google. Your company will never be Google." That is, there was a big desire around that time period to "build it how the big successful companies built it." But since then, a lot of us have realized that complexity isn't necessary for 99% of companies. When you couple that with hardware and standard databases getting much better, there are just fewer and fewer companies who need all of these "scalability tricks". My bar for "Is there a reason we can't just do this all in Postgres?" is much, much higher than it was a decade ago.

XCSme 2y ago
So true, people optimize prematuriley for when they'll have 100m monthly users, when they have no product yet and won't likely reach 100k users in many years (which can run on a single $100/m dedicated machine...).
nosefrog 2y ago
To be fair, I worked on multiple projects removing queues at Google, so it's more than just that.
- keredson 2y ago
  and mandates that virtually all new projects not directly use borg/kubernetes.
  - azaras 2y ago
    Can you extend it? How do they deploy, and where do they deploy their projects?
    - deleted 2y ago
      [deleted]
  - benterix 2y ago
    That's interesting. What's the rationale behind that?
    - arantius 2y ago
      I left Google in 2022, but I remember seeing the beginning of this. The system that got built up to do all the things Google wanted to do -- previously "simple" things, orchestrated on raw hosting (Borg) -- got too big and complex. (Partially because it was too flexible, so too many projects did too many unique things, and running them grew difficult to match.) So they started adding abstraction layers, to make it "simpler". (Which as always means the common/easy case got easier, and the rare/difficult case got harder. The team I was on at the time couldn't use the new thing for (most) of what we did, because our needs were incompatible. But we did enough other compatible things that I started learning about it/using it.) (Personally I'm sure it's doomed to failure. But partially because most all technology operates in cycles/like a pendulum. The abstraction layers will grow too onerous and limiting, and the solution will be to dive closer to the metal again.)
    - samus 2y ago
      I guess such a service gets coupled too strongly to that platform, and major engineering effort is required to deploy it the old-school way.
steve1977 2y ago
> because people realized "You're not Google. Your company will never be Google." Is that also why almost no one is using microservices and Kubernetes?
- k8sToGo 2y ago
  Kubernetes brings more than just being Google. In a way it’s also an ecosystem.
  - steve1977 2y ago
    That‘s nice, but the question is if you (i.e. your company) needs this ecosystem.
    - k8sToGo 2y ago
      Need no. But it’s nice especially since I have the know how. Creating a new deployment is just super fast. Developers can deploy their own apps etc. And then there is Helm. If we only decide by “need” then most of the time we also wouldn’t need object oriented programming.
      - sofixa 2y ago
        > And then there is Helm Right, who doesn't want to template hundreds of lines of code in a language that uses whitespace for logic and was never made neither for templating nor complex long documents(YAML)? What could possibly go wrong ("error missing xxx at line 728, but it might be a problem elsewhere").
        xnickb 2y ago
        But my infrastructure is code! Can't you see how it's all in git?
        benterix 2y ago
ithkuil 2y ago
We also have much much bigger single machines available for reasonable money. So a lot of reasonable workloads can fit in one machine now that used to require a small cluster
- didgetmaster 2y ago
  Hardware has certainly progressed significantly over the years, but the size of workloads has also grown. The big question is: Does a 'reasonable' workloads today fit on a single machine better than a 'reasonable' workload did 20 years ago?
- Dalewyn 2y ago
  It's kind of mind boggling just how powerful mundane desktop computers have gotten, let alone server hardware. Think about it: That 20 core CPU (eg: i7 14700K) you can buy for just a couple hundred dollars today would have been supercomputer hardware costing tens or hundreds of thousands of dollars just a decade ago.
  - EVa5I7bHFq9mnYK 2y ago
    According to geekbench, an i9 4790 processor released a decade ago is ~5 times slower than i7 14700. 4790's go for $30 at ebay, vs $300 for 14700, so price/performance seems to be in favor of older hardware:)
    - Breza 2y ago
      You might like https://labgopher.com/ https://labgopher.com/
    - Dalewyn 2y ago
      On the other hand, the E7-8890 v3 (the closest equivalent to a 14700K in core count at the time from a quick glance) had an MSRP of $7174.00[1]. So maybe I was a bit too high on the pricing earlier, but my point still stands that the computing horsepower we have such easy access to today was literal big time magic just a decade ago. [1]: https://ark.intel.com/content/www/us/en/ark/products/84685/intel-xeon-processor-e7-8890-v3-45m-cache-2-50-ghz.html https://ark.intel.com/content/www/us/en/ark/products/84685/i...
peoplefromibiza 2y ago
> "You're not Google. Your company will never be Google." I'm not sure people realize this now more than then. I was there back then and we surely knew we would never be Google hence we didn't need to "scale" the same way they did. Nowadays every project I start begins with a meeting where is presented a document describing the architecture we are going to implement, using AWS of course, because "auto-scale" right?, and 9/10 it includes CloudFront, which is a CDN and I don't really understand why this app I am developing, which is basically an API gateway with some customization that made Ngnix slightly less than ideal (but still perfect for the job), and that averages to 5 rps needs a CDN in front of it... (or AWS or auto-scaling or AWS lambda, for that matter)
- samus 2y ago
  The autoscaling is nice because a lot of performance issues just get resolved without much meddling by the ops team, buying time for proper optimizations should it get out of hand. The disadvantage is that people don't think hard about performance requirements anymore. Premature optimization is bad, but it's also a warning sign if a project has no clue whatsoever how intensely the system is going to be used.
- _xiaz 2y ago
  In defense of CDNs they're also pretty neat for cutting down latency, which benefits even the first customer. Of course that only helps if you don't end up shoving dozens of MBs in Javascript/JSON over the wire.
  - ElevenLathe 2y ago
    Putting your app behind a CDN also gives you some cheap defense against (most, casual) DDoS.
    - peoplefromibiza 2y ago
      usually, but this app is not even exposed publicly to the internet
- 2y ago
w10-1 2y ago
> You're not Google. Your company will never be Google True, but the CTO comes from twitter/meta/google/some open-source big-data project, the director loves databases, etc. So we have 40-100 people managing queues with events driven from database journals. Everyone sees how and why it evolved that way. No one has the skill or political capital to change it. And we spend most of our time on maintenance tasked as "upgrades", in a culture with "a strong role for devops".
- phyrex 2y ago
  Meta, where they run a PHP monolith with mysql? ;)

XCSme 2y ago

So true, people optimize prematuriley for when they'll have 100m monthly users, when they have no product yet and won't likely reach 100k users in many years (which can run on a single $100/m dedicated machine...).

nosefrog 2y ago

To be fair, I worked on multiple projects removing queues at Google, so it's more than just that.

keredson 2y ago
and mandates that virtually all new projects not directly use borg/kubernetes.
- azaras 2y ago
  Can you extend it? How do they deploy, and where do they deploy their projects?
  - deleted 2y ago
    [deleted]
- benterix 2y ago
  That's interesting. What's the rationale behind that?
  - arantius 2y ago
    I left Google in 2022, but I remember seeing the beginning of this. The system that got built up to do all the things Google wanted to do -- previously "simple" things, orchestrated on raw hosting (Borg) -- got too big and complex. (Partially because it was too flexible, so too many projects did too many unique things, and running them grew difficult to match.) So they started adding abstraction layers, to make it "simpler". (Which as always means the common/easy case got easier, and the rare/difficult case got harder. The team I was on at the time couldn't use the new thing for (most) of what we did, because our needs were incompatible. But we did enough other compatible things that I started learning about it/using it.) (Personally I'm sure it's doomed to failure. But partially because most all technology operates in cycles/like a pendulum. The abstraction layers will grow too onerous and limiting, and the solution will be to dive closer to the metal again.)
  - samus 2y ago
    I guess such a service gets coupled too strongly to that platform, and major engineering effort is required to deploy it the old-school way.

steve1977 2y ago

> because people realized "You're not Google. Your company will never be Google." Is that also why almost no one is using microservices and Kubernetes?

k8sToGo 2y ago
Kubernetes brings more than just being Google. In a way it’s also an ecosystem.
- steve1977 2y ago
  That‘s nice, but the question is if you (i.e. your company) needs this ecosystem.
  - k8sToGo 2y ago
    Need no. But it’s nice especially since I have the know how. Creating a new deployment is just super fast. Developers can deploy their own apps etc. And then there is Helm. If we only decide by “need” then most of the time we also wouldn’t need object oriented programming.
    - sofixa 2y ago
      > And then there is Helm Right, who doesn't want to template hundreds of lines of code in a language that uses whitespace for logic and was never made neither for templating nor complex long documents(YAML)? What could possibly go wrong ("error missing xxx at line 728, but it might be a problem elsewhere").
      - xnickb 2y ago
        But my infrastructure is code! Can't you see how it's all in git?
        benterix 2y ago

ithkuil 2y ago

We also have much much bigger single machines available for reasonable money. So a lot of reasonable workloads can fit in one machine now that used to require a small cluster

didgetmaster 2y ago
Hardware has certainly progressed significantly over the years, but the size of workloads has also grown. The big question is: Does a 'reasonable' workloads today fit on a single machine better than a 'reasonable' workload did 20 years ago?
Dalewyn 2y ago
It's kind of mind boggling just how powerful mundane desktop computers have gotten, let alone server hardware. Think about it: That 20 core CPU (eg: i7 14700K) you can buy for just a couple hundred dollars today would have been supercomputer hardware costing tens or hundreds of thousands of dollars just a decade ago.
- EVa5I7bHFq9mnYK 2y ago
  According to geekbench, an i9 4790 processor released a decade ago is ~5 times slower than i7 14700. 4790's go for $30 at ebay, vs $300 for 14700, so price/performance seems to be in favor of older hardware:)
  - Breza 2y ago
    You might like https://labgopher.com/ https://labgopher.com/
  - Dalewyn 2y ago
    On the other hand, the E7-8890 v3 (the closest equivalent to a 14700K in core count at the time from a quick glance) had an MSRP of $7174.00[1]. So maybe I was a bit too high on the pricing earlier, but my point still stands that the computing horsepower we have such easy access to today was literal big time magic just a decade ago. [1]: https://ark.intel.com/content/www/us/en/ark/products/84685/intel-xeon-processor-e7-8890-v3-45m-cache-2-50-ghz.html https://ark.intel.com/content/www/us/en/ark/products/84685/i...

peoplefromibiza 2y ago

> "You're not Google. Your company will never be Google." I'm not sure people realize this now more than then. I was there back then and we surely knew we would never be Google hence we didn't need to "scale" the same way they did. Nowadays every project I start begins with a meeting where is presented a document describing the architecture we are going to implement, using AWS of course, because "auto-scale" right?, and 9/10 it includes CloudFront, which is a CDN and I don't really understand why this app I am developing, which is basically an API gateway with some customization that made Ngnix slightly less than ideal (but still perfect for the job), and that averages to 5 rps needs a CDN in front of it... (or AWS or auto-scaling or AWS lambda, for that matter)

samus 2y ago
The autoscaling is nice because a lot of performance issues just get resolved without much meddling by the ops team, buying time for proper optimizations should it get out of hand. The disadvantage is that people don't think hard about performance requirements anymore. Premature optimization is bad, but it's also a warning sign if a project has no clue whatsoever how intensely the system is going to be used.
_xiaz 2y ago
In defense of CDNs they're also pretty neat for cutting down latency, which benefits even the first customer. Of course that only helps if you don't end up shoving dozens of MBs in Javascript/JSON over the wire.
- ElevenLathe 2y ago
  Putting your app behind a CDN also gives you some cheap defense against (most, casual) DDoS.
  - peoplefromibiza 2y ago
    usually, but this app is not even exposed publicly to the internet
2y ago

w10-1 2y ago

> You're not Google. Your company will never be Google True, but the CTO comes from twitter/meta/google/some open-source big-data project, the director loves databases, etc. So we have 40-100 people managing queues with events driven from database journals. Everyone sees how and why it evolved that way. No one has the skill or political capital to change it. And we spend most of our time on maintenance tasked as "upgrades", in a culture with "a strong role for devops".

phyrex 2y ago
Meta, where they run a PHP monolith with mysql? ;)