6 ms·

I love these Twitter takes. They say something to get attention but if you look at it for more than a second it's.. it's just nothing data Comparing a troubles

by corobo 5y ago

I love these Twitter takes. They say something to get attention but if you look at it for more than a second it's.. it's just nothing data Comparing a troubleshooting guide to running a site on a couple of servers is a bit too different for me. Compare it to a troubleshooting guide for those two servers, let's see how they stack up. No using any "ask {specific person}" either Don't get me wrong, kubernetes is overkill for most side project level things. I don't disagree, I just like to see things knocked down a peg fairly! Also as mentioned by this tweeter they use more than 2 servers anyway https://twitter.com/shadowmanos/status/1434980544740306947 https://twitter.com/shadowmanos/status/1434980544740306947 They could probably save on resources and maintenance effort if they switched to containers assuming this is still the same or more > This is #1 in a very long series of posts on Stack Overflow’s architecture. Welcome.

Cthulhu_ 5y ago
It's taking one source - look at how these people solved it! - and trying to apply it to others. SO is relatively simple; it's basically customized forum software which is a solved problem that has been around for decades. A junior dev can build an alternative, and it can be built using tried and true solutions like MySQL + PHP, which are horizontally scalable with database sharding, read replicas, and maybe stuff like memcached to accumulate votes before updating the database or a CDN for caching static files. Google has different problems and different workloads, and they have hundreds of times more applications with thousands of times more load. Apples and oranges.
- JimDabell 5y ago
  > it can be built using tried and true solutions like MySQL + PHP, which are horizontally scalable with database sharding, read replicas, and maybe stuff like memcached to accumulate votes before updating the database or a CDN for caching static files. > Google has different problems and different workloads Which of these do you think most organisations most closely resemble? I don’t think anybody would disagree if you said that you should use Kubernetes for organisations that resemble Google. But most organisations don’t look anything like Google. They look a lot more like Stack Overflow. So the “You don’t need this…” statement holds true for almost everyone.
  - p_l 5y ago
    Stack Overflow is also a) old system built before a lot of current management tech was available openly b) Very single-application centered. EDIT: To expand on it - they had to manually build a lot of what could be much easier handled today, and thus had no incentive to change later. Interestingly enough, SO has a lot of moving parts distributed over multiple servers, even if all of those servers used to fit into 1-3 racks per DC.
    - andrewingram 5y ago
      The single vs multiple application centric thing is a point that's not made enough. Google is basically now a company all about spinning up new products, rather than just periodically adding a new piece of tech to an existing product. As a layperson, Kubernetes _appears_ to be about making it easier to spin up (and run) new products, so it feels more appropriate for Google-like org types.
      - p_l 5y ago
        There are also a lot many more applications one might need to run, especially when scaling beyond one-two people running all development on their laptops/desktops and not having a company yet - especially when one doesn't have valley-style funding. Also these days you might not want to implement login functionality from scratch, or have better logging, monitoring, etc. and that might involve running more applications than just your LAMP stack.
- tester34 5y ago
  >A junior dev can build an alternative Of course junior dev can do it, the same way junior dev can make Youtube it'll work as long as there's less than 100 concurrent users on SO and less than 50 4K 20min videos on youtube
  - piva00 5y ago
    Sometimes I question myself if people posting these "a junior dev can build this product" haven't really worked with systems at scale and all of the myriad of issues that scale brings or if they are being the usual hyper-optimistic-dev that isn't considering much past the proof-of-concept point of a product. Either way is quite baffling how common this kind of comment is, almost a decade reading Hacker News and it pops up constantly.
    - corobo 5y ago
      It also doesn't matter if a junior dev could do it if they just don't do it If it's so trivial, go be rich!
      - piva00 5y ago
        I remember this post from gregdoesit [1] as a good example of how much complexity is hidden under seemingly trivial interfaces. It's about Uber's supposedly bloated app, I believe that anyone who has worked at a scale where they had to support different national legal systems in their code would know that maintaining a large-scale product is nothing trivial. [1] https://news.ycombinator.com/item?id=25376346 https://news.ycombinator.com/item?id=25376346
- krageon 5y ago
  > a solved problem Except 99% of forums run terrible software that doesn't perform, is not easily usable and won't work right on phones. That tells me it's not a solved problem at all.
codeulike 5y ago
Here's the stats on Stackoverflow https://stackexchange.com/performance https://stackexchange.com/performance 1.3 billion page views per month, 9 web servers, 4 sql servers Stackoverflow is notable because they went down the C#/MVC/SQL Server route from the start, which meant much better performance per server. Thats why they make an interesting counterexample to the usual way...
- sofixa 5y ago
  > Stackoverflow is notable because they went down the C#/MVC/SQL Server route from the start, which meant much better performance per server And also notable because everything is under an expensive license, so big performant servers is the cheaper option. Edit: everything = Windows servers for their .NET app ( apparently in the process of migrating to .NET Core) and SQL Server
  - tester34 5y ago
    what do you mean by saying "everything"? just SQL Server?
    - foepys 5y ago
      Technically Windows licences, too, but maybe they migrated to Linux with their switch to .NET (Core).
      - tester34 5y ago
        Indeed, afaik they have been migrating long time ago 2018: >This is the query pattern that caused StackOverflow.com to go offline yesterday: https://github.com/dotnet/efcore/issues/13524 https://github.com/dotnet/efcore/issues/13524 but on the other hand their tech stack site says: C# + ASP.NET MVC instead of ASP.NET Core, but I don't think it is proof of anything.
        foepys 5y ago
        They are definitely using ASP.NET Core, most likely version 3.1 LTS. EF Core (like ASP.NET Core) ran on .NET Framework until version 2.1. Everything after that requires .NET (Core).
tyingq 5y ago
People do tend to cherry pick, don't they? Most of Stack Overflow's workload is returning a blob of html for a given url, to a not-logged-in user. Where that html doesn't even have to be the most recently saved copy.
hnbad 5y ago
> No using any "ask {specific person}" either It's worth mentioning that the diagram is explicitly incomplete. The yellow endpoints are fixes but except for "END" the other endpoints are all either "unknown state" (i.e. "I have no idea what's broken") or problems that aren't addressed in further detail like "The issue could be with Kube Proxy" or even "Consult StackOverflow". I'm not sure what a complete diagram would even look like but I don't think there's any way to infer complexity by looking at them in comparison.

by corobo 5y ago

Cthulhu_ 5y ago

It's taking one source - look at how these people solved it! - and trying to apply it to others. SO is relatively simple; it's basically customized forum software which is a solved problem that has been around for decades. A junior dev can build an alternative, and it can be built using tried and true solutions like MySQL + PHP, which are horizontally scalable with database sharding, read replicas, and maybe stuff like memcached to accumulate votes before updating the database or a CDN for caching static files. Google has different problems and different workloads, and they have hundreds of times more applications with thousands of times more load. Apples and oranges.

JimDabell 5y ago
> it can be built using tried and true solutions like MySQL + PHP, which are horizontally scalable with database sharding, read replicas, and maybe stuff like memcached to accumulate votes before updating the database or a CDN for caching static files. > Google has different problems and different workloads Which of these do you think most organisations most closely resemble? I don’t think anybody would disagree if you said that you should use Kubernetes for organisations that resemble Google. But most organisations don’t look anything like Google. They look a lot more like Stack Overflow. So the “You don’t need this…” statement holds true for almost everyone.
- p_l 5y ago
  Stack Overflow is also a) old system built before a lot of current management tech was available openly b) Very single-application centered. EDIT: To expand on it - they had to manually build a lot of what could be much easier handled today, and thus had no incentive to change later. Interestingly enough, SO has a lot of moving parts distributed over multiple servers, even if all of those servers used to fit into 1-3 racks per DC.
  - andrewingram 5y ago
    The single vs multiple application centric thing is a point that's not made enough. Google is basically now a company all about spinning up new products, rather than just periodically adding a new piece of tech to an existing product. As a layperson, Kubernetes _appears_ to be about making it easier to spin up (and run) new products, so it feels more appropriate for Google-like org types.
    - p_l 5y ago
      There are also a lot many more applications one might need to run, especially when scaling beyond one-two people running all development on their laptops/desktops and not having a company yet - especially when one doesn't have valley-style funding. Also these days you might not want to implement login functionality from scratch, or have better logging, monitoring, etc. and that might involve running more applications than just your LAMP stack.
tester34 5y ago
>A junior dev can build an alternative Of course junior dev can do it, the same way junior dev can make Youtube it'll work as long as there's less than 100 concurrent users on SO and less than 50 4K 20min videos on youtube
- piva00 5y ago
  Sometimes I question myself if people posting these "a junior dev can build this product" haven't really worked with systems at scale and all of the myriad of issues that scale brings or if they are being the usual hyper-optimistic-dev that isn't considering much past the proof-of-concept point of a product. Either way is quite baffling how common this kind of comment is, almost a decade reading Hacker News and it pops up constantly.
  - corobo 5y ago
    It also doesn't matter if a junior dev could do it if they just don't do it If it's so trivial, go be rich!
    - piva00 5y ago
      I remember this post from gregdoesit [1] as a good example of how much complexity is hidden under seemingly trivial interfaces. It's about Uber's supposedly bloated app, I believe that anyone who has worked at a scale where they had to support different national legal systems in their code would know that maintaining a large-scale product is nothing trivial. [1] https://news.ycombinator.com/item?id=25376346 https://news.ycombinator.com/item?id=25376346
krageon 5y ago
> a solved problem Except 99% of forums run terrible software that doesn't perform, is not easily usable and won't work right on phones. That tells me it's not a solved problem at all.

codeulike 5y ago

Here's the stats on Stackoverflow https://stackexchange.com/performance https://stackexchange.com/performance 1.3 billion page views per month, 9 web servers, 4 sql servers Stackoverflow is notable because they went down the C#/MVC/SQL Server route from the start, which meant much better performance per server. Thats why they make an interesting counterexample to the usual way...

sofixa 5y ago
> Stackoverflow is notable because they went down the C#/MVC/SQL Server route from the start, which meant much better performance per server And also notable because everything is under an expensive license, so big performant servers is the cheaper option. Edit: everything = Windows servers for their .NET app ( apparently in the process of migrating to .NET Core) and SQL Server
- tester34 5y ago
  what do you mean by saying "everything"? just SQL Server?
  - foepys 5y ago
    Technically Windows licences, too, but maybe they migrated to Linux with their switch to .NET (Core).
    - tester34 5y ago
      Indeed, afaik they have been migrating long time ago 2018: >This is the query pattern that caused StackOverflow.com to go offline yesterday: https://github.com/dotnet/efcore/issues/13524 https://github.com/dotnet/efcore/issues/13524 but on the other hand their tech stack site says: C# + ASP.NET MVC instead of ASP.NET Core, but I don't think it is proof of anything.
      - foepys 5y ago
        They are definitely using ASP.NET Core, most likely version 3.1 LTS. EF Core (like ASP.NET Core) ran on .NET Framework until version 2.1. Everything after that requires .NET (Core).

tyingq 5y ago

People do tend to cherry pick, don't they? Most of Stack Overflow's workload is returning a blob of html for a given url, to a not-logged-in user. Where that html doesn't even have to be the most recently saved copy.

hnbad 5y ago

> No using any "ask {specific person}" either It's worth mentioning that the diagram is explicitly incomplete. The yellow endpoints are fixes but except for "END" the other endpoints are all either "unknown state" (i.e. "I have no idea what's broken") or problems that aren't addressed in further detail like "The issue could be with Kube Proxy" or even "Consult StackOverflow". I'm not sure what a complete diagram would even look like but I don't think there's any way to infer complexity by looking at them in comparison.