4 ms·

Honest question ... How far can you push "one server"? Given that HN runs on one server (plus one mirror) and this apparently serves a not that small community

by fho 3y ago

Honest question ... How far can you push "one server"? Given that HN runs on one server (plus one mirror) and this apparently serves a not that small community well ... Given "1 million connections" are possible on one server ... Given that green threads are a thing these days ... Given that SQLite is so much faster than networked databases ... Given that a monolith is faster than Microservices ... Given that it's possible to use a compiled language instead of trying to speed up interpreted languages ... What's the limit here? HN is a great example of what's possible, but can a single beefy server scale to handle the traffic of a Reddit?

robben1234 3y ago
Serving a social network is not the case where you'd want just one server. Mainly due to reliability concerns, spikes in traffic and different patterns at different times. But if you like the idea of ignoring best practices, there are EC2 instances with 192 vCPUs that go for several thousands an hour. And taking as example Golang benchmarks https://github.com/smallnest/go-web-framework-benchmark https://github.com/smallnest/go-web-framework-benchmark where it's not uncommon to be able to serve 6 digits of 30ms requests per second with just 4 cores. Then the conclusion is, the limit is the sky, if you are stubborn enough.
- fho 3y ago
  > But if you like the idea of ignoring best practice ... do you know me? ;-)
- fho 3y ago
  Another dumb question ... for "several thousand an hour" I could buy a pretty decent server myself ... last time I checked (on https://yourdatafitsinram.net/ https://yourdatafitsinram.net/) the largest physical server there was like 10 grand. Combine that with the ability to actually throttle a physical server (ie you only pay what you actually use, not per hour) ... I guess I would propose the following scaling strategy: 1. get cheap physical server 2. get your software up and running 3. get a second physical server that is roughly twice as capable 4. use that second server as a "beta" and development server ... do a lot of profiling there 5. once your real world usage spike on the first server routinely reach 80-90%, switch production over to the bigger server 6. get a new development server that is twice as beefy as the former development server
  - robben1234 3y ago
    I mixed up pricing. Several thousands a month. But if you could buy a 192 CPUs server for just 10 grand, seems like not a bad deal. Especially if this hypothetical project is able to attract the load we are discussing for longer than a few weeks.
- ihattendorf 3y ago
  Several thousands an hour is a bit of a stretch, maybe you meant monthly? but your point still stands. For those interested: right now the largest instance I see is a u-12tb1.112xlarge (448 vCPUs, 12 TB RAM, 100 gb networking) which is ~$109/hr on demand or $40-80/hr with a 1-3 year commitment. This works out to ~$29k, ~$58k, or ~$79k per month. If you don't need 12 TB of memory, there's a 6 TB instance for roughly half the cost.
  - fho 3y ago
    Honestly, I am completely clueless, but what would a beefy server with 240 physical cores cost? (8 CPUs with 30 cores each) edit: physical server that is
  - robben1234 3y ago
    Whoops. Mixed up pricing. 192 vCPUs instances on-demand depending on the type indeed fluctuate around $10/hr, leading to several thousands a month. Not to forget the data transfer costs though https://aws.amazon.com/ec2/pricing/on-demand/ https://aws.amazon.com/ec2/pricing/on-demand/ And also the potential minefield of API GW and NAT costs.
taskforcegemini 3y ago
this really depends on the software. a website using a "slow" interpreted programming language (php, python, ruby etc) in a "heavy" application that runs many tasks in the background on each pageload does not compare to a website that merely returns static information, either directly from the webserver or mostly from cache (no or much less need to waiting for cpu to process, or disks to fetch data vs ram). it is easier to just archive and provide access to existing content than to also run all the "invisible" services around that content.

by fho 3y ago

robben1234 3y ago

Serving a social network is not the case where you'd want just one server. Mainly due to reliability concerns, spikes in traffic and different patterns at different times. But if you like the idea of ignoring best practices, there are EC2 instances with 192 vCPUs that go for several thousands an hour. And taking as example Golang benchmarks https://github.com/smallnest/go-web-framework-benchmark https://github.com/smallnest/go-web-framework-benchmark where it's not uncommon to be able to serve 6 digits of 30ms requests per second with just 4 cores. Then the conclusion is, the limit is the sky, if you are stubborn enough.

fho 3y ago
> But if you like the idea of ignoring best practice ... do you know me? ;-)
fho 3y ago
Another dumb question ... for "several thousand an hour" I could buy a pretty decent server myself ... last time I checked (on https://yourdatafitsinram.net/ https://yourdatafitsinram.net/) the largest physical server there was like 10 grand. Combine that with the ability to actually throttle a physical server (ie you only pay what you actually use, not per hour) ... I guess I would propose the following scaling strategy: 1. get cheap physical server 2. get your software up and running 3. get a second physical server that is roughly twice as capable 4. use that second server as a "beta" and development server ... do a lot of profiling there 5. once your real world usage spike on the first server routinely reach 80-90%, switch production over to the bigger server 6. get a new development server that is twice as beefy as the former development server
- robben1234 3y ago
  I mixed up pricing. Several thousands a month. But if you could buy a 192 CPUs server for just 10 grand, seems like not a bad deal. Especially if this hypothetical project is able to attract the load we are discussing for longer than a few weeks.
ihattendorf 3y ago
Several thousands an hour is a bit of a stretch, maybe you meant monthly? but your point still stands. For those interested: right now the largest instance I see is a u-12tb1.112xlarge (448 vCPUs, 12 TB RAM, 100 gb networking) which is ~$109/hr on demand or $40-80/hr with a 1-3 year commitment. This works out to ~$29k, ~$58k, or ~$79k per month. If you don't need 12 TB of memory, there's a 6 TB instance for roughly half the cost.
- fho 3y ago
  Honestly, I am completely clueless, but what would a beefy server with 240 physical cores cost? (8 CPUs with 30 cores each) edit: physical server that is
- robben1234 3y ago
  Whoops. Mixed up pricing. 192 vCPUs instances on-demand depending on the type indeed fluctuate around $10/hr, leading to several thousands a month. Not to forget the data transfer costs though https://aws.amazon.com/ec2/pricing/on-demand/ https://aws.amazon.com/ec2/pricing/on-demand/ And also the potential minefield of API GW and NAT costs.

taskforcegemini 3y ago

this really depends on the software. a website using a "slow" interpreted programming language (php, python, ruby etc) in a "heavy" application that runs many tasks in the background on each pageload does not compare to a website that merely returns static information, either directly from the webserver or mostly from cache (no or much less need to waiting for cpu to process, or disks to fetch data vs ram). it is easier to just archive and provide access to existing content than to also run all the "invisible" services around that content.