2 ms·

Still, for comparison a huge site like reddit is serving around 1000 pageviews per second. The hardware in this post isn't that good, nowadays you can do quite

by jules 15y ago

Still, for comparison a huge site like reddit is serving around 1000 pageviews per second. The hardware in this post isn't that good, nowadays you can do quite a bit better. If you could get even 0.02% of these pageviews per second out of more modern hardware that's already 100 pageviews per second. Only a handful of sites need more than that. For example HN certainly does not. And AFAIK HN runs single core on a language (Arc) that compiles to Scheme, not the fastest in town. Since reddit is only about 70x larger than HN, we are probably close to the point where with a lot of optimization a site like reddit could be run on a single large box (if we're not there already).

ma2rten 15y ago
Dude... I am sorry, but your reply does not make sense on many levels: I don't know if reddit could run on a single server, but this benchmark does nothing to prove it. A pageview on reddit consist of several http requests. A http requests translates into several requests into a database backend. In this syntactic benchmark he is requesting the same static file over and over again, so nginx and/or linux can cache it in memory (or CPU cache). Also even if they could put reddit on a single server, it would require a lot of optimization and engineering time is more expensive than hardware. It would also make adding new futures in the future a pain in ass, because you would have to be very greedy with CPU time (and other resources like memory, bandwidth, etc...). Also you want a multi-server setup for redundancy and handling extra capacity. EDIT: I don't if you also planning to put the database on the same server? Web servers are basically stateless so they are easy to scale. It would be interesting to calculate how much disks that server would need just to handle the IO throughput. Full disclosure: This is some speaking who used to work for social network with 9M users and 3,000 servers.
- jules 15y ago
  This benchmark does not, but the fact that Hacker News which is just 70x smaller but conceptually similar runs on a single core on a slow language does. I completely agree that it's probably not be a good idea, but the fact that it's possible or almost possible is still interesting in my opinion. If nothing else the power of modern hardware puts the obsession with horizontal scalability into perspective, since most sites are nowhere near the size of reddit. Even though a pageview on reddit translates to several HTTP requests, only one of those requests has to serve dynamic HTML. The requests that get the JS, CSS and images can go to a content delivery network (and they probably already do). You'd probably indeed want to put the database on the same server, with the vast majority of DB accesses hitting main memory (as you can easily get a server with hundreds of gigabytes or even terabytes of RAM nowadays -- it helps that reddit's access pattern is very heavily concentrated on new posts). Of course you have to replicate to multiple servers to prevent data loss. 9M users and 3,000 servers sounds like a lot. What kind of servers are these? What is the size of the database, roughly?

by jules 15y ago

ma2rten 15y ago

Dude... I am sorry, but your reply does not make sense on many levels: I don't know if reddit could run on a single server, but this benchmark does nothing to prove it. A pageview on reddit consist of several http requests. A http requests translates into several requests into a database backend. In this syntactic benchmark he is requesting the same static file over and over again, so nginx and/or linux can cache it in memory (or CPU cache). Also even if they could put reddit on a single server, it would require a lot of optimization and engineering time is more expensive than hardware. It would also make adding new futures in the future a pain in ass, because you would have to be very greedy with CPU time (and other resources like memory, bandwidth, etc...). Also you want a multi-server setup for redundancy and handling extra capacity. EDIT: I don't if you also planning to put the database on the same server? Web servers are basically stateless so they are easy to scale. It would be interesting to calculate how much disks that server would need just to handle the IO throughput. Full disclosure: This is some speaking who used to work for social network with 9M users and 3,000 servers.

jules 15y ago
This benchmark does not, but the fact that Hacker News which is just 70x smaller but conceptually similar runs on a single core on a slow language does. I completely agree that it's probably not be a good idea, but the fact that it's possible or almost possible is still interesting in my opinion. If nothing else the power of modern hardware puts the obsession with horizontal scalability into perspective, since most sites are nowhere near the size of reddit. Even though a pageview on reddit translates to several HTTP requests, only one of those requests has to serve dynamic HTML. The requests that get the JS, CSS and images can go to a content delivery network (and they probably already do). You'd probably indeed want to put the database on the same server, with the vast majority of DB accesses hitting main memory (as you can easily get a server with hundreds of gigabytes or even terabytes of RAM nowadays -- it helps that reddit's access pattern is very heavily concentrated on new posts). Of course you have to replicate to multiple servers to prevent data loss. 9M users and 3,000 servers sounds like a lot. What kind of servers are these? What is the size of the database, roughly?