3 ms·
I've been hugged twice on 100 mbit domestic broadband, and it's honestly not been all that bad. In general I've seen somewhere between 1-5 visitors per second,
by marginalia_nu 5y ago
I've been hugged twice on 100 mbit domestic broadband, and it's honestly not been all that bad.
In general I've seen somewhere between 1-5 visitors per second, a couple of million over a day. I think keeping the number of requests per visit down is a big key to surviving this traffic with low resources.
The first hug was search.marginalia.nu, which used to load fonts and had a load of about 50 Kb/visit; but still only generated in maybe 4-5 GETs per visit. I measured about 2 Mbps.
The second time was search.marginalia.nu/browse/random, which loads 75 images at a resolution of 512x348 (mostly WebP; so anywhere between 2 and 30 Kb). These were randomly picked from a very large pool, so they can't be meaningfully cached. I presciently set them to lazy loading and I think that made a big difference, especially from mobile visitors as they'd only load a few at a time. I didn't measure the bandwidth usage at this point, but it stayed up and stayed responsive throughout the day.
I think something that possibly also makes a difference is that I don't do any sort of session cookies or mutable non-ephemeral state in my backend. I can only imagine having to validate sessions and manage access/roles for every request would make for a lot more work.
I don't think it's necessarily the bandwidth that's going to be the bottleneck, rather than something like contested locks in the database, resulting in back pressure in turn causing traffic jam of lingering connections. Reducing mutable server state helps a lot with that.
- sgtnoodle 5y agoI made a very pretty retro control panel for Kerbal Space Program a few years back, and made a nice writeup on my personal website about it, hosted from a low power Intel baytrail based server I keep in my garage. It got picked up by hackaday, then popular mechanics and a few other similar sites, and hacker news. I vaguely recall my upload rate was about 5 Mbps at the time, and my connection was quite saturated for a few days. The page has half a dozen jpg photos, so I suspect that was the bulk of the data.
- _xnmw 5y agoWait, what, one of the best text search engines for the whole internet runs on a residential broadband connection on a home server? My mind is blown.
- nicbou 5y agoSometimes I wonder what kind of scaling startups are scared of. I keep reading about popular websites running on a single server, and how cheap bare metal is compared to an equivalent AWS setup. Am I missing something?
- speed_spread 5y agoServing static HTTP GETs is easy. The moment you start trying to make money off the generated traffic its no longer a read only affair. Your system is not stateless anymore and tracking costs spiral up quickly.
- marginalia_nu 5y agoI largely agree with the analysis, with the caveat that a search engine is a bit more involved than serving static GETs.
- speed_spread 5y agoSorry, I lost context and was not thinking of the search engine but of a statically generated blog with small pages and little media. Also, thanks for confirming my hunch about the bigger factor being statefulness rather than dynamicity. CPU, RAM storage are incredibly cheap nowadays. But so much can be going on at a time that _certainty_ is now what's pricey.
- marginalia_nu 5y agoI don't think most startups really understand how their systems scale, so assume they'll scale like a social media platform (which do not scale well at all). I attribute a lot of my performance to initially targeting very low powered hardware. I ran it on a raspberry pi cluster until the index was around 1 million documents (I think). Having those limitations early on excluded a lot of design mistakes that would have been a lot harder to correct later. Serving search requests is a embarrassingly parallel operation that could be very easily load balanced to different instances. If so Max Schrems hectored every other search engine out of Europe, and Ursula von der Leyen, taking pity on my living room search operation, sent me a bunch of money, I'd probably be able to deal with 1000x the traffic about as fast as I'd be able to lease space in a datacenter. I am a bit choked in terms of how big I can grow my index, though. I could probably double, maybe even a couple of times, but not too many times if I can't find serious optimizations.
- mysterydip 5y agoI think your math might be off, or I misunderstood: 2mil/day = ~23/s?
- marginalia_nu 5y agoYou're correct. I was pulling the numbers from memory. I think it may have been 2 mil requests/day, which would mean 23 rps for 1-5 vps (roughly lining up with the math of 5 requests/visit).