59 ms·

My £4 a month server can handle 4.2M requests a day

• yupper32 5y ago
  Well the front page of HN won't get you 4.2M views today, but it's a pretty good real world test!
  • mark_mcnally_je 5y ago
    It is! It's crashed my analytics but the website seems to be doing fine.
  • jonplackett 5y ago
    True, but it may very well get you a lot more than 50 a minute.
• 5faulker 5y ago
  That's a good spec for 4 bucks. With cloud hosting you might be able to push the cost down a bit with less CPU resource and memory.
  • mark_mcnally_je 5y ago
    I am cloud hosting using OVH cloud, I brought the server a couple of years ago now so they probably have some better specs for the price I am paying.
• ppeetteerr 5y ago
  How did this make it to the number two spot on HackerNews?
  • deleted 5y ago
    [deleted]
  • deleted 5y ago
    [deleted]
  • ok_coo 5y ago
    IMHO, it's better than a deluge of political posts. I don't mind reading about politics but I come to HN to read about tech. We can go elsewhere to get whatever politics we desire.
  • dreyfan 5y ago
    Broadly speaking people on HN have no clue how to setup a performant httpd/app server and are impressed by abysmal performance/cost metrics like this or the MangaDex post. Everything these days is obscured through multiple layers of SaaS offerings and unnecessary bloat like kubernetes. ~10k rps (it was concurrent connections but close enough) was state of the art in 1999. Now 22 years later ~50 rps is somehow impressive.
    • watermelon0 5y ago
      I would guess this is close to a default Apache + mod_wsgi setup, and this is one of the easiest way of hosting Python web apps, so basically achievable by anyone on HN. I assume (based on 180 req/s for static page) that he is using mpm_prefork, where each Apache child handles a single connection. If he switched to mpm_event, which uses event loop like nginx, ~10k rps should easily be achievable, but I don't think WSGI would work with that.
      • mark_mcnally_je 5y ago
        Yeah it's all default apache + mod_wsgi. This is also my first Django setup and I made it over-complicated as a learning exercise. mpm_event is something I have not heard of before, thanks for bringing that to my attention.
    • reilly3000 5y ago
      In my opinion it isn’t so much of a lost art or lamentable accretion of useless abstractions, but an increase in the scope of what web apps do these days. Most of us aren’t working on static sites or simple CMS publishing- those are a solved problem. Instead we’re building mobile banks, diagnostic systems, software tooling, 3D games, and shopping malls. The complexity is inherent in the maturity of the web and it’s many uses, as well as its global scale. Hugs of death are rare these days thanks to better architecture and infrastructure, even though the scale of users has grown 100X. Yes there are wildly unnecessary abstractions that are used for small sites/apps, but I would contend they are artifacts of someone who it’s trying to learn something new, and/or get promoted. I have no problem with the former.
    • 5y ago
  • wpietri 5y ago
    An awful lot of professional programmers work in such heavyweight contexts that they don't have a good idea of how fast modern hardware can be. I was talking with an architect at a bank whose team was having trouble getting under a 2-second maximum for page views. They blamed it on having to make TCP requests to other services, and said something like "at a couple hundred milliseconds per request, it adds up quickly!" My head nearly exploded at that. I spun up some quick tests in AWS to show exactly how many requests one could make in 2000 ms. I don't have the numbers handy, but the number is very large. This junky slice of a server handling full page requests in 20 ms is a fine example to counter thinking that's endemic in enterprise spaces.
    • dragontamer 5y ago
      I had a discussion with a coworker about something about slow memcpys, that roughly went the same way except... you know, DDR4 RAM has a speed of roughly 40GBps. Also, that "awful" 1MB memcpy is likely all in L3 cache these days. But even if it weren't in cache, we're talking about an operation that takes 50 microseconds (1MB read + 1MB written == 25microseconds + 25 microseconds). Given that modern CPUs have like 16+ MBs of L3 cache (and more), and some mainstream desktop CPUs have 1MB of L2 cache... its very possible that this memcpy is far faster in practice than you can imagine. 1MB is big, its a million bytes. But CPUs are on the scale of billions, so 1MB is actually rather small by modern standards. Its surprisingly difficult to get intuition correct these days...
      • wpietri 5y ago
        For sure. I think keeping one's intuition for systems updated is one of the biggest quests for anybody trying to have a career in tech.
    • ppeetteerr 5y ago
      This is a person running a blog and pulling HTML data from a database. Is it that impressive that they are getting 50r/s?
      • 5y ago
• ComputerGuru 5y ago
  Absolutely. Even a terrible Wordpress instance can be beautifully (and transparently!) cached behind either nginx or varnish with ease, in which case you’re just serving static html pages and can probably handle any traffic you are likely to ever get.
• johnklos 5y ago
  This is a good, simple way to show how much can be done with modest resources. Sometimes we see people fetishizing bigger and faster, then gatekeeping when people want to do the same work with modest means, whether it a four quid a month hosting service or a first generation Raspberry Pi. Not everyone has the money or desire for bigger & faster, and it's nice to see that here.
  • mark_mcnally_je 5y ago
    That's why I made this post, was very happy to see how much it could theoretically handle.
  • danjac 5y ago
    I'm not sure it's so much about fetishizing, and more about realistic expectations around software development in larger teams. If you are the sole developer working on your own site - be it a side project/hobby/labour of love or your source of income - you have complete control up and down the stack and have the leeway to tweak performance wherever needed - whether that's indexing and optimizing queries in the backend, reducing the size of your static assets, caching, whatever. You can even yank whole features if you feel their inherent complexity and load outweighs their usefulness. In anything including and above a medium sized company, a single developer will rarely have the leeway to do anything beyond tinker with their small slice of the stack. They might spend some hours carefully optimizing a query, but it's for naught because the frontend team have screwed up the webpack settings and the JS load runs into many MB. Or you have both done your jobs but the PM wants a ton of analytics on every page. And the CEO's pet feature is a maintenance and performance nightmare but nobody has the clout to have it removed or even simplified. Nobody wants to waste sprints on paying down tech debt in a feature factory, so it becomes progressively harder to fix performance issues. At that point, the cheaper and politically easier option is to just fire the money cannon at expensive cloud services and hope the extra spend squeezes out some performance gains.
• jonplackett 5y ago
  #1 on HN and still up. That speaks for itself.
  • mark_mcnally_je 5y ago
    :)
    • jonplackett 5y ago
      Are your starting to feel the pressure yet?
      • mark_mcnally_je 5y ago
        My blog is fine, my self hosted analytics (on another server) not so much!
        • jonplackett 5y ago
          Do you know what the peak requests per second was?
          • mark_mcnally_je 5y ago
            Unfortunately not, I guess I could measure that using log analytics. Perhaps I'll do another right up on the impact of getting to the #1 of HN
• jaymzcampbell 5y ago
  I'm not sure there was ever an argument saying otherwise. The ease of processing X million requests is heavily dependant on what those requests actually do. Trivial use cases shouldn't be a surprise to have high throughput.
• fbrchps 5y ago
  I understand your excitement for being able to handle a decent amount of requests on such a small server, but just like many other websites that get on the frontpage of HN, your site is taking multiple seconds to load for me, depending on when I refresh. As you said in your post, adding caching to your site increased your throughput by ~20% (or +10/req/sec). What you and other sites seem to lack is a more distributed caching, a la CloudFlare, S3 CloudFront, Azure CDN, etc. Those last two only really work well for a static site, however as mentioned in your post that's essentially what you're serving. While I'm all for having a free-as-in-freedom hosting solution and keeping things lean, the internet is a fickle beast, and nothing looks worse for a company who posts on HN when their technology-oriented site can't handle a few thousand requests per minute. (Or in this case, when a blog claims to handle 4.2M requests a day -- 2.9k req/min)
  • crazy_horse 5y ago
    200 rps is not great.
    • fridif 5y ago
      Wikipedia as a whole has 8k rps, and that's with multiple racks in multiple data centers. I haven't read recently, but they were only doing 200 rps per server.
  • floren 5y ago
    Looks fine over here, and he doesn't have to route through a fucking Internet gatekeeper like Cloudflare or Amazon... let's enjoy this golden era before Chrome starts flagging any site which isn't fronted by a "reputable" cache like Cloudflare, Amazon, or whatever Google decides to introduce.
    • fbrchps 5y ago
      It would also be possible for OP to spin up their own Redis cache, and have multiple POPs near their target audience, and handle DoS type attacks against their site if need be, and easily be able to brush aside bot traffic, and... Not all the above apply to a hobby-blog style site, but I wasn't referring only to OP's site in my original comment. I understand that not everyone needs to feed into "fucking Internet gatekeeper"s as you described, but the fact that they provide valuable services is undeniable. They make a complex operation -- one that could mean the difference between a company being able to sell their product or not -- simple.
• nostrademons 5y ago
  People tend to severely underestimate how fast modern machines are and overestimate how much you need to spend on hardware. Back in my last startup, I was doing a crypto market intelligence website that subscribed to full trade & order book feeds from the top 10 exchanges. It handled about 3K incoming messages/second (~260M per day), including all of the message parsing, order book update, processing, streaming to websocket connections on any connected client, and archival to PostGres for historical processing. Total hardware required was 1 m4.large + 1 r5.large AWS instances, for a bit under $200/month, and the boxes would regularly run at about 50% CPU.
  • fizwhiz 5y ago
    Would you mind describing your stack in more detail? Did you use gRPC with Go?
    • nostrademons 5y ago
      Sure, startup is defunct now and I think arbitrage & data on centralized exchanges is a dead market now. Wall Street HFTs got into the arbitrage game, and the data sites laypeople actually visit are the ones started in 2014. Codebase was pure server-side Kotlin running on the JVM. Jackson for JSON parsing, when the exchange didn't provide their own client library (I used the native client libraries when they did). Think I used Undertow for exchange websockets, and Jetty for webserving & client websockets. Postgres for DB. The threading model was actually the biggest bottleneck, and took a few tries to get right. I did JSON parsing and conversion to a common representation on the incoming IO thread. Then everything would get dumped into a big producer/consumer queue, and picked up by a per-CPU threadpool. Main thread handled price normalization (many crypto assets don't trade in USD, so you have to convert through BTC/ETH/USDT to get dollar prices), order book update, volume computations, opportunity detection, and other business logic. It also compared timestamps on incoming messages, and each new second, it'd aggregate the messages for that second (I only cared about historical data on a 1s basis) and hand them off to a separate DB thread. DB would do a big bulk insert every second; this is how I kept database writes below Postgres's QPS limit. Client websocket connections were handled internally within Jetty, which I think uses a threadpool and NIO. Key architectural principles were 1) do everything in RAM - the RDS machine was the only one that touched disk, and writes to it were strictly throttled 2) throw away data as soon as you're done with it - I had a bunch of OOM issues by trying to put unparsed messages in the main producer/consumer queue rather than parsing and discarding them 3) aggregate & compute early - keep final requirements in mind and don't save raw data you don't need 4) separate blocking and non-blocking activities on different threads, preferring non-blocking whenever possible and 5) limit threads to only those activities that are actively doing work.
• deleted 5y ago
  [deleted]
• bob229 5y ago
  Pepe ftw
• jka 5y ago
  There are a few comments in here that predictably suggest that simple static sites can handle large request rates easily. Sure, that's true - but to try to progress the conversation: how would you measure the complexity of serving web requests, in order to perform more advanced cost comparisons? (bandwidth wouldn't be quite right.. or at least not sufficient - maybe something like I/O, memory and compute resource used?)
• fabian2k 5y ago
  50 rps not that much, though of course easily sufficient for many situations. This is also Django which certainly isn't the fastest choice. I played around with it a long time ago and liked it quite a bit, but you don't choose Django for performance but for the other benefits. I'm really more surprised that static serving is so slow at 180 rps. This should be able to easily saturate the network, statically serving files is very, very fast. From what I see in the blog I doubt that the files are very large, so there is probably some other bottleneck or I'm missing something here.
• janmo 5y ago
  There is a difference between being able to handle 4.2M requests a day, and handling 4.2M requests per day. Visitors don't come neatly one after the other. You might only have 1M requests a day but get random spikes with 100 requests at the same time.
  • bArray 5y ago
    Very true. It also makes a difference as to which resource is being pulled, whether it is cached, what transport is being requested (SSL, compression, etc). I really suspect the website would fall long before it hits anything close to 4.2 million requests (which the author also seems to except). That all said - long live tiny web servers!
• great-potential 5y ago
  Dont mean to be the negative Joe but you dont need a webserver if you're serving a static-able website.
  • nayuki 5y ago
    You mean use someone else's web server instead?
    • great-potential 5y ago
      I mean this is just basic stuff, for me it just sounds like a developer putting a site in production, no auto-scaling, patching ... might as well outsource this to a CDN since there is no database/redis/varnish ...
      • sleepyhead 5y ago
        So...someone else's web server instead?
        • great-potential 5y ago
          No offense but I think you've missed the point.
          • icehawk 5y ago
            A CDN is just a web server somewhere else.
  • deleted 5y ago
    [deleted]
  • bruce343434 5y ago
    How are you supposed to _serve_ that static _web_site then?
• pluc 5y ago
  > Not taking into account any issues that may occur around CPU/RAM/Disk IO due to sustained levels of traffic as well as bandwidth issues congrats?
• markandrewj 5y ago
  Normally benchmarks for things like this are measured in how many concurrent requests can be handled, i.e the C10K problem, not by how many requests you are able to serve in a day. It's also well known that you can serve a large amount of requests on limited hardware. https://en.wikipedia.org/wiki/C10k_problem https://en.wikipedia.org/wiki/C10k_problem "By the early 2010s millions of connections on a single commodity 1U rackmount server became possible: over 2 million connections (WhatsApp, 24 cores, using Erlang on FreeBSD),[6][7] 10–12 million connections (MigratoryData, 12 cores, using Java on Linux).[5][8]" Although I do understand the boxes listed above have more resources then the VPS you are using. I am also not criticizing your write up, or results, bench-marking is in general interesting to do. I just wanted to provide some additional information.
  • ijidak 5y ago
    How does a single server run millions of active connections? Wouldn't you run out of TCP sockets? What am I missing?
    • dreyfan 5y ago
      A connection isn't just a dest_port, it's the unique combination of 4 components: source_ip:source_port:dest_ip:dest_port
      • deleted 5y ago
        [deleted]
    • arthurcolle 5y ago
      socket multiplexing
    • jfrunyon 5y ago
      I would guess they may be muxed over fewer sockets, by their LBs, but that's not strictly necessary. I'm not sure exactly what you mean by "run out of TCP sockets", but theoretically speaking, the only limitation is how much memory is available to store the necessary info about the socket (like address/protocol info and process info). In practice, OS's do have a "max socket" or "max FD" limit, but that's usually configurable and (with enough RAM) could easily be set to "millions".
• polote 5y ago
  What's the point of this post? OP is serving a file at 50req/sec. There is not even mention of a dB query. How is that able to relate to any kind of normal app? I guess that the post was written as an answer to the mangadex post [1]. Mangadex was handling 3k req/sec involving dB queries. It was not just a cached Html page. 50req/sec for a Html file is super low which shows that a $4 month server cant do much actually. So yes this is enough for a blog, but a lot of websites are not blogs [1] https://news.ycombinator.com/item?id=28440742 https://news.ycombinator.com/item?id=28440742
  • throwdecro 5y ago
    > How is that able to relate to any kind of normal app? There's too much competition involved in writing normal apps, which often attract significant investment that bootstrapped startups struggle to compete with. It's interesting to see what kind of performance is possible for next to no money, when you throw out basic assumptions like using a database, and then start thinking about what you could build out of it.
    • quickthrower2 5y ago
      My recent submission of HNBadges was made like this. It's just 3 files (html, css, JS) which I hosted for free on Netlify, but could have been hosted on a setup like OP. I used other services for XHR requests. I imagine it got a tonne of traffic from being on the first page, but I wasn't taking metrics. Another example of clever use of resources is the https://haveibeenpwned.com/ https://haveibeenpwned.com/ website. Using a bloom filter (I think) to turn what could have been a back-end lookup into a "front end lookup" by requesting a small file from the server based on the password hash. The only issue I have with the OP is his assumption that you'd get a nice smooth 60 request/second throughout the day! Most likely will be lumpy, and in the top of the lumpy periods (where most of your visitors visit) performance will be bad.
  • great-potential 5y ago
    Exactly, I dont see the point bragging about this nevertheless posting about it on HN ...
• Rd6n6 5y ago
  Re: benchmarking, sometimes the bottleneck is the machine or server that issues the requests, not the receiver that you are testing. To figure out your actual capacity, you sometimes need multiple request servers or a more powerful request server. This was the case for a project I did a few years ago. Not a critique of the blog post, just remembering something out loud His site, https://peepopoll.com/ https://peepopoll.com/, took about 10s to load for me. It’s also good to chart other metrics like response times while you benchmark. Requests per second isn’t the same as a low response time
  • hu3 5y ago
    Indeed. Recently a client needed to bench raw req/s processing power of their application server and I had to ask for a powerful server running on the same DC in order to discard any potential routing issues.
• deleted 5y ago
  [deleted]
• nicoburns 5y ago
  So 50req/sec. I'd hope it could handle a lot more than that!
• olingern 5y ago
  I might get downvoted for not getting on the 2000s style of development bandwagon, but do you really need a web server to serve static text?
  • bellyfullofbac 5y ago
    What do you suggest, Gopher?
    • crazy_horse 5y ago
      Nginx.
      • Zababa 5y ago
        Nginx is a web server though. You're assuming that by "web server" the first comment meant "web framework"?
    • CyanLite2 5y ago
      Cloudflare
      • mr_toad 5y ago
        Last I checked cloudflare was running nginx.
    • SilverRed 5y ago
      Github/Gitlab pages is my choice for being free, managing https and just generally being pretty good. You can even use the CI to compile the site for you.
      • bellyfullofbac 5y ago
        Ah, kids these days. So I guess by "webserver" grandparent meant "A server you actually control", and by "not webserver" people mean "It's on the cloud somewhere". GH/GL-pages still respond to HTTP/HTTPS requests, in my dictionary that's a webserver, but I guess for the millennials (I say with snark) it means "I don't have to think about what happens, it just serves my content for me."
• sigg3 5y ago
  Put Wordpress on it, and do a new battery of TTFB tests ;)
  • celsoazevedo 5y ago
    Even WordPress would work fine if we use a plugin like WP Super Cache (no idea why they don't cache things by default). It wouldn't beat a simple static page, but WordPress + Cache plugin + cheap VPS can easily handle #1 on HN.
• reilly3000 5y ago
  That estimate can be verified with load testing systems like Artillery. My theory is that things would break far sooner than estimated along the following lines: - Too many WSGI connections if the timeouts aren’t tweaked - Too many database connections, especially without caching and tuning - on the Apache side if MaxRequestWorkers isn’t set there will be memory issues with 1GB RAM - the disk could easily hit IOPS limits, especially if there is a noisy neighbor It’s not likely all or any of these things will hit IRL, but that all depends on traffic and usage patterns. It matters not, if you were getting 4.2 M requests each day you’d be in the Alexa Top 1000 and could probably shell out for the $8 server :)
• MangoCoffee 5y ago
  this story remind me, the dot com bubble. dotcom companies bought servers from Sun Microsystem. they needs to handle the large traffics that "PC" server can't handle. anyone remember Cobalt server? https://en.wikipedia.org/wiki/Cobalt_Networks#/media/File:Cobalt_RaQ2_Debian.jpg https://en.wikipedia.org/wiki/Cobalt_Networks#/media/File:Co...
  • sgt 5y ago
    I remember many stories and discussions on Slashdot about it.
• taf2 5y ago
  I wonder what the 99th - 95th percentile response times look like…
• Wronnay 5y ago
  If you host it on GitHub Pages, GitLab Pages or Vercel it's even £4 less :o
• louwrentius 5y ago
  I'm hosting my static blog site on a physical Raspberry Pi 3B+ powered by Solar [1]. That blog post got hugged by HN but it didn't even raise the CPU above 10% on a single core. And a Raspberry Pi 3B+ is dog slow. And severely limited by bandwidth, unlike the Raspberry Pi 4B+. (But it uses less power so that's why I use a 3B+). However I have another point to make. Professional rack-mount servers from HP and Dell can be had second hand for dirt cheap and you get a ton of CPU (20+ cores) and an ocean of RAM for next to nothing. For many applications, an old Gen8 or similar Dell server will perform more than adequately. Even more so if you have a little bit more to spend on Gen9. They are so cheap that you can like buy four to eight, sprinkle them across two different datacenters and even if one breaks, you won't be in any hurry. [1]: https://louwrentius.com/this-blog-is-now-running-on-solar-power.html https://louwrentius.com/this-blog-is-now-running-on-solar-po...
• idworks1 5y ago
  One of my proudest moment in my career is when I lowered our app processing time from ~8hrs to 17 minutes. When I deployed my first update, it reduced it to 2 hours. The sysadmin immediately contacted me that there was something unusual. I confirmed the results but he was skeptical. Then with my second update, he told me that the app must be broken or that the script must be dying. There is no way it could complete this fast. What was the issue? We processed terabytes of data. Each and every single line processed created a new connection to the database and left it hanging. A try catch was added when the connections failed and restarted the process. Removing the connection from the for loop and properly handling it reduced the time drastically. And... why would you loop through millions of records when you can use batches? Also this was a phperlbashton* script. I turned it into a single PHP script and called it a day. As a consequence, backup time was reduced to 2 hours as opposed to 12 hours (no one was allowed on the website until the back up was done). Modern machines are incredibly fast. * PHP/Perl/Bash/Python
  • great-potential 5y ago
    omg what is this, we need some Brendan Gregg posts to elevate the level here, this is HN not reddit :D
  • devortel 5y ago
    > no one was allowed on the website until the back up was done I'm assuming this was an internal website and backups were scheduled for evenings/weekends?
  • vmception 5y ago
    and then you were fired because your job was dependent on this taking forever
  • nicbou 5y ago
    I've done something like that at my last university internship. I wrote about it here: https://nicolasbouliane.com/projects/pratt-whitney-redesign https://nicolasbouliane.com/projects/pratt-whitney-redesign It's the same story as yours, but with human effort. I was about to cut the human out entirely.
  • 5y ago
• throwaway20371 5y ago
  You can also handle 50 requests per second on a 66MHz 486DX2 with 16MB of RAM and a 10Mbit/s network card. Not with modern "I have infinite resources" software, but we used to handle more than that traffic regularly in the early 90s.
• zxcvbn4038 5y ago
  Combined with a CDN and you can do a lot more, I like to think in terms of origin requests rather than raw req/s. The problem I have is getting people to really understand how caching works at the cdn and browser layers and design their frontends and backend responses around that. There is also lot you can do with edge compute to clean the incoming requests before the CDN evaluates them to increase cache hits. Even if you are trying to give a "real time" view of data, caching it for even a second and allowing stale data to be served while it is updating can reduce origin requests significantly. I've seen people hammer sites hundreds of times a second looking for changes that only happen once every fifteen seconds or once per minute - the best thing you can do for yourself is handle all those requests at the CDN level (eventually you'll do log analysis and see the activity and can take other measures, but in the mean time don't let all of those requests go to the origin). Your CDN is probably giving you better rates for network egress then Amazon or Google anyway - the later are more focused on incentivizing you to use their ecosystem exclusively by penalizing you for sending data "outside". Cheap VPS hosts discourage you exceeding your bandwidth allocation because they are overselling their capacity and heavy usage upsets that - so again you want to shift as much as you can to your CDN.
• modeless 5y ago
  You don't need £4. As long as you can structure your site as a set of static files with interactivity done client side, even if some of the files change every minute or two, you can serve everything for $0 with Cloudflare in front of any free host. I've served 1M pageviews a day for $0 with Cloudflare + App Engine free tier and there's no reason it wouldn't scale to 100M or beyond.
  • jnieminen 5y ago
    You could also use Github pages or Cloudflare pages to host.
    • modeless 5y ago
      The nice thing about using Cloudflare in front of a real host is that you can still do dynamic pages. You can instantly purge any file from the Cloudflare cache, so all you have to do is purge anything that changes and your users see the update instantly, while your backend only sees a couple of extra requests. My site updates some data every 10 minutes or so, I don't think that would work with GitHub Pages. Maybe you could do something with Cloudflare Pages combined with Workers, but Workers have a limited free tier. The normal Cloudflare CDN scales to infinity for free.
  • goodpoint 5y ago
    No thanks. Cloudflare is very harmful to Tor and to user's freedom and privacy in general.
• tristor 5y ago
  The comments in this thread surprise me quite a lot. I suppose it shouldn't, but it does. This post + the response calls to the surface how badly basic system operations knowledge is needed in the industry and how much of it is missing from the toolkit of most developers.
  • planet-and-halo 5y ago
    Any recommended reading? I've bought a few things in that vein but I'm self-taught and always looking to improve on the ops/perf side.
    • tristor 5y ago
      I don't have any one complete book that I can recommend, and I don't even really have a great reading list for this. But I'll make an attempt to share what I think is useful as a starting point. 1. Systems Operations is first and foremost about understanding systems, in all of their complexity, which means understanding the internals of your OS primarily. 2. Performance and networking, in particular, are super important areas to focus on understanding when it comes to learning the topic to help with software development. 3. A lot of it is about understanding concepts in abstract and being able to extrapolate to other situations and apply these concepts, so there's actually quite a lot of useful information that can be learned on one OS and still applied to another OS (or on one game engine and applied to another, et al). Here's a few books I think are worth reading, not in any particular order of prevalence, but loosely categorized Databases: High Performance MySQL: https://www.amazon.com/gp/product/1449314287/ https://www.amazon.com/gp/product/1449314287/ SQL Queries for Mere Mortals: https://www.amazon.com/gp/product/0321992474/ https://www.amazon.com/gp/product/0321992474/ The Art of SQL: https://www.amazon.com/gp/product/0596008945/ https://www.amazon.com/gp/product/0596008945/ Networking: TCP/IP Illustrated: https://www.amazon.com/exec/obidos/ISBN=0201633469/wrichardstevensA/ https://www.amazon.com/exec/obidos/ISBN=0201633469/wrichards... (updates on author's site at http://www.kohala.com/start/tcpipiv1.html http://www.kohala.com/start/tcpipiv1.html) The TCP/IP Guide: https://www.amazon.com/TCP-Guide-Comprehensive-Illustrated-Protocols/dp/159327047X/ https://www.amazon.com/TCP-Guide-Comprehensive-Illustrated-P... UNIX Network Programming: https://www.amazon.com/dp/0131411551 https://www.amazon.com/dp/0131411551 Beej's Guide to Network Programming: http://beej.us/guide/bgnet/ http://beej.us/guide/bgnet/ Operating Systems: Operating Systems Concepts: https://www.amazon.com/Operating-System-Concepts-Abraham-Silberschatz/dp/1118063333/ https://www.amazon.com/Operating-System-Concepts-Abraham-Sil... (various editions, I have the 7th edition... I recommend you find the latest) Modern Operating Systems: https://www.amazon.com/Modern-Operating-Systems-Andrew-Tanenbaum/dp/013359162X/ https://www.amazon.com/Modern-Operating-Systems-Andrew-Tanen... (the "Tanenbaum Book") Operating Systems Design and Implementation: https://www.amazon.com/Operating-Systems-Design-Implementat-Implementation/dp/9332550514/ https://www.amazon.com/Operating-Systems-Design-Implementat-... (the other one, the "MINIX Book") Windows Internals: Part 1: https://www.amazon.com/Windows-Internals-Part-architecture-management/dp/0735684189/ https://www.amazon.com/Windows-Internals-Part-architecture-m... Part 2: https://www.amazon.com/Windows-Internals-Part-2-7th/dp/0135462401/ https://www.amazon.com/Windows-Internals-Part-2-7th/dp/01354... (I had the pleasure of being taught from this book by Mark Russinovich and David Solomon at a previous employer, was an amazing class and these books are incredible resources even applied outside of Windows, we used 5th edition, I linked 7th, which has the 2nd part pending publication). MacOS Internals: Part 1: https://www.amazon.com/MacOS-iOS-Internals-User-Mode/dp/099105556X/ https://www.amazon.com/MacOS-iOS-Internals-User-Mode/dp/0991... Part 2: https://www.amazon.com/MacOS-iOS-Internals-II-Kernel/dp/0991055578/ https://www.amazon.com/MacOS-iOS-Internals-II-Kernel/dp/0991... Part 3: https://www.amazon.com/MacOS-iOS-Internals-III-Insecurity/dp/0991055535/ https://www.amazon.com/MacOS-iOS-Internals-III-Insecurity/dp... Linux Kernel Programming: Part 1: https://www.amazon.com/Linux-Kernel-Development-Cookbook-programming/dp/178995343X/ https://www.amazon.com/Linux-Kernel-Development-Cookbook-pro... Part 2: https://www.amazon.com/Linux-Kernel-Programming-Part-Synchronization/dp/180107951X/ https://www.amazon.com/Linux-Kernel-Programming-Part-Synchro... The Linux Programming Interface: https://www.amazon.com/Linux-Programming-Interface-System-Handbook/dp/1593272200/ https://www.amazon.com/Linux-Programming-Interface-System-Ha... General Systems Administration: Essential Systems Administration: https://www.amazon.com/gp/product/0596003439/ https://www.amazon.com/gp/product/0596003439/ UNIX and Linux Systems Administration Handbook: https://www.amazon.com/UNIX-Linux-System-Administration-Handbook/dp/0134277554/ https://www.amazon.com/UNIX-Linux-System-Administration-Hand... The Linux Command Line and Shell Scripting Bible: https://www.amazon.com/Linux-Command-Shell-Scripting-Bible/dp/111898384X/ https://www.amazon.com/Linux-Command-Shell-Scripting-Bible/d... UNIX Shell Programming: https://www.amazon.com/Unix-Shell-Programming-Stephen-Kochan/dp/0672324903/ https://www.amazon.com/Unix-Shell-Programming-Stephen-Kochan... BASH Hackers Wiki: https://wiki.bash-hackers.org/ https://wiki.bash-hackers.org/ TLDP Advanced BASH Scripting Guide: https://tldp.org/LDP/abs/html/ https://tldp.org/LDP/abs/html/ The Debian Administrator's Handbook: https://debian-handbook.info/browse/stable/ https://debian-handbook.info/browse/stable/ TLDP Linux System Administrator's Guide: https://tldp.org/LDP/sag/html/index.html https://tldp.org/LDP/sag/html/index.html Performance & Benchmarking: Systems Performance: https://www.amazon.com/Systems-Performance-Brendan-Gregg-dp-0136820158/dp/0136820158 https://www.amazon.com/Systems-Performance-Brendan-Gregg-dp-... (this is Brendan Gregg's book where you learn about the magic of dtrace) BPF Performance Tools: https://www.amazon.com/Performance-Tools-Addison-Wesley-Professional-Computing/dp/0136554822/ https://www.amazon.com/Performance-Tools-Addison-Wesley-Prof... (the newer Brendan Gregg book about BPF, stellar) The Art of Computer Systems Performance Analysis: https://www.cse.wustl.edu/~jain/books/perfbook.htm https://www.cse.wustl.edu/~jain/books/perfbook.htm (no longer available from Amazon, but is available direct from publisher. This is basically the one book you should read about creating and structuring benchmarks or performance tests) I guess that's a "reading list", but this is just a small part of what you need to know to excel in systems operations. I would say for the typical software developer writing web applications, the most important thing to know is how databases work and how networking works, since these are going to be the primary items affecting your application performance. But there's obviously topics not included in this list that are also worth understanding, such as browser/DOM internals, how caching and CDNs work, and web-specific optimizations that can be achievable with HTTP/2 or QUIC. For the average software developer writing desktop applications, I'd say make sure you /really/ understand OS internals... at the base everything you do on a computer system is based on what the OS provides to you. Even though you are abstracted (possibly many layers) away from this, being able to peel back the layers and understand what's /really/ happening is essential to writing high-quality application code that is performant and secure, as well as making you a champ at debugging issues. If you're trying to get into systems operations as a field, this is just a brush over the top surface and there's a lot deeper diving required.
• deleted 5y ago
  [deleted]
• Shadonototro 5y ago
  i'm running my SaaS for 10 years already, and still using the same $5 plan a month HTTPS and certificates? i have no clue how to setup that, i use dns from cloudflare and they have it all automatic for free if your employees are asking you to pay ton of money for your services, hire someone else
• tambourine_man 5y ago
  > These benchmarks show that a very cheap server can easily handle 50 requests a minute I think you mean a second, but yeah, old tech is fast. I find it funny when I read “raw html” emphatically, as if it was akin to writing assembly.
• wyager 5y ago
  When my stuff has been shared on HN and Reddit, I’ve seen peaks of around 500 rps (according to google analytics), so if you can hit 1k rps you’re almost certainly ready to weather any kind of viral sharing that a blog might experience. Several krps on cheap VPS hardware is easy with a good compiled language backend (Haskell’s Warp is a good one). If you use a node/python/ruby/other interpreted backend, you will need aggressive caching through a reverse proxy.
• muzani 5y ago
  I was skeptical that it would fail under real situations (the calculation says it's 50 requests/second). But it looks like reaching the top of HN didn't crash it.
• brokencode 5y ago
  It’s the database part that gets expensive for web applications. Serving up static web pages is absolutely trivial for modern servers. The database is also the part that doesn’t easily scale, unless you pick a highly scalable database from the outset, and those have their own complexity and tradeoffs as well. That’s why I believe every project should start with a bulletproof model of how the database will work first, then fill in the other details from there. It’s not always as easy as picking Postgres and calling it a day, unfortunately.
• danbrooks 5y ago
  I hosted draftsim.com on a $3/month hosting plan for a few years. We served 500GB of data the first month. I imagine that the hosting company lost money on us (but they never called to complain).
  • flobosg 5y ago
    Just wanted to thank you for draftsim.com! As someone who got into MTG Arena a few months ago it has been very useful to learn some basics.
• sam0x17 5y ago
  Impressed that it is front page of HN and it isn't "This page cannot be displayed" with that kind of premise. Props.
• welder 5y ago
  This one's even cheaper: My £0 a month server can handle 4.2M requests a day [1] [1] https://ahamlett.com/blog/post/My-%C2%A30-a-month-server-can-handle-4.2M-requests-a-day/ https://ahamlett.com/blog/post/My-%C2%A30-a-month-server-can...
• bob1029 5y ago
  All these "X requests per unit time" posts are starting to make me want to break out some of my experimental code... I have some services that can process several million events per second. This includes: compressing the event batch, persisting to disk, validation of business logic, execution of all view updates (state tracked server-side), aggregation and distribution of client update events, etc. These implementations are easily capable of saturating NVMe flash. If you want to see where the theoretical limits lie, check out some of the fringe work around the LMAX Disruptor and .NET/C#: https://medium.com/@ocoanet/improving-net-disruptor-performance-part-3-introducing-the-valuedisruptor-5b467730bbe https://medium.com/@ocoanet/improving-net-disruptor-performa... You will find the upper bound of serialized processing to be somewhere around 500 million events per second. Personally, I have not pushed much beyond 7 million per second, but I also use reference types, non-ideal allocation strategies, etc. For making this a web-friendly thing: The trick I have found is to establish a websocket with your clients, and then pipe all of their events down with DOM updates coming up the other way. These 2 streams are entirely decoupled by way of the ringbuffer and a novel update/event strategy. This is how you can chew through insane numbers of events per unit time. All client events get thrown into a gigantic bucket which gets dumped into the CPU furnace in perfectly-sized chunks. The latency added by this approach is measured in hundreds of microseconds to maybe a millisecond. The more complex the client interactions (i.e. more events per unit time), the better this works. Blazor was the original inspiration for this. I may share my implementation at some point in the near future.
  • mariushn 5y ago
    > The trick I have found is to establish a websocket with your clients, and then pipe all of their events down with DOM updates coming up the other way. These 2 streams are entirely decoupled by way of the ringbuffer and a novel update/event strategy. Could you detail this, please? I don't get it. What is the flow? 1. Browser is sending events to web server via web socket, instantly as the event is occurring (?) 2. ? (what exactly does the server do?)
    • bob1029 5y ago
      You got 1 correct. Everything that happens gets sent immediately as an event to the server (e.g. KeyDownEvent). These are pushed without blocking for a response to each - The websocket guarantees delivery and ordering. Upon receiving an event from the client socket, it is immediately inserted into the LMAX ring buffer for processing. Updates to the client are triggered by events+state determining when a redraw is required and issuing a special "ClientRedraw" event into the same queue. These events are grouped by client so that we can aggregate multiple potential updates in a single actual redraw. These result in view updates being pushed back down to the relevant clients. One performance trick here is that the client redraw is dispatched asynchronously from the server, so there is no blocking on processing the subsequent batches each time. You can think of an E2E client view update as always requiring 2 events - the client event that triggered the change to domain state, and the actual redraw event(s) that result. For applications where the client should update at a fixed interval (e.g. game), a high performance timer implementation injects periodic redraw events. Because the upper bound of the ring buffer latency is around a millisecond, this allows for incredibly low jitter on real time events. Scheduling client draws as simple domain events is feasible.
• zhuzhu 5y ago
  My website https://www.v2ph.com https://www.v2ph.com $6 VPS can handle 500,000 requests daily On this server, I have PHP-fpm workers, nginx and MariaDB The average CPU usage about 30%, load average is about 0.5
• deleted 5y ago
  [deleted]
• iambozdar 5y ago
  You have clearly stated in your footnote that if anything goes wrong then the given numbers can go down. To be honest, that's what happens all the time. Developers do something wrong and number of requests just drop down to the floor. Only static websites are the one which handle large amount of requests at low cost. Web hosting providers don't make money out of those clients, so they run shared plans.
• mkl95 5y ago
  Nginx can handle ~250M requests a day out of the box, and ~600M by tuning a few parameters. * * https://www.cloudbees.com/blog/tuning-nginx https://www.cloudbees.com/blog/tuning-nginx
• seandong 5y ago
  All these examples show even more why software engineer(not developer) is a discipline where 10x salary difference can be seen between the best and the worst. For the 10x you are paying, you are getting a (n^2 - logN) times performance gain, especially when you are dealing with problems with large amount of data. However, relying on people themselves is often not the best stable solution. I am wondering if all these N^2 mistakes people made can be prevented by innovative means like language features, framework improvements, tooling and etc. And I'm talking about prevention, not the post mortem perf measure and fix kind
• erdo 5y ago
  Working on other companies' mobile apps, about half the performance problems I've discovered have been down to some accidental crazy, like initialising something you only need once, in a loop, in 3 different places (because the code has become so unnecessarily complicated that no one really knows what it's doing). The rest are due to some piece of code accidentally blocking the UI thread. A well written mobile app doesn't really have any need to be sluggish at all, including smooth animations and fast scrolling lists, it was doable 10 years ago, it's doable now. (*I don't know about games). But unlike on the server side, the accepted wisdom in most places I've worked at is that the answer to the performance problems is: a new framework. (I feel like this is a lie that developers tell the business side, and maybe themselves. It avoids having to explain that software is hard, sometimes you don't get it right the first time, and if you don't spend time and effort tending to it, it can turn into an ungodly and expensive mess - and that's got nothing to do with the hardware or the framework)
• fnord77 5y ago
  apache + wsgi isn't even close to being the most performant webapp server software, either. Bet he'd get 5x the performance out of nginx + lua on the same virtual hardware.
• baby 5y ago
  I used to be on the top 500 of Alexa with a shared hosting server running PHP. Never had any issue. Mind you, I wasn't doing anything complicated, but still.
• noduerme 5y ago
  When I launched my former bitcoin casino in 2011 (it's gone, but it was a casino where all games, even roulette tables were multiplayer, on a platform built from scratch starting in '08), I handled all web requests through a server in Costa Rica that cost about $6/mo. Where I had a shell corporation for $250/year. Once the front end -- the bullet containing the entire casino code, about 250kb -- loaded, from Costa Rica, and once a user logged in, they were socketed to a server that handled the gaming action in the Isle of Man. Graphics and sounds were still sent from the Costa Rica server. I didn't have a gaming license in the IoM, though - that was around $400k to acquire legally. So I found a former IoM MP who was a lawyer, who wrote a letter to the IoM gov't stating that we didn't perform random calculations on their server, thus weren't a gambling site under IoM law. Technically that meant that no dice roll or card shuffle leading to a gambling win or loss took place on that server. So the IoM server handled the socketed user interactions, chat, hand rotation and tournament stuff. Also the bitcoin daemon and deposits/withdrawals. But to avoid casino licensing, I then set up a VPS in Switzerland that did only one thing: Return random numbers or shuffled decks of cards, with its own RNG. It was a quick backend cUrl call that would return a fresh deck or a dice roll, for any given random interaction on the casino servers. The IoM server would call the Swiss server every time a hand was dealt or a wheel was spun; the user was still looking at a site served from a cheap web host in Costa Rica. And thus... yeah, I guess I handled millions of requests a day over a $6 webserver, if you want to count it that way.
  • deleted 5y ago
    [deleted]
  • arcticbull 5y ago
    Very crafty. Nice. I'm legitimately impressed by every part of that.
  • Gepsens 5y ago
    You're a fricking genius.
    • noduerme 5y ago
      Sarcasm much? I'm a full blown idiot. I was just solving one problem at a time.
• kjgkjhfkjf 5y ago
  That's less than 50qps. I can't count that low.
• ignoramous 5y ago
  I run a FOSS Pi-Hole esque public DoH resolver in the most expensive way possible (over Cloudflare Workers) and it costs $2 for 4M requests. Granted the CPU slice is limited (50ms) but the IO slice (30s) is plenty for our workloads (stub dns-resolution). The reason this is cheaper in a sense is because Workers deploys globally and needs zero devops. Per our estimates, this setup (for our workload) gets expensive once the request range goes beyond 1.5 billion a month after which deploying to colos worldwide becomes cheaper even with associated cost of devops.
• stef25 5y ago
  And here I am building a simple API using Lumen (Laravel's stripped down, hopefully faster cousin) getting response times that are just abysmal. The raw queries themselves are fast enough, but for some reason running them in a framework, transforming them in to a Resource and dumping it as json takes so long that I'm scared to find out what this super popular framework is even doing under the hood. Once I learn enough Python I'd like to compare its performance to something like FastAPI. But even that probably won't come near what these recent posts are describing. (Disclaimer - it's just a side project and I haven't really looked in to making it faster)
  • Kiro 5y ago
    That's strange. Laravel should be able to handle thousands of requests a second even on the cheapest hardware.
    • euoia 5y ago
      In my experience with Laravel, it is often the transformation to JSON that takes some time for large responses.
• deleted 5y ago
  [deleted]
• bullen 5y ago
  My Java HTTP app. server manages that on a Raspberry 2 at 2 watts serving this reddit clone: http://talk.binarytask.com http://talk.binarytask.com Really we need to compare apples to apples (how many watt)! Most of you have an external IP address, open port 80 and put it to good use before they put you behind a shared IP!
• 1432132143 5y ago
  ROTFL Wikipedia 2010 handle 7mln request per seconds suqids in USA Europe in Japan. Some random dud who do benchmark with 0.00001 seconds TCP on localhost "HEllo World" only ycombinator lamerers can believe this without asking questions.
• privacyonsec 5y ago
  My 1cpu 2gb ram server: - nginx - slack bot behind Django - OpenVPN - Minecraft server (20 players) - tunneling (reverse proxy for local dev)
  • whatsthatabout 5y ago
    You're running an 20 players mc server with that hardware? How? In my experience Minecraft-servers are insanely resource hungry, especially whilst generating new parts of the map or larger red stone contraptions. Back when I played around with hosting servers I used the "most optimized" mc-server fork called Tuinity - and even with that I had to allocate way more cpu and ram to the vms then u use. Would love to hear about your setup.
  • mktk1001 5y ago
    Press X to doubt.
• iradik 5y ago
  "If we can handle 50 requests a second that means we can handle 4.2 million requests a day." Not really. Real world traffic won't be uniform over one entire day. 50 QPS would be more accurate.
• Ice_cream_suit 5y ago
  In contrast, the Australian 2016 National Census crashed and burned since the system could not cope with over 250 requests a second. Parliamentary enquiry:(PDF) https://www.aph.gov.au/DocumentStore.ashx?id=0a7f6bd5-8716-40d2-a491-980a1645ea81&subId=413514 https://www.aph.gov.au/DocumentStore.ashx?id=0a7f6bd5-8716-4... https://www.zdnet.com/article/census-2016-among-worst-it-debacles-in-australia-labor/ https://www.zdnet.com/article/census-2016-among-worst-it-deb... https://www.theguardian.com/australia-news/2016/aug/10/computer-says-no-australian-census-shambles-explanation-depends-on-who-you-ask https://www.theguardian.com/australia-news/2016/aug/10/compu...
• danielsamuels 5y ago
  Can it? > Service Unavailable > The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.
• shultays 5y ago
  The new metrics for server performance is "unique buzzwords per paragraph". If you don't write a 4 page blog post with at least 4-5 ubpp then it is shit.
• fimdomeio 5y ago
  Most of the times the more interesting question is not really how to make a server that can handle 4.2M requests a day, but how to make something so useful that it gets more than 100 pageviews a day.
• KronisLV 5y ago
  It feels to me like most websites out there could run on way less hardware, if only people would embrace a few things. #1 Minimalism. You don't need 400 KB of JS to display some mostly text content to your users with some interactivity sprinkled in. You don't need to reinvent office software, or very rich text editors in browsers, stop using the web as a universal delivery platform/mechanism, because that's not what it was meant for. When browsers will ship integrated dependencies so that even CDNs don't need to be hit (like versions of jQuery, Bootstrap and numerous JS frameworks as well as WASM code like Blazor which contains a .NET runtime), then you'll be able to do that, but arguably that will never happen. Use the web as a platform for displaying primarily text content with the occasional images, forms and a little bit of interactivity sprinkled in. Most sites out there simply aren't and shouldn't be like this (that said, when you have exceptional reasons for throwing aside that suggestion, do so): https://geargenerator.com https://geargenerator.com #2 Static content. You don't need to use Wordpress, Drupal, Joomla or many of the other CMSes out there, since they can get really heavyweight with numerous plugins and are not only a security challenge, but are also problematic from a performance perspective. Consider using static site generators instead. When reading an article of yours, the DB shouldn't even be hit, since most of the article contents are unlikely to change often, so you should be able to pre-render each of the article versions as a set of static HTML and use the common JS/CSS that you already have for the rest of the articles. Furthermore, it's easy to just jump into CMSes and introduce ungodly amounts of complexity, all of which cause your back end to process bunches of code for each request. Static files don't have that drawback. #3 Caching. Know when and what to cache, and how. Images, JS files, CSS files and even entire HTML pages should be cache friendly. Know which ones aren't, make exceptions for those and cache everything else. Not only is it not necessary to hit the DB for many of the pages in your site at all, but also sometimes you shouldn't even hit the back end either. The most popular pages of your site should just live in a cache somewhere, be it within your web servers or a separate solution, so that they can be returned instantly. HTML is good for this, use it. Furthermore, know what cache policies to use. Sometimes even the cache resources shouldn't be redownloaded, if the user already has these resources loaded from a different page. Use bundle splitting responsibly, extract common functionality in easily cacheable bundles and set the appropriate headers. And yet, i've seen a surprising amount of ignorance in regards to caching, static site generation and even how large webpages have gotten: https://idlewords.com/talks/website_obesity.htm https://idlewords.com/talks/website_obesity.htm I don't claim to know it all, but working towards the goal of efficiently using pages should definitely be viewed as an important one: be it because you want to pay less for your infrastructure, or care about the environment, or even just want to manage fewer nodes. Instead, nowadays far too many orgs just try to be the first to market and ignore the engineering based approach to ensuring that the solutions are not only functional but also sustainable. That saddens me.
• anaganisk 5y ago
  But can it run k8s /s
• gruellan 5y ago
  Hey! Someone else from Jersey! Nice one
• ksec 5y ago
  160 Request /s for a No DB Call, Simple File Serving on a Single Core System. Why am I the only one not impressed by this.
• _alex_ 5y ago
  tl;dr: a machine with 2 GB of ram that is doing pulling a blog post out of a DB and passing it through Django can do ~50tps