5 ms·
What kills me is that this was a rather pedestrian outcome on a much cheaper 2-core virtual machine back in 2007 or so. I easily got 3K requests / sec out of m
by jiggawatts 5y ago
What kills me is that this was a rather pedestrian outcome on a much cheaper 2-core virtual machine back in 2007 or so.
I easily got 3K requests / sec out of my laptop at the same time, and it was not a trivial app!
People's expectations have shifted so much it's absurd. If you look at the TechEmpower benchmarks, ordinary VMs can easily push 100K requests per second, no sweat, even with managed languages.
Trivial stuff like static content being treated as static content (files on the disk!) not as a distributed cache in front of a database can do wonders.
Am I just old and jaded?
- genewitch 5y agoIn 2011 a company i contracted for was testing some new dell 1U servers with around 1-2TB of ram. There was a postgres database with 4000qps that could fit into tmpfs, and so i restricted postgres to 640Kb of memory and we got replication working, it took about 6 hours of babysitting. We threw the switch and watched as postgres, with 640Kb of ram and a tmpfs backed store proceeded to handle all of the query traffic. There were some stored procedures or something that were long-querying or whatever - i'm not a DB person at all, so we switched back to the regular production server about 8 minutes later. Yes, we did it in production.
- bagels 5y agoFor real, 640 kilobits?
- sigstoat 5y agoK isn't the abbreviation for kilo, so if you're going to rag on the fellow for the 'b', then you should at least be asking what a Kelvin*bit is.
- iandinwoodie 5y ago“The binary meaning of the kilobyte for 1024 bytes typically uses the symbol KB, with an uppercase letter K.” [0] 0. https://en.m.wikipedia.org/wiki/Kilobyte https://en.m.wikipedia.org/wiki/Kilobyte
- jmiserez 5y ago640KiB is very little and I'm wondering if it's a typo, given that the servers had 1-2TiB available. Postgres 9.0 released in 2010 already had 32MiB as the default for shared_buffers (with a minimum of 128KiB): https://www.postgresql.org/docs/9.0/runtime-config-resource.html https://www.postgresql.org/docs/9.0/runtime-config-resource.... and 8.1 released in 2005 used 8MB (1000*8KiB): https://www.postgresql.org/docs/8.1/runtime-config-resource.html https://www.postgresql.org/docs/8.1/runtime-config-resource....
- sigstoat 5y agoi interpreted it as "we wanted to turn the shared buffers ~off, but in a hilarious way that would suggest to someone reading the configuration file that something was going on" (bill gates, mumble mumble) but, wtf do i know, i'm the crazy guy who tries to interpret comments generously.
- genewitch 5y agoYes, it was a direct reference to Bill Gates "640 kilobytes is enough for anyone" and i typed the comment right before i fell asleep.
- bagels 5y agoThe question was more about the kilo part, even though I didn't clarify. Seems orders of magnitude too small?
- winrid 5y agoPostgres handles low memory situations well. It'll kill memory intensive queries before it crashes. I wonder if your application was getting a lot of errors back instead of successful queries :)
- genewitch 5y agothe application performed fine, even though we made the switch around 15:00 PST. The DBA was concerned because of the few long queries. Obviously the tmpfs was doing the heavy lifting, there - and if i had to do a postmortem, i'd wager that filling the OS caches was the main reason the long queries took so long. We didn't do any sort of performance tracing. The main purpose was to show that these $35k servers could essentially replace the older machines if need be, even though the old ones had FusionIO. I just removed the middleman of the PCIe bus between the application and the memory. It was a near constant argument on the floor about whether or not we could feasibly switch to SSDs in some configuration over spinning rust or even FusionIO, i wanted a third option. Basically, serve out of registered, ECC memory in front, replicate to the fusionIO and let those handle the spindled backups, which iirc was a pain point.
- huijzer 5y agoTo add to that, in 2005, a Cloudflare engineer showed that you can receive 1 million packages per second (https://blog.cloudflare.com/how-to-receive-a-million-packets/ https://blog.cloudflare.com/how-to-receive-a-million-packets...). Without processing though.
- kragen 5y agoI think even a distributed cache in front of a database shouldn't have any trouble handling 2000 requests per second. The issue is not really the number of requests per second, probably, but the number of bytes, which they don't talk about at all in the article; reading manga with no ads is a pretty static kind of application, which could be satisfied amply with a web browser or even a much simpler program loading images from a filesystem directory. Valgrind claims httpdito runs a few thousand instructions per request, but that's not really accurate; what happens is that the kernel is doing all the work. httpdito on Linux can handle about 4000 requests per second per core, nearly a million clock cycles per request, almost all of which is in the kernel. Of course it doesn't ship its logs off to Grafana. In fact, it doesn't have logs at all. But it would work fine for reading manga.
- antupis 5y agoAlso that 2000 request per second has to happen 24/7 not only quick demo session.
- kragen 5y agohttpdito is nothing if not consistent in its performance. It doesn't have enough state to have many ways to perform well at first and then more poorly later, or for that matter vice versa. (Not saying it couldn't happen, but it's not that likely.) Linux is pretty good about consistent performance, too, though it has more state.
- buzer 5y ago> The issue is not really the number of requests per second, probably, but the number of bytes, which they don't talk about at all in the article; reading manga with no ads is a pretty static kind of application, which could be satisfied amply with a web browser or even a much simpler program loading images from a filesystem directory. I assume they are talking about their more dynamic content serving in this post (for things like search, tracking which chapters are read, new chapter listing based on what user follows etc.). They have a custom CDN that is hosted by volunteers to serve the images for the manga pages. They provide some metrics for that at https://mangadex.network https://mangadex.network, there are also some older screenshots where they hit 3.2GB/s.
- deleted 5y ago[deleted]
- z3t4 5y ago2k sockets on a test bed vs 2k real user request in production is very different. I doubt you ran a top 1000 Alexa site on your laptop. Today we need to deal with SSL which eats from the performance budget.
- jiggawatts 5y ago> SSL which eats from the performance budget. That was a short-lived thing, and has now become a myth perpetuated by companies like Citrix and F5 that sell "SSL offload" appliances for $$$. Have you benchmarked the overhead of TLS? In my experience, a single CPU core can easily put out multiple gigabytes of AES-256 (tens of gigabits). This benchmark shows 3 GB/s (24 Gbps) for recent AMD CPUs, and nearly 40 Gbps per core for an Intel CPU: https://calomel.org/aesni_ssl_performance.html https://calomel.org/aesni_ssl_performance.html A multi-core server is very unlikely to have more than a 1-5% overheard due to TLS. Even connection set up is a minor overhead with elliptic curve certificates. This is thanks to the AES offload instructions, which are present in all server CPUs made any time in the last 5-7 years or so. As long as the modern Galois Counter Mode (GCM) is used with AES, performance should be great. Meanwhile, Citrix ADC v13 with a hardware "SSL offload card" actually slows down connections! I had a very hard time getting more than 600 Mbps through one. It seems to be the way the ASIC offload chip is architected: it seems to use a large number of slow cores, a bit like a GPU. This means that any one TLS stream will have its bandwidth capped!
- tpetry 5y agoThe problem with these benchmarks is they measure the bandwidth you can push through an established tls connection. Try to build 2000 new tls connections a seconds (yes many are still active and dont need to be restarted) that is what is the really slow part. Not sending the data over already established channels.
- lossolo 5y agoThis exactly. Most CPU time in our Haproxy is spent on crypto/new TLS sessions handling thousands of new connections per second.
- adreamingsoul 5y agoNo. I think you have a healthy perspective and we should all be questioning if current trends are beneficial/sustainable. I haven’t read the article, but the headline alone to me seems alarming, $1,500 a month is a lot of money for only 2k rps.
- reilly3000 5y agoMaybe its a lot of money just for the web servers, but for the entire infrastructure stack its pretty reasonable IMHO.
- rk06 5y agoThere is more to it that http request response. Mangadex also need to store a lot of images and distribute them.
- hnlmorg 5y agoCDNs have already solved this problem and are much cheaper than $1500/month. I've ran far more complex sites with much higher traffic for less.
- rk06 5y agoMangadex can't use cloudflare because of privacy reasons. They may be facing similar issues with other popular CDNs. I am sure they must be using some kind of CDN for sure, however, those options are unlikely to be free
- hnlmorg 5y agoPrivacy reasons? It's all static content that is publicly accessible. I don't understand what the privacy reasons could be under this context. Are they worried about CDNs logging the images their visitors access? Seems like an absurd edge case to worry about in my opinion. > however, those options are unlikely to be free I wasn't even talking about free CDNs :)
- willvarfar 5y agoExcellent post, good technical content, amazing feat. That said, I echo that the amazing feat is that they can fit modern inefficient tool choices with poor mechanical sympathy into that budget. The last decade of web-dev tooling has been pushing the TCO of systems through the roof and this post is all about how to struggle against that whilst using those tools. If they went old-school they'd get another order of magnitude savings. Many veterans know of systems doing 10x that in 10x less cost. Remember C10K was in 1999.
- vymague 5y ago> If they went old-school they'd get another order of magnitude savings. Many veterans know of systems doing 10x that in 10x less cost. Remember C10K was in 1999. How to learn more about the old-school way without getting a job related to it? Like, topic or book recommendations.
- thoughtFrame 5y agoI'm also curious about this, since in many areas of computing (not only webdev) the old-school guys take some stuff as so obvious that they don't even bother writing about it or explaining it beyond "This is obviously possible, dude". They know how to achieve this level of performance, but for everyone else, we have to cobble together fragmented insights. So if anyone out there reads this and thinks like the GP, please do write about it ;)
- hsn915 5y agoWhat topics specifically are you interested in? And where do all the people like you hang out? I'm not an old-school guy by any means .. but I might have something to contribute.
- vymague 5y ago> What topics specifically are you interested in? Well, for example, what's the old-school alternative to mangadex's solution? > And where do all the people like you hang out? We are here on HN.
- colesantiago 5y agoNot just you, but if it works for them, that's completely fine. But there are many ways to achieve 20K RPS without this type architecture and especially without k8s, for less than $1,500.
- bob1029 5y ago>20k RPS. If this metric is what you are chasing, there are ways to reliably break 1 million RPS using a single box if you don't play the shiny BS tech game. The moment you involve multiple computers and containers, you are typically removed from this level of performance. Going from 2,000 to 2,000,000 RPS (serialized throughput) requires many ideological sacrifices. Mechanical sympathy (ring buffers, batching, minimizing latency) can save you unbelievable amounts of margin and time when properly utilized.
- nine_k 5y agoI frankly don't see where containers could lower the performance. Basically a container is a glorified chroot. It has the same networking unless you asked for isolation, then packets have to follow a local (inside the host) route. It has exactly no CPU or kernel interface penalty. Maybe you wanted to say about container orchestration like k8s, with its custom network fabric, etc.
- bob1029 5y agoEven loopback is significantly slower than a direct method invocation.
- krageon 5y ago> I frankly don't see where containers could lower the performance. Have you seen most k8s deployments? It's not the containers, it's the thoughtspace that comes with them. Even just using bare containers invites a level of abstraction and generally comes with a type of developer that just isn't desirable.
- 5y ago
- deleted 5y ago[deleted]
- cthalupa 5y agoYou're not alone in making comments like this, and as I read them I'm not sure if people are missing the point or being disingenuous. Much of the work I've done professionally has happened in the webhosting space, and I'm not a new hand at it - the first "professional" website I ever ran was hosted off of a machine running on a Quantum R5000. I have served (and still serve) plenty of static content as files off the disk. My own first impulse when building anything is to use as few moving parts and as simple of a setup as possible. Requests per second are not a good metric for the amount of work being done by a system, because not all requests are made equal. You say you got 3k requests out of your laptop and that it wasn't a trivial app, but you have only provided a trivial amount of information. Taking a quick look at the features provided by the site, they offer quite a bit of flexibility on both what is and how it's displayed. Filtering options based on original and translated language, adult content filtering, multiple types of search, request routing to low quality images to save on data, category and tag filtering, tracking of what you have read and what your progress is on it, follows and notifications, permissions systems for uploading and updating content, etc. This is all on top of the basic "Display images and metadata about the images" functionality. They have concerns around privacy and being able to withstand active attackers due to the content they host. They have their own requirements about logging, analytics, etc. Their own concerns about data availability. You could not meet the design requirements of this website and serve 2k requests per second on a 2 core VM in 2007. I'm not saying there aren't inefficiencies in their architecture and further places where they could save money/increase performance/etc., but acting like two cores of lower-clockrate lower-ipc compute from a decade and a half ago could do all of the work needed to support the features and design requirements they have for this website is pretty disparaging towards the people who built this infrastructure.