4 ms·

10k/s... is that a lot? Computers are insanely fast nowadays..!

lionkor 2y ago
No. It's not a lot. 20-30k req/s is easy for serving simple, small files. If you have beefy machine (say, a $50 hetzner rootserver), you get a few TB of storage and unlimited or cheap bandwidth. 8-16 cores can easily(!!!) push this kind of data without even heating up, not sure wtf OP is doing. Well, I know what OP is doing - they fell for the idea that the cloud is more scalable. The issue with this is that the cloud™ starts "scaling" at the first user, whereas a baremetal server needs to scale when you have saturated a 8-16 core modern CPU, a 1-10 Gb/s NiC, 30-60 GB of RAM. In other words, baremetal needs to scale when you actually run out of hardware resources, which is tens or hundreds of thousands of users later. Edit: for example, at BeamMP, we run on a few bare metal servers, and serve 22k unique users per day in a multiplayer videogame service. Funded by around 800 people donating.
- nirui 2y ago
  I'm not a fan of cloud too, but I have to admit that the networks of these big cloud providers built is just better than self-hosted ones. When they say they'll distribute your file globally, they mean it, as long as you pay of course. But I would rather say, cloud is not for everyone. Especially in the case mentioned in the article. Think this: do you really REALLY need to distribute enlarged images globally at top speed? I bet most people just don't. Same thing goes for "scaling", it's true the cloud can do that very well, but do you really need it that bad?
- jeroenhd 2y ago
  Based on the names of the endpoints, I get the idea that they're altering the image files on the fly (and probably caching processed files) based on the URL. I've seen this quite often on blogs and such. Serving files shouldn't take much CPU power, but resizing images can get quite expensive, especially if you want to achieve lower egress fees by using better compression methods. Still, you need to deal with bad scrapers. Plus, this scraper downloaded at a consistent 650mbps, taking up half the unlimited Hetzner pipe by itself; if you'd go for a 10gbps Hetzner machine, you suddenly start paying egress fees once you hit 20TB of traffic. Even then, if you go the cheapo Hetzner server route, you probably still want at least some kind of CDN to keep latency down. Add to that costs of backups and synchronising failovers, and you may end up with more traffic than you'd expect. I think going bare metal would save more than the signed URLs would, at least until the ten thousanth customer, but not everyone is proficient in maintaining servers. A lot of cloud projects I see are coming from programmers who don't want to/don't know how to maintain a Linux server and just want to run their code. If you're in that category, taking time off to learn server maintenance or hiring a sysadmin can easily be a lot more expensive than paying the extortionate rates cloud providers demand.
quectophoton 2y ago
You also need to take into account the size of each response, how long your server needs to keep the data in memory (e.g. because of latency, the requester's bandwidth, etc), whether requests to the same file can share a buffer or not, how much data you can be sending at the same time while still being responsive (e.g. without slowing down other responses, causing them to take longer, requiring you to keep those resources in memory for longer, and snowballing from there), ..., stuff like that. For short text messages, probably not an issue. With larger stuff like images or video, I would be more careful. Still, even for text-only, if you're using PostgreSQL, by default you have a limit of (I think) 100 parallel connections (or 97, because I think 3 are reserved for superusers), but each connection can only be executing one transaction at a time, so that can quickly become a bottleneck depending on your application and how fast you need to make queries vs how long your queries take to return a response. So then you might need to tune some PostgreSQL settings, or add caching, or some other way to work around the issue. If you add more services, then you also need to keep in mind the latency between those services. And so on and so on. So RAM and network would probably become an issue way earlier than CPU in most cases. TL;DR: "It depends".

No. It's not a lot. 20-30k req/s is easy for serving simple, small files. If you have beefy machine (say, a $50 hetzner rootserver), you get a few TB of storage and unlimited or cheap bandwidth. 8-16 cores can easily(!!!) push this kind of data without even heating up, not sure wtf OP is doing. Well, I know what OP is doing - they fell for the idea that the cloud is more scalable. The issue with this is that the cloud™ starts "scaling" at the first user, whereas a baremetal server needs to scale when you have saturated a 8-16 core modern CPU, a 1-10 Gb/s NiC, 30-60 GB of RAM. In other words, baremetal needs to scale when you actually run out of hardware resources, which is tens or hundreds of thousands of users later. Edit: for example, at BeamMP, we run on a few bare metal servers, and serve 22k unique users per day in a multiplayer videogame service. Funded by around 800 people donating.

nirui 2y ago
I'm not a fan of cloud too, but I have to admit that the networks of these big cloud providers built is just better than self-hosted ones. When they say they'll distribute your file globally, they mean it, as long as you pay of course. But I would rather say, cloud is not for everyone. Especially in the case mentioned in the article. Think this: do you really REALLY need to distribute enlarged images globally at top speed? I bet most people just don't. Same thing goes for "scaling", it's true the cloud can do that very well, but do you really need it that bad?
jeroenhd 2y ago
Based on the names of the endpoints, I get the idea that they're altering the image files on the fly (and probably caching processed files) based on the URL. I've seen this quite often on blogs and such. Serving files shouldn't take much CPU power, but resizing images can get quite expensive, especially if you want to achieve lower egress fees by using better compression methods. Still, you need to deal with bad scrapers. Plus, this scraper downloaded at a consistent 650mbps, taking up half the unlimited Hetzner pipe by itself; if you'd go for a 10gbps Hetzner machine, you suddenly start paying egress fees once you hit 20TB of traffic. Even then, if you go the cheapo Hetzner server route, you probably still want at least some kind of CDN to keep latency down. Add to that costs of backups and synchronising failovers, and you may end up with more traffic than you'd expect. I think going bare metal would save more than the signed URLs would, at least until the ten thousanth customer, but not everyone is proficient in maintaining servers. A lot of cloud projects I see are coming from programmers who don't want to/don't know how to maintain a Linux server and just want to run their code. If you're in that category, taking time off to learn server maintenance or hiring a sysadmin can easily be a lot more expensive than paying the extortionate rates cloud providers demand.

quectophoton 2y ago

You also need to take into account the size of each response, how long your server needs to keep the data in memory (e.g. because of latency, the requester's bandwidth, etc), whether requests to the same file can share a buffer or not, how much data you can be sending at the same time while still being responsive (e.g. without slowing down other responses, causing them to take longer, requiring you to keep those resources in memory for longer, and snowballing from there), ..., stuff like that. For short text messages, probably not an issue. With larger stuff like images or video, I would be more careful. Still, even for text-only, if you're using PostgreSQL, by default you have a limit of (I think) 100 parallel connections (or 97, because I think 3 are reserved for superusers), but each connection can only be executing one transaction at a time, so that can quickly become a bottleneck depending on your application and how fast you need to make queries vs how long your queries take to return a response. So then you might need to tune some PostgreSQL settings, or add caching, or some other way to work around the issue. If you add more services, then you also need to keep in mind the latency between those services. And so on and so on. So RAM and network would probably become an issue way earlier than CPU in most cases. TL;DR: "It depends".