4 ms·

Running a database accessed that many times on a $4 Digital Ocean droplet? I'd be very curious to see that ;) The web hosting costs basically nothing. Most of

Running a database accessed that many times on a $4 Digital Ocean droplet? I'd be very curious to see that ;) The web hosting costs basically nothing. Most of the cost comes from the database.

tosh 2y ago
[flagged]
- deleted 2y ago
  [deleted]
diggan 2y ago
> Running a database accessed that many times on a $4 Digital Ocean droplet? How many times per second is the DB actually accessed? As far as I can tell my the metrics, they're doing ~1.7 requests/minute, you'll have a hard time finding a DB that couldn't handle that. In fact, I'd bet you'd be able to host that website (the database) in a text file on disk without any performance issues whatsoever.
ndriscoll 2y ago
6k visits per week * 5 page views per visit is one view per 20 seconds on average. Even very modest hardware with naively written application code should have no problem handling thousands of CRUD database queries per second (assuming every query doesn't need a table scan or something). Modern computers are mind-bogglingly powerful. An old laptop off eBay can probably handle the load for business needs for all but the very largest corporations.
- npsomaratna 2y ago
  My understanding is that DO VPS’ are underpowered (as are VPS offerings from most other VPS vendors). Dollar for dollar, bare metal stuff from Hetzner, OVH, etc are far more powerful. That said, I completely agree-a $4/month DO VPS can run MySQL, and should easily handle this load; in fact I’ve handled far bigger loads in practice. On a tangent: any recommendations for good US-based bare metal providers (with a convenience factor comparable to OVH, etc)?
  - diggan 2y ago
    > good US-based bare metal providers The times I've needed it, DataPacket (not based in US, but has servers in the US) and Vultr (based in the US) have both been good to me.
    - npsomaratna 2y ago
      Thank you so much. I’ll take a look at these.
    - mminer237 2y ago
      In a trademark travesty, I must ask DataPacket.com or DataPacket.net?
      - diggan 2y ago
        Sorry, I was referring to datapacket.com
  - mxuribe 2y ago
    Hetzner is of course not U.S. based, but has expanded to have 2 U.S. sites (Oregon i think, and Virginia)....so that could be an option maybe. Caveat: i have not leveraged Hetzner in the U.s.....so can not speak to their quality.
    - npsomaratna 2y ago
      That’s news thank you. I’ll check this out.
      - mxuribe 2y ago
        Uh, actually at a quick glance, seems the U.s. sites are more for their cloud offering and maybe not bare metal servers.....i think (sadly): https://www.hetzner.com/cloud https://www.hetzner.com/cloud
- horsawlarway 2y ago
  So many people don't seem to understand how efficient modern machines are. As someone who is literally using old laptops to host things from my basement on my consumer line (personal, non-commercial) and a business line (commercial)... I can host this for under 50 bucks a year, including the domain and power costs, and accounting for offsite backup of the data. I wish people understood just how much the "cloud" is making in pure profit. If you're already a software dev... you can absolutely manage the complexity of hosting things yourself for FAR cheaper. You won't get five 9s of reliability (not that you're getting that from any major cloud vendor anyways without paying through the nose and a real SLA) but a small UPS will easily get you to 99% uptime - which is absolutely fine for something like this.
  - beeboobaa3 2y ago
    The problem is that my coworkers are morons who seem incapable of remembering to run a simple `explain analyze` on their queries. They'd rather just write monstrosities that kindasorta work without giving a single damn about performance. It seems like computers are getting more capable, but developers are becoming less capable at roughly the same pace.
    - 9dev 2y ago
      > It seems like computers are getting more capable, but developers are becoming less capable at roughly the same pace. And that makes perfect sense. Why should humans inconvenience themselves to please the machine? If anyone’s at fault, it’s the database for not being smart enough to optimize the query on its own.
      - beeboobaa3 2y ago
        [flagged]
        9dev 2y ago
        That’s besides the point I’m making. Technology should develop towards simplifying humanity’s life, not making it more complicated. It’s a good thing we don’t have to use clever loop constructs anymore, because compilers do it for us. It’s a good thing we don’t have to obsess about the right varchar size anymore, because Postgres‘ text does the right thing anyway. It’s a systemic problem. You’re going to loose the battle against human nature: Ever noticed how, after moving from a college dorm into a house, people suddenly manage to fill all the space with things? It’s not like the dorm was enough to fit everything they ever needed, but they had to accommodate themselves to it. This constraint is artificial, exhausting to keep up, and, if gone, will no longer be adhered to. If a computer suddenly becomes more powerful, developers aren’t going to keep up their good habits performance optimisation, because they had those only out of necessity in the first place.
- tmpz22 2y ago
  You're right but I'll play devil's advocate for teaching purposes: * Usage won't be uniformly distributed and you may need to deal with burst traffic for example when a new version is released and all your users are pulling new config data. * Your application data may be very important to your users and keeping it on a single server is a significant risk. * You're users may be geographically distributed such that a user on the other side of the world may have a severely degraded experience. * Not all traffic is created equal and, especially paired with burst traffic, could have one expensive operation like heavy analytical query from one user cause timeouts for another user. Vercel does not solve all of these problems, but they are problems that may be exasperated by a $4 droplet. All said I still highly encourage developers to not sell their soul to a SaaS product that could care less about them and their use case and consider minimal infrastructure and complexity in order to have more success with their projects.
  - ffsm8 2y ago
    * that's just static files. Even a $4 droplets will hardly ever get into issues serving that, even with hundreds of simultaneous requests. * Okay, I guess that means we should use 2? So that's $8 now. * Vercel really doesn't help you there beyond serving static files from cdn. That hardly matters at this scale, you should keep in mind that you "only" add about 100ms of latency by serving from the other side of the globe. While that has an impact, it's not really that much. And you can always use another cdn too. They're very often free for html/js/css * Burst traffic is an issue, especially trolls that just randomly DOS your public servers for shits and giggles. That's pretty much the only one vercel actually helps you against. But so would others, they're not the only ones providing that service, and most do it for free. Frankly, the only real and valid reason is the previously mentioned: they've likely got the money and don't mind spending it for the ecosystem. And if they like it... Who are we to interfere? Aside from pointing out how massively they're overpaying, but they've gotta be able to handle that if they're willing to publish an article like this
    - jasonm23 2y ago
      People use Vercel ... because... ...haven't worked it out yet, all I can come up with is "they don't know any better". Surely that can't be true?
fs0c13ty00 2y ago
My open source service, lrclib.net, handles approximately 200 requests per second at peak (yes you read that right, it's approximately 12000 requests per minute) on a simple €13 Hetzner cloud server (4 AMD based VCPU, 8GB RAM). I'd love to write a blog post about how I made it possible sometime in the future, but basically, I cheated by using Rust together with SQLite3 and some caching. I was surprised by the cost of Vercel in that blog post too, which is why I dislike all kinds of serverless/lambda/managed services. For me, having a dozen people subscribing to $1-$2/month sponsorship on GitHub Sponsors is enough to cover all the costs. Even if no one donates, I’d still have no trouble keeping the project running on my own.
Dachande663 2y ago
I didn't mean it quite so insultingly, but yes, even a very modest server would handle that kind of load easily. You're not particularly high throughput (a few requests per second?) and I imagine the database is fairly efficient (you're not storing pages of text or binary blobs). I think you'd be pleasently surprised by what a little VPS can do.
explain 2y ago
Running the 800th most popular website in the world (25-50M pageviews per day) on a 1GB VPS (Spring Boot, MariaDB, Redis) Very possible.
avree 2y ago
It’s surprising that you ask for advice on this topic in your blog, but then are very dismissive (complete with sarcastic wink) to the advice?
codazoda 2y ago
I think it would be fine. I run a little private analytics service for my own websites. That service isn't as busy but handles ~11k requests per month. It logs to a SQLite database. It does this on a little Raspberry Pi 400 in my home office and it's not too busy. The CPU sits at 1% to 3% on average. Obviously there are a lot of differences in my setup but I would think you could handle 10x the traffic with a small VPS without any trouble at all. You can read a little bit more about my analytics setup here: https://joeldare.com/private-analtyics-and-my-raspberry-pi-400 https://joeldare.com/private-analtyics-and-my-raspberry-pi-4...
klabb3 2y ago
You could run it on Cloudflare workers for free with plenty to spare. You get 5M reads/100k writes per day on D1. OTOH If you want managed Postgres it seems like you always have to pay a fairy high minimum.
bdlowery 2y ago
https://f5bot.com/ https://f5bot.com/ was free for like 8 years and it processed hundreds of thousands of db records a day, and it barely cost anything.
trollied 2y ago
"that many times". It's nearly zero traffic.
Saris 2y ago
They have under 1k visits per day, unless it's a really heavy app for some reason just about any basic VPS should handle a Webserver + DB for that just fine. It does feel like the tech community as a whole has forgotten how simple and low resource usage hosting most things is, maybe due to the proliferation of stuff like AWS trying to convince us that we need all this crazy stuff to do it?

Running a database accessed that many times on a $4 Digital Ocean droplet? I'd be very curious to see that ;) The web hosting costs basically nothing. Most of the cost comes from the database.

tosh 2y ago

[flagged]

deleted 2y ago
[deleted]

diggan 2y ago

> Running a database accessed that many times on a $4 Digital Ocean droplet? How many times per second is the DB actually accessed? As far as I can tell my the metrics, they're doing ~1.7 requests/minute, you'll have a hard time finding a DB that couldn't handle that. In fact, I'd bet you'd be able to host that website (the database) in a text file on disk without any performance issues whatsoever.

ndriscoll 2y ago

6k visits per week * 5 page views per visit is one view per 20 seconds on average. Even very modest hardware with naively written application code should have no problem handling thousands of CRUD database queries per second (assuming every query doesn't need a table scan or something). Modern computers are mind-bogglingly powerful. An old laptop off eBay can probably handle the load for business needs for all but the very largest corporations.

npsomaratna 2y ago
My understanding is that DO VPS’ are underpowered (as are VPS offerings from most other VPS vendors). Dollar for dollar, bare metal stuff from Hetzner, OVH, etc are far more powerful. That said, I completely agree-a $4/month DO VPS can run MySQL, and should easily handle this load; in fact I’ve handled far bigger loads in practice. On a tangent: any recommendations for good US-based bare metal providers (with a convenience factor comparable to OVH, etc)?
- diggan 2y ago
  > good US-based bare metal providers The times I've needed it, DataPacket (not based in US, but has servers in the US) and Vultr (based in the US) have both been good to me.
  - npsomaratna 2y ago
    Thank you so much. I’ll take a look at these.
  - mminer237 2y ago
    In a trademark travesty, I must ask DataPacket.com or DataPacket.net?
    - diggan 2y ago
      Sorry, I was referring to datapacket.com
- mxuribe 2y ago
  Hetzner is of course not U.S. based, but has expanded to have 2 U.S. sites (Oregon i think, and Virginia)....so that could be an option maybe. Caveat: i have not leveraged Hetzner in the U.s.....so can not speak to their quality.
  - npsomaratna 2y ago
    That’s news thank you. I’ll check this out.
    - mxuribe 2y ago
      Uh, actually at a quick glance, seems the U.s. sites are more for their cloud offering and maybe not bare metal servers.....i think (sadly): https://www.hetzner.com/cloud https://www.hetzner.com/cloud
horsawlarway 2y ago
So many people don't seem to understand how efficient modern machines are. As someone who is literally using old laptops to host things from my basement on my consumer line (personal, non-commercial) and a business line (commercial)... I can host this for under 50 bucks a year, including the domain and power costs, and accounting for offsite backup of the data. I wish people understood just how much the "cloud" is making in pure profit. If you're already a software dev... you can absolutely manage the complexity of hosting things yourself for FAR cheaper. You won't get five 9s of reliability (not that you're getting that from any major cloud vendor anyways without paying through the nose and a real SLA) but a small UPS will easily get you to 99% uptime - which is absolutely fine for something like this.
- beeboobaa3 2y ago
  The problem is that my coworkers are morons who seem incapable of remembering to run a simple `explain analyze` on their queries. They'd rather just write monstrosities that kindasorta work without giving a single damn about performance. It seems like computers are getting more capable, but developers are becoming less capable at roughly the same pace.
  - 9dev 2y ago
    > It seems like computers are getting more capable, but developers are becoming less capable at roughly the same pace. And that makes perfect sense. Why should humans inconvenience themselves to please the machine? If anyone’s at fault, it’s the database for not being smart enough to optimize the query on its own.
    - beeboobaa3 2y ago
      [flagged]
      - 9dev 2y ago
        That’s besides the point I’m making. Technology should develop towards simplifying humanity’s life, not making it more complicated. It’s a good thing we don’t have to use clever loop constructs anymore, because compilers do it for us. It’s a good thing we don’t have to obsess about the right varchar size anymore, because Postgres‘ text does the right thing anyway. It’s a systemic problem. You’re going to loose the battle against human nature: Ever noticed how, after moving from a college dorm into a house, people suddenly manage to fill all the space with things? It’s not like the dorm was enough to fit everything they ever needed, but they had to accommodate themselves to it. This constraint is artificial, exhausting to keep up, and, if gone, will no longer be adhered to. If a computer suddenly becomes more powerful, developers aren’t going to keep up their good habits performance optimisation, because they had those only out of necessity in the first place.
tmpz22 2y ago
You're right but I'll play devil's advocate for teaching purposes: * Usage won't be uniformly distributed and you may need to deal with burst traffic for example when a new version is released and all your users are pulling new config data. * Your application data may be very important to your users and keeping it on a single server is a significant risk. * You're users may be geographically distributed such that a user on the other side of the world may have a severely degraded experience. * Not all traffic is created equal and, especially paired with burst traffic, could have one expensive operation like heavy analytical query from one user cause timeouts for another user. Vercel does not solve all of these problems, but they are problems that may be exasperated by a $4 droplet. All said I still highly encourage developers to not sell their soul to a SaaS product that could care less about them and their use case and consider minimal infrastructure and complexity in order to have more success with their projects.
- ffsm8 2y ago
  * that's just static files. Even a $4 droplets will hardly ever get into issues serving that, even with hundreds of simultaneous requests. * Okay, I guess that means we should use 2? So that's $8 now. * Vercel really doesn't help you there beyond serving static files from cdn. That hardly matters at this scale, you should keep in mind that you "only" add about 100ms of latency by serving from the other side of the globe. While that has an impact, it's not really that much. And you can always use another cdn too. They're very often free for html/js/css * Burst traffic is an issue, especially trolls that just randomly DOS your public servers for shits and giggles. That's pretty much the only one vercel actually helps you against. But so would others, they're not the only ones providing that service, and most do it for free. Frankly, the only real and valid reason is the previously mentioned: they've likely got the money and don't mind spending it for the ecosystem. And if they like it... Who are we to interfere? Aside from pointing out how massively they're overpaying, but they've gotta be able to handle that if they're willing to publish an article like this
  - jasonm23 2y ago
    People use Vercel ... because... ...haven't worked it out yet, all I can come up with is "they don't know any better". Surely that can't be true?

fs0c13ty00 2y ago

My open source service, lrclib.net, handles approximately 200 requests per second at peak (yes you read that right, it's approximately 12000 requests per minute) on a simple €13 Hetzner cloud server (4 AMD based VCPU, 8GB RAM). I'd love to write a blog post about how I made it possible sometime in the future, but basically, I cheated by using Rust together with SQLite3 and some caching. I was surprised by the cost of Vercel in that blog post too, which is why I dislike all kinds of serverless/lambda/managed services. For me, having a dozen people subscribing to $1-$2/month sponsorship on GitHub Sponsors is enough to cover all the costs. Even if no one donates, I’d still have no trouble keeping the project running on my own.

Dachande663 2y ago

I didn't mean it quite so insultingly, but yes, even a very modest server would handle that kind of load easily. You're not particularly high throughput (a few requests per second?) and I imagine the database is fairly efficient (you're not storing pages of text or binary blobs). I think you'd be pleasently surprised by what a little VPS can do.

explain 2y ago

Running the 800th most popular website in the world (25-50M pageviews per day) on a 1GB VPS (Spring Boot, MariaDB, Redis) Very possible.

avree 2y ago

It’s surprising that you ask for advice on this topic in your blog, but then are very dismissive (complete with sarcastic wink) to the advice?

codazoda 2y ago

I think it would be fine. I run a little private analytics service for my own websites. That service isn't as busy but handles ~11k requests per month. It logs to a SQLite database. It does this on a little Raspberry Pi 400 in my home office and it's not too busy. The CPU sits at 1% to 3% on average. Obviously there are a lot of differences in my setup but I would think you could handle 10x the traffic with a small VPS without any trouble at all. You can read a little bit more about my analytics setup here: https://joeldare.com/private-analtyics-and-my-raspberry-pi-400 https://joeldare.com/private-analtyics-and-my-raspberry-pi-4...

klabb3 2y ago

You could run it on Cloudflare workers for free with plenty to spare. You get 5M reads/100k writes per day on D1. OTOH If you want managed Postgres it seems like you always have to pay a fairy high minimum.

bdlowery 2y ago

https://f5bot.com/ https://f5bot.com/ was free for like 8 years and it processed hundreds of thousands of db records a day, and it barely cost anything.

trollied 2y ago

"that many times". It's nearly zero traffic.

Saris 2y ago

They have under 1k visits per day, unless it's a really heavy app for some reason just about any basic VPS should handle a Webserver + DB for that just fine. It does feel like the tech community as a whole has forgotten how simple and low resource usage hosting most things is, maybe due to the proliferation of stuff like AWS trying to convince us that we need all this crazy stuff to do it?