4 ms·

A $40/mo VPS. Caddy web server. Supervisor to restart automatically. Git to deploy. CI using GitHub actions which build, run tests, and push to server on succes

by alxmng 3y ago

A $40/mo VPS. Caddy web server. Supervisor to restart automatically. Git to deploy. CI using GitHub actions which build, run tests, and push to server on success. Automatic VPS snapshots and a cronjob to backup the DB to cloud storage. DB runs on the same box. This has worked for years with practically zero maintenance.

oops 3y ago
> A $40/mo VPS. > This has worked for years with zero maintenance. Do you apply security updates, e.g. for Caddy, the OS, db, etc.?
- alxmng 3y ago
  Yeah, every so often I spend 5 minutes and run apt upgrade. I don't think the DB has ever been upgraded because there's no reason to. It's not exposed to the Internet anyway.
  - chatmasta 3y ago
    Your DB doesn't need to be connected to the internet for untrusted users to have a path for interacting with it. I assume your application sends queries to it, for example. And users give your application the parameters for those queries. And that's when things are working as expected. All it takes is for one of your non-DB services to be compromised, and an attacker can now connect to the DB on localhost. That's why it's best practice to put a secure password on your DB even if you only expect connections from local services. And yes, you should upgrade it too... or at least apply security patches.
    - aprdm 3y ago
      Lol, the amount of databases running real production heavy workloads for the biggest companies in the world that haven't been updated in 10y is probably very high :) at least the ones I've seen ..
    - alxmng 3y ago
      Yes, database connections are password protected.
mrngilles 3y ago
What's the size of the team working on this? I'm guessing you're OK with the tradeoff of your machine going down sometimes?
- alxmng 3y ago
  4 people but sometimes 10 with contractors. Sure, there's downtime when the VPS provider is down. That's no different than "managed" services.
  - mrngilles 3y ago
    So you just have the dns record pointing to your machine's IP? Do you have some kind of monitoring? What about logging?
    - alxmng 3y ago
      That's correct. We used to use CloudFlare, but it added too much time to each request (speed is important to us). So now it's just pointed directly at the VPS. Crashes are logged but I don't think anybody has ever looked at the logs. I'm not sure a crash has ever happened (the app is written in Haskell). Unexpected errors are saved to the database and can be viewed using the web. If we needed logs people could tail them from the VPS (ssh -t $WEB_HOST 'tail -f webserver.error.log'). I know, I know, that's crazy. I'm told real software companies are supposed to ship them to someone else, and then pay them to view the logs using a crappy web interface. No monitoring. If the site is down customers will let us know, which has happened a few times. Monitoring wouldn't help much with shipping bugs anyway.
    - aprdm 3y ago
      One can easily set up prometheus/grafana or ELK or sentry when you're that small on another VM.
ttymck 3y ago
Are you hiring :D
aprdm 3y ago
yes this but with NGINX/Systemd! Haven't used caddy - will check

by alxmng 3y ago

oops 3y ago

> A $40/mo VPS. > This has worked for years with zero maintenance. Do you apply security updates, e.g. for Caddy, the OS, db, etc.?

alxmng 3y ago
Yeah, every so often I spend 5 minutes and run apt upgrade. I don't think the DB has ever been upgraded because there's no reason to. It's not exposed to the Internet anyway.
- chatmasta 3y ago
  Your DB doesn't need to be connected to the internet for untrusted users to have a path for interacting with it. I assume your application sends queries to it, for example. And users give your application the parameters for those queries. And that's when things are working as expected. All it takes is for one of your non-DB services to be compromised, and an attacker can now connect to the DB on localhost. That's why it's best practice to put a secure password on your DB even if you only expect connections from local services. And yes, you should upgrade it too... or at least apply security patches.
  - aprdm 3y ago
    Lol, the amount of databases running real production heavy workloads for the biggest companies in the world that haven't been updated in 10y is probably very high :) at least the ones I've seen ..
  - alxmng 3y ago
    Yes, database connections are password protected.

mrngilles 3y ago

What's the size of the team working on this? I'm guessing you're OK with the tradeoff of your machine going down sometimes?

alxmng 3y ago
4 people but sometimes 10 with contractors. Sure, there's downtime when the VPS provider is down. That's no different than "managed" services.
- mrngilles 3y ago
  So you just have the dns record pointing to your machine's IP? Do you have some kind of monitoring? What about logging?
  - alxmng 3y ago
    That's correct. We used to use CloudFlare, but it added too much time to each request (speed is important to us). So now it's just pointed directly at the VPS. Crashes are logged but I don't think anybody has ever looked at the logs. I'm not sure a crash has ever happened (the app is written in Haskell). Unexpected errors are saved to the database and can be viewed using the web. If we needed logs people could tail them from the VPS (ssh -t $WEB_HOST 'tail -f webserver.error.log'). I know, I know, that's crazy. I'm told real software companies are supposed to ship them to someone else, and then pay them to view the logs using a crappy web interface. No monitoring. If the site is down customers will let us know, which has happened a few times. Monitoring wouldn't help much with shipping bugs anyway.
  - aprdm 3y ago
    One can easily set up prometheus/grafana or ELK or sentry when you're that small on another VM.

ttymck 3y ago

Are you hiring :D

aprdm 3y ago

yes this but with NGINX/Systemd! Haven't used caddy - will check