k3s on single VPS. cert-manager and external-dns for maintenance free LE certs. nginx ingress (I would probably go Contour+Envoy if I was to set it up now). K

k3s on single VPS. cert-manager and external-dns for maintenance free LE certs. nginx ingress (I would probably go Contour+Envoy if I was to set it up now). Kotlin w/Gradle. Build + Deploy single command `gradle deployStage`. jib for container images, don't need Docker, reproducible super fast builds. Prometheus + Grafana for metrics and alerting. Things I like about it: - JVM is super low maintenance runtime, tune it once and pretty much good to go. Very easy to diagnose memory or performance problems in production. I even have the debugger agent loaded in prod listen on localhost so I can use kubectl port-forward to access it and diagnose bugs that only occur in production or hung threads etc. - Prometheus seems like overkill but it's just so much less effort because everything already has a prom exporter. - Using k8s API saves a bunch of time again. No need to screw around with docker-compose or some godawful config management from the 2000s like Ansible (ick). Things I don't like: - It's not HA. The DB pods push their WALs to S3 and I have script to spawn a new PG from S3 using PITR. This is how I test new code so it's known working but it still means that if my server goes down it's down until I do something about it. - The box is a bit snowflaky. I didn't set it up using a script, it was hand configured up until the k8s level. - I rarely upgrade k3s because I have to do it manually. If I was using EKS or GKE I would probably press the upgrade button more often. - Grafana is a chore. grafonnet helps (I also use Tanka w/jsonnet for k8s manifests).