3 ms·

It's wild to me that we are talking about a site running on a single 2 core VPS costing $15/mo and the developer didn't stop to consider that maybe they don't n

by paxys 3y ago

It's wild to me that we are talking about a site running on a single 2 core VPS costing $15/mo and the developer didn't stop to consider that maybe they don't need Kubernetes for this. Like...just run your damn services on the box like we've been doing since web hosting first became a thing.

CactusBlue 3y ago
For me, the main benefit of Kubernetes is declarative deployments. You can have the entire deployment automated through this and GitHub actions, without much fiddling with the production environment manually. I know, it's overkill and I don't like it that much either, but it's the industry standard.
- ben_jones 3y ago
  > but it's the industry standard. I strongly disagree with this statement. Even at the most bubbly of Silicon Valley events you will not see kubernetes for a smaller deployment be an industry standard. It’s ok to make architectural mistakes, or otherwise over engineer a thing. We all do it. Let’s not standardize it. And to your credit thank you for publishing this example as a learning opportunity to others.
  - CactusBlue 3y ago
    I mean as in what the big (and small) clouds support. I haven't seen any clouds supporting Hashicorp Nomad or Docker Swarm!
    - SJC_Hacker 3y ago
      You shoudl be able to run any of those things on any cloud. A box with sshd is a box with sshd.
  - jitl 3y ago
    k3s is a teensy sqlite based orchestrator, something tons of devs run on their laptops for local dev. I think those are all single box clusters, why is it crazy to run it on a server but not a laptop?
- paxys 3y ago
  You can get declarative deployments via a systemd config. If you really want to use Docker then use a Compose file. If jobs are what you are concerned about then I can assure you no company is skipping on software developers because they don't know Kubernetes syntax.
  - jitl 3y ago
    imo systemd is more complicated than k3s
  - everforward 3y ago
    That doesn't capture the same thing. It doesn't capture volumes, versions of binaries, nginx/postgres/whatever configs, etc. Ansible/Salt/Puppet/Chef/CFEngine are closer comparisons.
    - lolinder 3y ago
      With Docker Compose you can define volumes, versions, and configs can be volumes that point to a file that lives alongside your compose.yaml.
      - everforward 3y ago
        Sure, depending on how you want to draw a line around features. Compose won’t do multi-node orchestration of deploys (I.e. update backends 1-3 sequentially, and revert the change on all of them if any fail to deploy). It also doesn’t manage your configs; ie if you need to roll back, it won’t do that for you (though git will let you do that pretty easily). I think Compose is great for the niches it serves; I have a lot of stuff that still runs on Compose because it doesn’t need any k8s stuff yet. I just don’t think it can be honestly called a k8s replacement. Compose offers equivalent features for some kinds of apps in certain situations, but k8s is a broader tool by far (for better or worse).
        lolinder 3y ago
        Agreed—people on here are only recommending compose because OP only has the single node, which places their app squarely within the realm of what compose is very good at.
- cqqxo4zV46cp 3y ago
  Overkill is an understatement and “industry standard” is a misrepresentation. There are PLENTY of more appropriate options here, for this scale, without going off the beaten track. This is wanting to use the big kids’ tools when it doesn’t make sense to.
- dboreham 3y ago
  Use docker compose or ansible for this scenario.
  - kikimora 3y ago
    If I have to choose I’d rather learn k8s than ansible.
  - SJC_Hacker 3y ago
    Those seem like different things, I thought Ansible was more for configuring individual boxes.. then you would use something like Helm to configure Kubernetes. Tech stack at my last workplace was insane like that. To deploy, they woud have a Buildkite job would launch Terraform, which would then be configured by Ansible, then k8s would deploy on the cluster. I think they could have avoided all this ess k8s and probably Terraform and gotten away with one big box and shell scripts (which they had plenty of anyway), and used swarm/compose, but what do I know.
- lolinder 3y ago
  For a tiny deployment like this, Docker Compose gets you declarative deployments in much the same way with arguably a much smaller learning curve and fewer moving parts. I self-host ~10 services on a single physical server in my home, defined in a series of docker-compose.yaml files stored in a git repo. It's trivial to set up, declarative, and easier to run locally than k8s.
  - silverquiet 3y ago
    It’s a bit weird to me to mention compose and kubernetes without bringing up Docker Swarm which feels like a nice middle ground. We actually run it at my very small SaaS company and it seems like a good fit for us though it’s still relatively early days for us; it’s been a very slow transition.
    - lolinder 3y ago
      I've never used Swarm myself, so I don't recommend it. I've heard several people like you, though, who found it works. Obviously not needed for a single node, though.
- firxworx 3y ago
  A different angle -- Ansible and many others provide declarative deployments too.
  - deathanatos 3y ago
    No, … Ansible is not declarative. Nowhere near the same as a k8s workload. And I know they say that, but it's more or less YAML-as-a-programming language. It is pretty trivial to have side-effects that will persist well beyond and outside an Ansible playbook. It's certainly better than a complete snowflake of a VM, but it's really closer to saying "we init this VM with this shell script."
moomoo11 3y ago
Promotion driven development. Fwiw I’ve deployed services that could handle 30-50k rps on 1vcpu and 1gb ram. Like a couple bucks a month. It used ssd storage and only hit a main rdbms when there was a cache miss. When I hear stories like this I’m reminded of how dumb and inefficient most organizations are. My colleagues would over engineer stuff so much and then leave a huge bill and mess behind. The primary reason was so their clueless eng manager would consider promoting them.
jitl 3y ago
Honestly if I was gonna set up a CI (or some other system that wants to add and remove a lot of containers dynamically) on my home server I might consider adding k3s so it’s easier to install / manage / monitor.
awoimbee 3y ago
For a single node cluster, just use minikube (or RKE2 if you have masochistic tendencies). And if everything runs on a single small vps, why even use postgres and not SQLite.
- jitl 3y ago
  k3s (512mb ram suggested) is smaller than minikube (2gb ram suggested)
windexh8er 3y ago
I've said it a few times before: k8s is such a distraction for so many startups I've been in (~5). All of them fall into the same trap: write entire arch around k8s and then not really invest the engineering into building it out fully. Instead they all seemed to settle on "hey this works" which kicks off the technical debt receipt printing. In all of them fixing things often went like this... * A: XYZ is broke. * B: OK, is the pod running? * A: Yeah. * B: Did you try killing it? That usually fixes the problem. It's madness k8s is generally always the root cause of the problem because of resource constraints, node issues, a misunderstanding of H/V scaling, and the list seems to just go on. At this point I feel like k8s is the c4r (cancer) of a lot of startups that would be better off just running in a more traditional VM / container strategy and dealing with growing pains if and when they become apparent. Also, it's almost guaranteed to cost less. Kaa$ is a racket.
- icedchai 3y ago
  Many startups would also be fine with some serverless container solution (AWS Fargate, Google Cloud Run, etc.) It doesn't fit all applications, but when it does, it's generally less overhead than K8s, both operationally and $ wise.
- scrubs 3y ago
  Let's put k8s aside. If I had a complex system needing say five boxes, I might be able to admin it well - startup, shutdown, upgrade - if there was a simple orchestrated glue between the boxes e.g. on the control box 1, run a script to start daemons/pids on boxes 2, 5 which blocks until success or failure ... something more sophisticated than remsh/ssh. This is old school, but it's just a script or three.

by paxys 3y ago

CactusBlue 3y ago

For me, the main benefit of Kubernetes is declarative deployments. You can have the entire deployment automated through this and GitHub actions, without much fiddling with the production environment manually. I know, it's overkill and I don't like it that much either, but it's the industry standard.

ben_jones 3y ago
> but it's the industry standard. I strongly disagree with this statement. Even at the most bubbly of Silicon Valley events you will not see kubernetes for a smaller deployment be an industry standard. It’s ok to make architectural mistakes, or otherwise over engineer a thing. We all do it. Let’s not standardize it. And to your credit thank you for publishing this example as a learning opportunity to others.
- CactusBlue 3y ago
  I mean as in what the big (and small) clouds support. I haven't seen any clouds supporting Hashicorp Nomad or Docker Swarm!
  - SJC_Hacker 3y ago
    You shoudl be able to run any of those things on any cloud. A box with sshd is a box with sshd.
- jitl 3y ago
  k3s is a teensy sqlite based orchestrator, something tons of devs run on their laptops for local dev. I think those are all single box clusters, why is it crazy to run it on a server but not a laptop?
paxys 3y ago
You can get declarative deployments via a systemd config. If you really want to use Docker then use a Compose file. If jobs are what you are concerned about then I can assure you no company is skipping on software developers because they don't know Kubernetes syntax.
- jitl 3y ago
  imo systemd is more complicated than k3s
- everforward 3y ago
  That doesn't capture the same thing. It doesn't capture volumes, versions of binaries, nginx/postgres/whatever configs, etc. Ansible/Salt/Puppet/Chef/CFEngine are closer comparisons.
  - lolinder 3y ago
    With Docker Compose you can define volumes, versions, and configs can be volumes that point to a file that lives alongside your compose.yaml.
    - everforward 3y ago
      Sure, depending on how you want to draw a line around features. Compose won’t do multi-node orchestration of deploys (I.e. update backends 1-3 sequentially, and revert the change on all of them if any fail to deploy). It also doesn’t manage your configs; ie if you need to roll back, it won’t do that for you (though git will let you do that pretty easily). I think Compose is great for the niches it serves; I have a lot of stuff that still runs on Compose because it doesn’t need any k8s stuff yet. I just don’t think it can be honestly called a k8s replacement. Compose offers equivalent features for some kinds of apps in certain situations, but k8s is a broader tool by far (for better or worse).
      - lolinder 3y ago
        Agreed—people on here are only recommending compose because OP only has the single node, which places their app squarely within the realm of what compose is very good at.
cqqxo4zV46cp 3y ago
Overkill is an understatement and “industry standard” is a misrepresentation. There are PLENTY of more appropriate options here, for this scale, without going off the beaten track. This is wanting to use the big kids’ tools when it doesn’t make sense to.
dboreham 3y ago
Use docker compose or ansible for this scenario.
- kikimora 3y ago
  If I have to choose I’d rather learn k8s than ansible.
- SJC_Hacker 3y ago
  Those seem like different things, I thought Ansible was more for configuring individual boxes.. then you would use something like Helm to configure Kubernetes. Tech stack at my last workplace was insane like that. To deploy, they woud have a Buildkite job would launch Terraform, which would then be configured by Ansible, then k8s would deploy on the cluster. I think they could have avoided all this ess k8s and probably Terraform and gotten away with one big box and shell scripts (which they had plenty of anyway), and used swarm/compose, but what do I know.
lolinder 3y ago
For a tiny deployment like this, Docker Compose gets you declarative deployments in much the same way with arguably a much smaller learning curve and fewer moving parts. I self-host ~10 services on a single physical server in my home, defined in a series of docker-compose.yaml files stored in a git repo. It's trivial to set up, declarative, and easier to run locally than k8s.
- silverquiet 3y ago
  It’s a bit weird to me to mention compose and kubernetes without bringing up Docker Swarm which feels like a nice middle ground. We actually run it at my very small SaaS company and it seems like a good fit for us though it’s still relatively early days for us; it’s been a very slow transition.
  - lolinder 3y ago
    I've never used Swarm myself, so I don't recommend it. I've heard several people like you, though, who found it works. Obviously not needed for a single node, though.
firxworx 3y ago
A different angle -- Ansible and many others provide declarative deployments too.
- deathanatos 3y ago
  No, … Ansible is not declarative. Nowhere near the same as a k8s workload. And I know they say that, but it's more or less YAML-as-a-programming language. It is pretty trivial to have side-effects that will persist well beyond and outside an Ansible playbook. It's certainly better than a complete snowflake of a VM, but it's really closer to saying "we init this VM with this shell script."

moomoo11 3y ago

Promotion driven development. Fwiw I’ve deployed services that could handle 30-50k rps on 1vcpu and 1gb ram. Like a couple bucks a month. It used ssd storage and only hit a main rdbms when there was a cache miss. When I hear stories like this I’m reminded of how dumb and inefficient most organizations are. My colleagues would over engineer stuff so much and then leave a huge bill and mess behind. The primary reason was so their clueless eng manager would consider promoting them.

jitl 3y ago

Honestly if I was gonna set up a CI (or some other system that wants to add and remove a lot of containers dynamically) on my home server I might consider adding k3s so it’s easier to install / manage / monitor.

awoimbee 3y ago

For a single node cluster, just use minikube (or RKE2 if you have masochistic tendencies). And if everything runs on a single small vps, why even use postgres and not SQLite.

jitl 3y ago
k3s (512mb ram suggested) is smaller than minikube (2gb ram suggested)

windexh8er 3y ago

I've said it a few times before: k8s is such a distraction for so many startups I've been in (~5). All of them fall into the same trap: write entire arch around k8s and then not really invest the engineering into building it out fully. Instead they all seemed to settle on "hey this works" which kicks off the technical debt receipt printing. In all of them fixing things often went like this... * A: XYZ is broke. * B: OK, is the pod running? * A: Yeah. * B: Did you try killing it? That usually fixes the problem. It's madness k8s is generally always the root cause of the problem because of resource constraints, node issues, a misunderstanding of H/V scaling, and the list seems to just go on. At this point I feel like k8s is the c4r (cancer) of a lot of startups that would be better off just running in a more traditional VM / container strategy and dealing with growing pains if and when they become apparent. Also, it's almost guaranteed to cost less. Kaa$ is a racket.

icedchai 3y ago
Many startups would also be fine with some serverless container solution (AWS Fargate, Google Cloud Run, etc.) It doesn't fit all applications, but when it does, it's generally less overhead than K8s, both operationally and $ wise.
scrubs 3y ago
Let's put k8s aside. If I had a complex system needing say five boxes, I might be able to admin it well - startup, shutdown, upgrade - if there was a simple orchestrated glue between the boxes e.g. on the control box 1, run a script to start daemons/pids on boxes 2, 5 which blocks until success or failure ... something more sophisticated than remsh/ssh. This is old school, but it's just a script or three.