5 ms·

Kubernetes is incredibly daunting for someone who only used VMs, but after spending some time with it, it's not really that hard. I also unironically run my per

Kubernetes is incredibly daunting for someone who only used VMs, but after spending some time with it, it's not really that hard. I also unironically run my personal stuff on K8s just because it's familiar to me, and it is also a good exercise for a very useful skill. People complain about the price, but you can easily get a 3-node cluster on GCP for less than $15/month.

Hamuko 6y ago
Yeah, I've been learning Kubernetes recently and a simple setup isn't too difficult. However, there's a lot of concepts involved with Kubernetes to grasp and I've definitely not learned everything there is. For example, Jobs still baffle me in regards to what is the purpose outside of CronJobs.
- mpb88 6y ago
  I use Jobs for helm hooks like integration testing and DB migrations associated with deployments. They work great for one off workloads like that.
  - thedevelopnik 6y ago
    You just beat me to it!
- thedevelopnik 6y ago
  Jobs can get useful in more complex workflows. The most common use I’ve had for them us running a smoke test suite or a database migration as part of the larger workflow for a big Rails app.
  - Hamuko 6y ago
    But why wouldn't I just run a database migration as part of my CI flow? I don't understand why I would add a persistent element to my Kubernetes cluster for just a one-off task, since Jobs do not actually automatically remove themselves from the cluster. And as far as I know, you can't even reuse the already created Job, so if I want to migrate the database again, I need to make it from scratch.
    - kube-system 6y ago
      Well, CI doesn't necessarily mean CD. In more than a few cases, I've found it advantageous to use CI to do automatic container builds, but manually attend to the deployment upgrades, particularly when things like db migrations are happening. > Jobs do not actually automatically remove themselves from the cluster. set `ttlSecondsAfterFinished` and they will > And as far as I know, you can't even reuse the already created Job, so if I want to migrate the database again, I need to make it from scratch. If you need to run a job on demand, you can always 'kubectl apply -f' the metadata, or you can create a cronjob and `kubectl create job --from` your cronjob. Or, if it needs to run at a particular part of your installation/upgrade workflow, use helm hooks. Jobs are intended to be a one-time thing, so the lack of functionality to repeatedly run them is intentional.
    - uberduper 6y ago
      There's a TTLAfterFinished feature you can enable since 1.12. It's been in alpha forever. :/ Lets you add a ttl to your job spec to automatically clean up completed jobs.
- kodeninja 6y ago
  I developed a simple wrapper around vegeta, called vajra, that can be given the URL to a file containing test URLs (in vegeta format), a target QPS/duration, and then easily run a load test. Gets deployed as a k8s `Job` via GitLab (can be scheduled/on-demand), with a simple script echoing back the vegeta status every 1 minute. Jobs don't yet support automatic cleanup, so another simple GitLab job deletes the job from k8s upon completion. The execution log is anyway available in GitLab/Kibana. All in all, an extremely simple way to run load tests against a service deployed in k8s. What grew as a personal utility is now being used by many teams for quick load tests.
- atomicity 6y ago
  The term "batch job" (as opposed to "service") appears frequently in the Google K8s-like scheduler papers: https://research.google/pubs/pub41684/ https://research.google/pubs/pub41684/ https://research.google/pubs/pub43438/ https://research.google/pubs/pub43438/ In other words, jobs seem like the go-to way to take advantage of unused CPU/memory.
arcticfox 6y ago
One more unironic personal user here - I use k3s for cheap-as-possible.
- KronisLV 6y ago
  I actually compared K3s with Docker Swarm in my master's thesis where i also wrote a simply Python tool to automate setting up servers for deploying containerized workloads on then. While i can't really share the text (because it's all in Latvian), i did find that K3s is a pretty good option for when you want to use Kubernetes, but have pretty limited hardware resources. So much so, actually, that it used only slightly more resources than Docker Swarm (and, as a consequence, the deployments that were running on the nodes could serve almost equal amounts of requests under load). Personally, i do believe that Kubernetes is sometimes overrated for simpler deployments, but if people don't mind the manifest syntax and are comfortable with the tool and its ecosystem, i think that K3s is probably the way to go in those situations (maybe with something like Portainer/dashboard running for graphical administration, if necessary). Such a good distribution, could probably even run it in prod with a HA setup.
jhanschoo 6y ago
> but you can easily get a 3-node cluster on GCP for less than $15/month. Just wondering, how are you exposing your services from the cluster? (I assume you aren't using a load-balancer ingress)

Hamuko 6y ago

Yeah, I've been learning Kubernetes recently and a simple setup isn't too difficult. However, there's a lot of concepts involved with Kubernetes to grasp and I've definitely not learned everything there is. For example, Jobs still baffle me in regards to what is the purpose outside of CronJobs.

mpb88 6y ago
I use Jobs for helm hooks like integration testing and DB migrations associated with deployments. They work great for one off workloads like that.
- thedevelopnik 6y ago
  You just beat me to it!
thedevelopnik 6y ago
Jobs can get useful in more complex workflows. The most common use I’ve had for them us running a smoke test suite or a database migration as part of the larger workflow for a big Rails app.
- Hamuko 6y ago
  But why wouldn't I just run a database migration as part of my CI flow? I don't understand why I would add a persistent element to my Kubernetes cluster for just a one-off task, since Jobs do not actually automatically remove themselves from the cluster. And as far as I know, you can't even reuse the already created Job, so if I want to migrate the database again, I need to make it from scratch.
  - kube-system 6y ago
    Well, CI doesn't necessarily mean CD. In more than a few cases, I've found it advantageous to use CI to do automatic container builds, but manually attend to the deployment upgrades, particularly when things like db migrations are happening. > Jobs do not actually automatically remove themselves from the cluster. set `ttlSecondsAfterFinished` and they will > And as far as I know, you can't even reuse the already created Job, so if I want to migrate the database again, I need to make it from scratch. If you need to run a job on demand, you can always 'kubectl apply -f' the metadata, or you can create a cronjob and `kubectl create job --from` your cronjob. Or, if it needs to run at a particular part of your installation/upgrade workflow, use helm hooks. Jobs are intended to be a one-time thing, so the lack of functionality to repeatedly run them is intentional.
  - uberduper 6y ago
    There's a TTLAfterFinished feature you can enable since 1.12. It's been in alpha forever. :/ Lets you add a ttl to your job spec to automatically clean up completed jobs.
kodeninja 6y ago
I developed a simple wrapper around vegeta, called vajra, that can be given the URL to a file containing test URLs (in vegeta format), a target QPS/duration, and then easily run a load test. Gets deployed as a k8s `Job` via GitLab (can be scheduled/on-demand), with a simple script echoing back the vegeta status every 1 minute. Jobs don't yet support automatic cleanup, so another simple GitLab job deletes the job from k8s upon completion. The execution log is anyway available in GitLab/Kibana. All in all, an extremely simple way to run load tests against a service deployed in k8s. What grew as a personal utility is now being used by many teams for quick load tests.
atomicity 6y ago
The term "batch job" (as opposed to "service") appears frequently in the Google K8s-like scheduler papers: https://research.google/pubs/pub41684/ https://research.google/pubs/pub41684/ https://research.google/pubs/pub43438/ https://research.google/pubs/pub43438/ In other words, jobs seem like the go-to way to take advantage of unused CPU/memory.

arcticfox 6y ago

One more unironic personal user here - I use k3s for cheap-as-possible.

KronisLV 6y ago
I actually compared K3s with Docker Swarm in my master's thesis where i also wrote a simply Python tool to automate setting up servers for deploying containerized workloads on then. While i can't really share the text (because it's all in Latvian), i did find that K3s is a pretty good option for when you want to use Kubernetes, but have pretty limited hardware resources. So much so, actually, that it used only slightly more resources than Docker Swarm (and, as a consequence, the deployments that were running on the nodes could serve almost equal amounts of requests under load). Personally, i do believe that Kubernetes is sometimes overrated for simpler deployments, but if people don't mind the manifest syntax and are comfortable with the tool and its ecosystem, i think that K3s is probably the way to go in those situations (maybe with something like Portainer/dashboard running for graphical administration, if necessary). Such a good distribution, could probably even run it in prod with a HA setup.

jhanschoo 6y ago

> but you can easily get a 3-node cluster on GCP for less than $15/month. Just wondering, how are you exposing your services from the cluster? (I assume you aren't using a load-balancer ingress)