3 ms·

Yeah, I've been learning Kubernetes recently and a simple setup isn't too difficult. However, there's a lot of concepts involved with Kubernetes to grasp and I'

by Hamuko 6y ago

Yeah, I've been learning Kubernetes recently and a simple setup isn't too difficult. However, there's a lot of concepts involved with Kubernetes to grasp and I've definitely not learned everything there is. For example, Jobs still baffle me in regards to what is the purpose outside of CronJobs.

mpb88 6y ago
I use Jobs for helm hooks like integration testing and DB migrations associated with deployments. They work great for one off workloads like that.
- thedevelopnik 6y ago
  You just beat me to it!
thedevelopnik 6y ago
Jobs can get useful in more complex workflows. The most common use I’ve had for them us running a smoke test suite or a database migration as part of the larger workflow for a big Rails app.
- Hamuko 6y ago
  But why wouldn't I just run a database migration as part of my CI flow? I don't understand why I would add a persistent element to my Kubernetes cluster for just a one-off task, since Jobs do not actually automatically remove themselves from the cluster. And as far as I know, you can't even reuse the already created Job, so if I want to migrate the database again, I need to make it from scratch.
  - kube-system 6y ago
    Well, CI doesn't necessarily mean CD. In more than a few cases, I've found it advantageous to use CI to do automatic container builds, but manually attend to the deployment upgrades, particularly when things like db migrations are happening. > Jobs do not actually automatically remove themselves from the cluster. set `ttlSecondsAfterFinished` and they will > And as far as I know, you can't even reuse the already created Job, so if I want to migrate the database again, I need to make it from scratch. If you need to run a job on demand, you can always 'kubectl apply -f' the metadata, or you can create a cronjob and `kubectl create job --from` your cronjob. Or, if it needs to run at a particular part of your installation/upgrade workflow, use helm hooks. Jobs are intended to be a one-time thing, so the lack of functionality to repeatedly run them is intentional.
  - uberduper 6y ago
    There's a TTLAfterFinished feature you can enable since 1.12. It's been in alpha forever. :/ Lets you add a ttl to your job spec to automatically clean up completed jobs.
kodeninja 6y ago
I developed a simple wrapper around vegeta, called vajra, that can be given the URL to a file containing test URLs (in vegeta format), a target QPS/duration, and then easily run a load test. Gets deployed as a k8s `Job` via GitLab (can be scheduled/on-demand), with a simple script echoing back the vegeta status every 1 minute. Jobs don't yet support automatic cleanup, so another simple GitLab job deletes the job from k8s upon completion. The execution log is anyway available in GitLab/Kibana. All in all, an extremely simple way to run load tests against a service deployed in k8s. What grew as a personal utility is now being used by many teams for quick load tests.
atomicity 6y ago
The term "batch job" (as opposed to "service") appears frequently in the Google K8s-like scheduler papers: https://research.google/pubs/pub41684/ https://research.google/pubs/pub41684/ https://research.google/pubs/pub43438/ https://research.google/pubs/pub43438/ In other words, jobs seem like the go-to way to take advantage of unused CPU/memory.

by Hamuko 6y ago

mpb88 6y ago

I use Jobs for helm hooks like integration testing and DB migrations associated with deployments. They work great for one off workloads like that.

thedevelopnik 6y ago
You just beat me to it!

thedevelopnik 6y ago

Jobs can get useful in more complex workflows. The most common use I’ve had for them us running a smoke test suite or a database migration as part of the larger workflow for a big Rails app.

Hamuko 6y ago
But why wouldn't I just run a database migration as part of my CI flow? I don't understand why I would add a persistent element to my Kubernetes cluster for just a one-off task, since Jobs do not actually automatically remove themselves from the cluster. And as far as I know, you can't even reuse the already created Job, so if I want to migrate the database again, I need to make it from scratch.
- kube-system 6y ago
  Well, CI doesn't necessarily mean CD. In more than a few cases, I've found it advantageous to use CI to do automatic container builds, but manually attend to the deployment upgrades, particularly when things like db migrations are happening. > Jobs do not actually automatically remove themselves from the cluster. set `ttlSecondsAfterFinished` and they will > And as far as I know, you can't even reuse the already created Job, so if I want to migrate the database again, I need to make it from scratch. If you need to run a job on demand, you can always 'kubectl apply -f' the metadata, or you can create a cronjob and `kubectl create job --from` your cronjob. Or, if it needs to run at a particular part of your installation/upgrade workflow, use helm hooks. Jobs are intended to be a one-time thing, so the lack of functionality to repeatedly run them is intentional.
- uberduper 6y ago
  There's a TTLAfterFinished feature you can enable since 1.12. It's been in alpha forever. :/ Lets you add a ttl to your job spec to automatically clean up completed jobs.

kodeninja 6y ago

I developed a simple wrapper around vegeta, called vajra, that can be given the URL to a file containing test URLs (in vegeta format), a target QPS/duration, and then easily run a load test. Gets deployed as a k8s `Job` via GitLab (can be scheduled/on-demand), with a simple script echoing back the vegeta status every 1 minute. Jobs don't yet support automatic cleanup, so another simple GitLab job deletes the job from k8s upon completion. The execution log is anyway available in GitLab/Kibana. All in all, an extremely simple way to run load tests against a service deployed in k8s. What grew as a personal utility is now being used by many teams for quick load tests.

atomicity 6y ago

The term "batch job" (as opposed to "service") appears frequently in the Google K8s-like scheduler papers: https://research.google/pubs/pub41684/ https://research.google/pubs/pub41684/ https://research.google/pubs/pub43438/ https://research.google/pubs/pub43438/ In other words, jobs seem like the go-to way to take advantage of unused CPU/memory.