6 ms·

Kubernetes jobs (https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/ https://kubernetes.io/docs/concepts/workloads/controllers/jo.

Kubernetes jobs (https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/ https://kubernetes.io/docs/concepts/workloads/controllers/jo...) and kubernetes cron jobs (https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/ https://kubernetes.io/docs/concepts/workloads/controllers/cr...) make a task like running a fairly robust FaaS infrastructure (this involves things like loadbalancing jobs, tracking state, etc) much simpler -- really excited for a world where people are empowered to create their own clouds rather than get on someone else's. Personal Anecdote: I switched all my personal project infrastructure to use Kubernetes (granted, it's a single node cluster), and went through the process of setting it up myself (no tectonic). The process was difficult but immensely rewarding (I have a super long blog post series incoming about it), and various bits of documentation helped me every step of the way. For the curious, my latest ops iteration was docker-containers-managed-by-systemd-provisioned-by-ansible setup to docker-containers-managed-by-kubernetes-provisioned-by-ansible-on-coreos. I say all that to say that after having personal experience both good and bad with kubernetes, I am thoroughly impressed with it, and have become a huge fan. It is internally consistent with it's principles and concepts, and once you have things set up they mostly just work. When things go wrong (as they do inevitably), the concepts are useful tools and generally hold up, and it's often just a case of finding examples, and giving the documentation a more thorough reading. You can tell when you've learned a truly significant piece of knowledge if you can't stop thinking of all the ways you can apply it. I've had that experience with kubernetes -- just wanted to share, if anyone was on the fence about playing with it.

alexellisuk 9y ago
OpenFaaS is an "open platform" - supporting Docker Swarm, Kubernetes and whatever else you would like to write a back-end for. Kubernetes jobs are a potential back-end for asynchronous processing of functions (only). Right now we're achieving that through a NATS Streaming integration - which is agnostic of the orchestration platform. Checkout the video walk-through for OpenFaaS on K8s.
philjones88 9y ago
Could I get a link to that blog of yours?
- hardwaresofton 9y ago
  Sorry about that, here's the post on ansible -- https://vadosware.io/post/ansible-is-awesome/ https://vadosware.io/post/ansible-is-awesome/ I switched to that infrastructure (I describe the preceeding Makefile+SSH setup a little bit in the post). The kubernetes posts are pretty much next in the pipe, since people have asked I'm going to probably start translating it this week (I wrote rough notes as I went, there are a LOT of notes).
lionyo 9y ago
I'm in the process of doing this with Rancher -- is it worthwhile to use both to see the difference, or go with Kubernetes?
- icebraining 9y ago
  Rancher actually uses Kubernetes, they're slightly different tools.
  - seabrookmx 9y ago
    CAN use Kubernetes. They support Cattle (Rancher's own), Kubernetes, or Swarm for orchestration. I talked with some of the folks that work at Rancher at the last Dockercon and IIRC they even had some rudimentary Mesos support.
- alexellisuk 9y ago
  Process of doing which part? An OpenFaaS backend?
- hardwaresofton 9y ago
  Hey so I actually went through that exact same decision. So "Rancher" means one of at least two things (I'm not an expert but just tried really hard to RTFM) -- RancherOS or Rancher the container platform management system -- for those who may not be familiar. As another person noted Rancher sits at a level of abstraction ABOVE kubernetes -- you can set up Rancher to work with Docker Swarm, Mesos, Kubernetes, and Rancher's own libraries. Since this was how I understood it, I went with just vanilla Kubernetes first, and figured that once I got really comfortable with that, I could throw the Rancher UI on top of it (the install documentation is actually super duper short, it's just another container) when I got tired of typing things in at the console. I didn't opt to go with RancherOS because CoreOS was already a step away from what I knew -- most of my machines are either arch, alpine (inside a container) or debian/ubuntu. RancherOS takes the stripped-down OS paradigm even further by dockerizing a bunch of system services -- I wasn't quite ready for that. CoreOS not coming with a package manager is already jarring enough for me (I'm used to it a little bit more now) -- I wanted to take a small step rather than a huge one.
kuschku 9y ago
> Personal Anecdote: I switched all my personal project infrastructure to use Kubernetes (granted, it's a single node cluster), and went through the process of setting it up myself (no tectonic). The process was difficult but immensely rewarding (I have a super long blog post series incoming about it), and various bits of documentation helped me every step of the way. For the curious, my latest ops iteration was docker-containers-managed-by-systemd-provisioned-by-ansible setup to docker-containers-managed-by-kubernetes-provisioned-by-ansible-on-coreos. I’ve done something very similar myself, and you’re exactly right. The effort to set up a new project, or tear it down, is basically zero. I can properly version my entire system via git, by just exporting the configs as json, putting them in a git repo, and importing them back if I want to. I honestly recommend everyone to at least try it out, it’s immensely useful (and a good skill to have)
- hardwaresofton 9y ago
  Yup -- whole heartedly agree. It's so easy that at this point, most of my worry is wondering how I would deal with a disaster scenario, whether the server getting wiped or any other kind of catastrophic failure -- is my automation good enough that I can go from bare-machine to all my projects fully running? One thing I was a little disappointed with was the ignition provisioning system (https://coreos.com/ignition/docs/latest/ https://coreos.com/ignition/docs/latest/) -- it's revolutionary (I mean not really, but it is to me) in the amount of the setting-up-the-server routine that you can skip, but I was disappointed that you can only run it once (by design, that's exactly what it's supposed to do, set up the machine ONCE). I currently manage one larger dedicated host that is more pet than cattle, but if I was truly in someone's cloud spinning up instances, then I could take time to build my perfect ignition config, but I only got one shot (and it worked the first time so), and didn't want to wipe the harddrive.
  - kuschku 9y ago
    > Yup -- whole heartedly agree. It's so easy that at this point, most of my worry is wondering how I would deal with a disaster scenario, whether the server getting wiped or any other kind of catastrophic failure -- is my automation good enough that I can go from bare-machine to all my projects fully running? I’m running my system currently on Ubuntu (I’d have to pay extra for additional iSCSI drives), and I had to recreate the system hundreds of times due to mistakes. So I built an automated system to do that, and tested it hundreds of times. So I’m at least sure that mine works.
brazzledazzle 9y ago
Would you mind providing a link to your blog? Definitely interested in reading your post when you publish it.
- hardwaresofton 9y ago
  Sorry I didn't do the usual "link in profile thing", but here's a link to my blog post on the current setup (ansible): https://vadosware.io/post/ansible-is-awesome/ https://vadosware.io/post/ansible-is-awesome/. That post in particular is a bit exuberant, but I did really like ansible that much when I started using it -- it's so rare to have a tool that works just at the abstraction level you want and with most things just working. I'm literally about to head on a vacation which should really give me more time to write, and the kubernetes posts are next (there is one post in between on building a static haskell executable but I might skip it since the kubernetes stuff is so long). I wrote them as I was going through it so they're messy, and tectonic (terraform) or `kubeadm` are the current "blessed" way to do it on most systems but I enjoyed setting it up myself.
  - brazzledazzle 9y ago
    Awesome. Thanks for sharing.
mewm 9y ago
I would very much appreciate a link to your blog as well
- hardwaresofton 9y ago
  Here's the post on the ansible setup -- it's mostly ramblings but hope it's worth a read, I think the most interesting part is the part where I break down what I think the "levels" of devops are (what I call "operational intelligence", in an attempt to make a new garbage-tier buzzword): https://vadosware.io/post/ansible-is-awesome/ https://vadosware.io/post/ansible-is-awesome/
hardwaresofton 9y ago
Just to add more to the post -- there are lots of options outside of SSH-to-the-box and kubernetes out there, you don't have to go to the new hotness quite so quick. I heavily considered: Dokku (https://github.com/dokku/dokku https://github.com/dokku/dokku) Flynn (https://flynn.io/ https://flynn.io/) I didn't discuss them in the blog post I linked everywhere (I should have), but they're a great option for no-worries deployment, and simpler to set up than kubernetes. I didn't ultimately use them because I felt even messing with copying code to a server wasn't the way forward -- I wanted to just pass a container, and be done, not even adhere to some heroku buildpack (even though they're pretty darn convenient) or anything, just push a container up and things are in production.
pstadler 9y ago
That's exactly how I set up my own cluster running outside of well integrated and isolated platforms (AWS, GCE, you name it). The experiences gathered during this process are available in the shape of a guide, including Terraform modules for fully automated provisioning: https://github.com/hobby-kube/guide https://github.com/hobby-kube/guide
ksajadi 9y ago
Having tried both Kubernetes and Docker Swarm, I have to say Kubernetes has a much steeper learning curve but is a much more stable and consistent tool that focuses on its set boundaries. Mesos is sometimes referred to as an alternative, but in reality, it's more of a lower level OS for the server workload than a container native orchestration utility. I think using Kubernetes as a backend for any FaaS (and in particular OpenFaaS) is a great idea. Can't wait to try it out. We use Cloud 66 to setup Kubernetes clusters and deploy our stacks to them: https://www.cloud66.com/containers https://www.cloud66.com/containers
- alexellisuk 9y ago
  I agree that there's a steeper curve with K8s - I have a short blog series on it - http://blog.alexellis.io/tag/k8s/ http://blog.alexellis.io/tag/k8s/

alexellisuk 9y ago

OpenFaaS is an "open platform" - supporting Docker Swarm, Kubernetes and whatever else you would like to write a back-end for. Kubernetes jobs are a potential back-end for asynchronous processing of functions (only). Right now we're achieving that through a NATS Streaming integration - which is agnostic of the orchestration platform. Checkout the video walk-through for OpenFaaS on K8s.

philjones88 9y ago

Could I get a link to that blog of yours?

hardwaresofton 9y ago
Sorry about that, here's the post on ansible -- https://vadosware.io/post/ansible-is-awesome/ https://vadosware.io/post/ansible-is-awesome/ I switched to that infrastructure (I describe the preceeding Makefile+SSH setup a little bit in the post). The kubernetes posts are pretty much next in the pipe, since people have asked I'm going to probably start translating it this week (I wrote rough notes as I went, there are a LOT of notes).

lionyo 9y ago

I'm in the process of doing this with Rancher -- is it worthwhile to use both to see the difference, or go with Kubernetes?

icebraining 9y ago
Rancher actually uses Kubernetes, they're slightly different tools.
- seabrookmx 9y ago
  CAN use Kubernetes. They support Cattle (Rancher's own), Kubernetes, or Swarm for orchestration. I talked with some of the folks that work at Rancher at the last Dockercon and IIRC they even had some rudimentary Mesos support.
alexellisuk 9y ago
Process of doing which part? An OpenFaaS backend?
hardwaresofton 9y ago
Hey so I actually went through that exact same decision. So "Rancher" means one of at least two things (I'm not an expert but just tried really hard to RTFM) -- RancherOS or Rancher the container platform management system -- for those who may not be familiar. As another person noted Rancher sits at a level of abstraction ABOVE kubernetes -- you can set up Rancher to work with Docker Swarm, Mesos, Kubernetes, and Rancher's own libraries. Since this was how I understood it, I went with just vanilla Kubernetes first, and figured that once I got really comfortable with that, I could throw the Rancher UI on top of it (the install documentation is actually super duper short, it's just another container) when I got tired of typing things in at the console. I didn't opt to go with RancherOS because CoreOS was already a step away from what I knew -- most of my machines are either arch, alpine (inside a container) or debian/ubuntu. RancherOS takes the stripped-down OS paradigm even further by dockerizing a bunch of system services -- I wasn't quite ready for that. CoreOS not coming with a package manager is already jarring enough for me (I'm used to it a little bit more now) -- I wanted to take a small step rather than a huge one.

kuschku 9y ago

> Personal Anecdote: I switched all my personal project infrastructure to use Kubernetes (granted, it's a single node cluster), and went through the process of setting it up myself (no tectonic). The process was difficult but immensely rewarding (I have a super long blog post series incoming about it), and various bits of documentation helped me every step of the way. For the curious, my latest ops iteration was docker-containers-managed-by-systemd-provisioned-by-ansible setup to docker-containers-managed-by-kubernetes-provisioned-by-ansible-on-coreos. I’ve done something very similar myself, and you’re exactly right. The effort to set up a new project, or tear it down, is basically zero. I can properly version my entire system via git, by just exporting the configs as json, putting them in a git repo, and importing them back if I want to. I honestly recommend everyone to at least try it out, it’s immensely useful (and a good skill to have)

hardwaresofton 9y ago
Yup -- whole heartedly agree. It's so easy that at this point, most of my worry is wondering how I would deal with a disaster scenario, whether the server getting wiped or any other kind of catastrophic failure -- is my automation good enough that I can go from bare-machine to all my projects fully running? One thing I was a little disappointed with was the ignition provisioning system (https://coreos.com/ignition/docs/latest/ https://coreos.com/ignition/docs/latest/) -- it's revolutionary (I mean not really, but it is to me) in the amount of the setting-up-the-server routine that you can skip, but I was disappointed that you can only run it once (by design, that's exactly what it's supposed to do, set up the machine ONCE). I currently manage one larger dedicated host that is more pet than cattle, but if I was truly in someone's cloud spinning up instances, then I could take time to build my perfect ignition config, but I only got one shot (and it worked the first time so), and didn't want to wipe the harddrive.
- kuschku 9y ago
  > Yup -- whole heartedly agree. It's so easy that at this point, most of my worry is wondering how I would deal with a disaster scenario, whether the server getting wiped or any other kind of catastrophic failure -- is my automation good enough that I can go from bare-machine to all my projects fully running? I’m running my system currently on Ubuntu (I’d have to pay extra for additional iSCSI drives), and I had to recreate the system hundreds of times due to mistakes. So I built an automated system to do that, and tested it hundreds of times. So I’m at least sure that mine works.

brazzledazzle 9y ago

Would you mind providing a link to your blog? Definitely interested in reading your post when you publish it.

hardwaresofton 9y ago
Sorry I didn't do the usual "link in profile thing", but here's a link to my blog post on the current setup (ansible): https://vadosware.io/post/ansible-is-awesome/ https://vadosware.io/post/ansible-is-awesome/. That post in particular is a bit exuberant, but I did really like ansible that much when I started using it -- it's so rare to have a tool that works just at the abstraction level you want and with most things just working. I'm literally about to head on a vacation which should really give me more time to write, and the kubernetes posts are next (there is one post in between on building a static haskell executable but I might skip it since the kubernetes stuff is so long). I wrote them as I was going through it so they're messy, and tectonic (terraform) or `kubeadm` are the current "blessed" way to do it on most systems but I enjoyed setting it up myself.
- brazzledazzle 9y ago
  Awesome. Thanks for sharing.

mewm 9y ago

I would very much appreciate a link to your blog as well

hardwaresofton 9y ago
Here's the post on the ansible setup -- it's mostly ramblings but hope it's worth a read, I think the most interesting part is the part where I break down what I think the "levels" of devops are (what I call "operational intelligence", in an attempt to make a new garbage-tier buzzword): https://vadosware.io/post/ansible-is-awesome/ https://vadosware.io/post/ansible-is-awesome/

hardwaresofton 9y ago

Just to add more to the post -- there are lots of options outside of SSH-to-the-box and kubernetes out there, you don't have to go to the new hotness quite so quick. I heavily considered: Dokku (https://github.com/dokku/dokku https://github.com/dokku/dokku) Flynn (https://flynn.io/ https://flynn.io/) I didn't discuss them in the blog post I linked everywhere (I should have), but they're a great option for no-worries deployment, and simpler to set up than kubernetes. I didn't ultimately use them because I felt even messing with copying code to a server wasn't the way forward -- I wanted to just pass a container, and be done, not even adhere to some heroku buildpack (even though they're pretty darn convenient) or anything, just push a container up and things are in production.

pstadler 9y ago

That's exactly how I set up my own cluster running outside of well integrated and isolated platforms (AWS, GCE, you name it). The experiences gathered during this process are available in the shape of a guide, including Terraform modules for fully automated provisioning: https://github.com/hobby-kube/guide https://github.com/hobby-kube/guide

ksajadi 9y ago

Having tried both Kubernetes and Docker Swarm, I have to say Kubernetes has a much steeper learning curve but is a much more stable and consistent tool that focuses on its set boundaries. Mesos is sometimes referred to as an alternative, but in reality, it's more of a lower level OS for the server workload than a container native orchestration utility. I think using Kubernetes as a backend for any FaaS (and in particular OpenFaaS) is a great idea. Can't wait to try it out. We use Cloud 66 to setup Kubernetes clusters and deploy our stacks to them: https://www.cloud66.com/containers https://www.cloud66.com/containers

alexellisuk 9y ago
I agree that there's a steeper curve with K8s - I have a short blog series on it - http://blog.alexellis.io/tag/k8s/ http://blog.alexellis.io/tag/k8s/