7 ms·

This is something I see a lot of people ignoring about Kubernetes. There are two completely different sides to it: the developer experience, and the sysadmin/de

This is something I see a lot of people ignoring about Kubernetes. There are two completely different sides to it: the developer experience, and the sysadmin/devops/sre/what-have-you experience. This post completely ignores that second part, which is arguably much harder. You can ignore large swaths of those topics by using managed K8s platforms, but the pricing for those means that running a blog like this might run you $50-$80/mo

tw04 6y ago
Exactly what I was thinking. So you have yaml file that makes it easy to deploy your blog with k8s and you don't understand why people would tell you that's silly. Meanwhile you're spending $50+/month for something you could run on a $2.50/month VM from digital ocean or vultr. And oh by the way, there aren't a dozen moving parts that you don't fully understand that could create security vulnerabilities if misconfigured.
- mrkeen 6y ago
  I'm on a free plan on Google cloud. I'm currently using docker-compose, but I'll probably switch to minikube when I get some free time.
bryanlarsen 6y ago
A single node Kubernetes "cluster" on DigitalOcean is $10/month. Adding a loadbalancer is another $15, but you can avoid that by using a hostNetwork ingress (https://stackoverflow.com/a/60726977/91365 https://stackoverflow.com/a/60726977/91365) Single node Kubernetes is pretty silly on the surface, but it is useful in some cases.
- mamon 6y ago
  Nothing silly about it. Yes, with one node you don't have High Availability, but you get other benefits, like a standardized way to manage configuration.
- khalilravanna 6y ago
  What's the point of a load balancer if you only have one node? Am I dense or is the idea that you'd only use this if you started to have N numbers of nodes sitting behind the load balancer (e.g. $15 + N * $10)?
  - bryanlarsen 6y ago
    A Load Balancer is the standard way to set up an Ingress on Digital Ocean's Kubernetes. (Google's and Amazon's too). This makes sense for the vast majority of clusters, which will have more than one node. As usual, to do things the non-default way takes a little bit more work.
  - threeseed 6y ago
    It can do things like handle SSL and traffic balancing. So if you ever have more than one application e.g. a search engine for that blog then it's much easier to add. SSL in particular is often a real pain to setup and varies in capability wildly between applications.
    - bryanlarsen 6y ago
      For Digital Ocean, the ingress is handling SSL termination, not the load balancer. Ingresses can also do traffic balancing.
throwanem 6y ago
Yeah, I came here to call this out. Speaking from the perspective of having also unironically sought to use Kubernetes for personal projects, the options I found available were these: - Expensive in money: Pay Amazon or Google on the order of $100 a month for reasonably reliable managed k8s, with the option always available of a goofed deployment increasing your bill by an order of magnitude - Expensive in time and money: Set up and run your own k8s cluster, at a monthly VM hosting cost not too far from what you'd pay a managed k8s provider, and spend a ton of time dealing with 100% of the ops burden It's a shame. I use Kubernetes at work and I like it a lot, because it does a great deal to make complex tasks simple. It would be really nice to have that same kind of fungible resource pool available for personal stuff too, and be able to just knock out a few lines of YAML and deploy a new project and have the orchestrator take it from there to running without any further effort on my part. But the economics just don't seem to be there. (I did try DO's managed k8s service, which is considerably cheaper than the big players. It was also very new, and very flaky - not a knock on DO, it's reasonable that a new kind of service would have some teething problems. This was also a year ago, so I wouldn't be surprised if they've gotten it considerably more stable since then.)
- rektide 6y ago
  I spend $6/mo on my singlehnode kubernetes vps and I'd say my ops burden is about zero. it takes about 20 minutes to set up k3s and boom, it's running. since all my projects & manifests are in git, it takes another 40 minutes tops to reapply them on a fresh node, & I don't feel particularly worried about maintaining this one. fear is the mind killer.
  - deleted 6y ago
    [deleted]
  - throwanem 6y ago
    That's really interesting - I was aware of k3s, but hadn't looked closely at it; perhaps I'll give it a try now. Thanks for the info!
- tomphoolery 6y ago
orthoxerox 6y ago
That's exactly what I was thinking. Defining an ingress is simple, setting up an ingress controller less so. Defining a PVC is simple, setting up a storage class and its backing storage less so. Running your app on multiple nodes is simple, setting up these nodes and keeping them up to date and secure is a full time job. Managed k8s or a simple installation in a home lab behind a NAT make it much more tolerable.
- pas 6y ago
  Setting up an Ingress controller is a one liner, no? The problem is setting up the network. MetalLB is nice, but you need a /24 basically. Using klipper-lb (from k3s) would be great, but then you are basically tied to k3s.
turtlebits 6y ago
You can easily run something like K3s on a single node. Works fine and installing is very simple. I have felt no need for a managed platform for simple deployment.
deleted 6y ago
[deleted]

tw04 6y ago

Exactly what I was thinking. So you have yaml file that makes it easy to deploy your blog with k8s and you don't understand why people would tell you that's silly. Meanwhile you're spending $50+/month for something you could run on a $2.50/month VM from digital ocean or vultr. And oh by the way, there aren't a dozen moving parts that you don't fully understand that could create security vulnerabilities if misconfigured.

mrkeen 6y ago
I'm on a free plan on Google cloud. I'm currently using docker-compose, but I'll probably switch to minikube when I get some free time.

bryanlarsen 6y ago

A single node Kubernetes "cluster" on DigitalOcean is $10/month. Adding a loadbalancer is another $15, but you can avoid that by using a hostNetwork ingress (https://stackoverflow.com/a/60726977/91365 https://stackoverflow.com/a/60726977/91365) Single node Kubernetes is pretty silly on the surface, but it is useful in some cases.

mamon 6y ago
Nothing silly about it. Yes, with one node you don't have High Availability, but you get other benefits, like a standardized way to manage configuration.
khalilravanna 6y ago
What's the point of a load balancer if you only have one node? Am I dense or is the idea that you'd only use this if you started to have N numbers of nodes sitting behind the load balancer (e.g. $15 + N * $10)?
- bryanlarsen 6y ago
  A Load Balancer is the standard way to set up an Ingress on Digital Ocean's Kubernetes. (Google's and Amazon's too). This makes sense for the vast majority of clusters, which will have more than one node. As usual, to do things the non-default way takes a little bit more work.
- threeseed 6y ago
  It can do things like handle SSL and traffic balancing. So if you ever have more than one application e.g. a search engine for that blog then it's much easier to add. SSL in particular is often a real pain to setup and varies in capability wildly between applications.
  - bryanlarsen 6y ago
    For Digital Ocean, the ingress is handling SSL termination, not the load balancer. Ingresses can also do traffic balancing.

throwanem 6y ago

Yeah, I came here to call this out. Speaking from the perspective of having also unironically sought to use Kubernetes for personal projects, the options I found available were these: - Expensive in money: Pay Amazon or Google on the order of $100 a month for reasonably reliable managed k8s, with the option always available of a goofed deployment increasing your bill by an order of magnitude - Expensive in time and money: Set up and run your own k8s cluster, at a monthly VM hosting cost not too far from what you'd pay a managed k8s provider, and spend a ton of time dealing with 100% of the ops burden It's a shame. I use Kubernetes at work and I like it a lot, because it does a great deal to make complex tasks simple. It would be really nice to have that same kind of fungible resource pool available for personal stuff too, and be able to just knock out a few lines of YAML and deploy a new project and have the orchestrator take it from there to running without any further effort on my part. But the economics just don't seem to be there. (I did try DO's managed k8s service, which is considerably cheaper than the big players. It was also very new, and very flaky - not a knock on DO, it's reasonable that a new kind of service would have some teething problems. This was also a year ago, so I wouldn't be surprised if they've gotten it considerably more stable since then.)

rektide 6y ago
I spend $6/mo on my singlehnode kubernetes vps and I'd say my ops burden is about zero. it takes about 20 minutes to set up k3s and boom, it's running. since all my projects & manifests are in git, it takes another 40 minutes tops to reapply them on a fresh node, & I don't feel particularly worried about maintaining this one. fear is the mind killer.
- deleted 6y ago
  [deleted]
- throwanem 6y ago
  That's really interesting - I was aware of k3s, but hadn't looked closely at it; perhaps I'll give it a try now. Thanks for the info!
tomphoolery 6y ago

orthoxerox 6y ago

That's exactly what I was thinking. Defining an ingress is simple, setting up an ingress controller less so. Defining a PVC is simple, setting up a storage class and its backing storage less so. Running your app on multiple nodes is simple, setting up these nodes and keeping them up to date and secure is a full time job. Managed k8s or a simple installation in a home lab behind a NAT make it much more tolerable.

pas 6y ago
Setting up an Ingress controller is a one liner, no? The problem is setting up the network. MetalLB is nice, but you need a /24 basically. Using klipper-lb (from k3s) would be great, but then you are basically tied to k3s.

turtlebits 6y ago

You can easily run something like K3s on a single node. Works fine and installing is very simple. I have felt no need for a managed platform for simple deployment.

deleted 6y ago

[deleted]