6 ms·

I've given up running my own k8s at home a couple of times. It does seem to be getting easier, but then upgrading and maintaining breaks me again. + the other

by rb808 6y ago

I've given up running my own k8s at home a couple of times. It does seem to be getting easier, but then upgrading and maintaining breaks me again. + the other stuff like running your own container registry. Then with new centos podman breaks everything that worked OK in docker. I hate this stuff, so many problems its worth ignoring the whole stack.

StreamBright 6y ago
I do not blame you. I try to avoid it as long as it is possible. The amount of time, energy and effort wasted on this is just insane.
- brianwawok 6y ago
  Hosted k8s has changed my life. Let me focus on my business and not my infra.
  - StreamBright 6y ago
    Have you had any serious yet? If you are on the happy path does not really matter what you use.
    - brianwawok 6y ago
      Any serious what? I am running a mildly successful SaaS business on hosted kubernetes, yes.
      - StreamBright 6y ago
        Sorry, HN made me retry couple of times. Any serious troubles with Kubernetes. Have you had any of those?
        brianwawok 6y ago
        No I have not. Going on 3 years now?
        StreamBright 6y ago
        Great. So you do not have any experience with the worst-case scenario. That is exactly my point. I would like to hear your opinion once you fixed an outage that is caused by K8s.
        brianwawok 6y ago
        Well what is this outage caused by? Not understanding how k8s work, or actual provider screwing up? if the former - I try to read carefully and test my workloads in a test cluster before I do it in prod. if the latter, I think google has excellent operational excellence. This is like saying you wouldn't take driving advice from me until I total a few cars. Perhaps I didn't total a few cars because I make good driving decisions?
        StreamBright 6y ago
        An outage that is caused by k8s. You can pick a random from this list. https://github.com/hjacobs/kubernetes-failure-stories https://github.com/hjacobs/kubernetes-failure-stories >>This is like saying you wouldn't take driving advice from me until I total a few cars No, I am saying you do not have enough experience with k8s. That is all. Totaling a car implying you are driving, using k8s implying that whoever wrote that particular part of k8s is driving. You have some control but ultimately you need to understand what is going on. As that list shows many companies who put k8s into production learned it in a hard way.
        brianwawok 6y ago
        Sorry I’m not following Should I compile a list of failure stores for companies that ran Linux because Linux is complicated or has had various bugs? How about a list of failure stories of how Ruby causes problems? You can get errors with any system. You would need to prove to me that k8s has say more errors per day than a competitor (lambda? VMs). I just don’t understand...
  - rb808 6y ago
    expensive though?
    - brianwawok 6y ago
      Not really, total cloud costs are under 5% of my revenue. If I want to shave pennies, I have many places to shave before the very thing that is the heart of my business.
tuananh 6y ago
what do you end up using instead of k8s?
- StavrosK 6y ago
  docker run -d --restart=unless-stopped
- 0xEFF 6y ago
  Plain old systemd unit files work great. If you really need containers for some reason and love UID and GID mapping then podman in systemd is great. The only reason I’ve found to run k8s at home is if your full time job is operating k8s at work.
- Bombthecat 6y ago
  I mostly use docker compose. Perfect fit for tinkering but not over complicated and huge ram eater. Also:k3s might be an option.
  - bradgessler 6y ago
    Compose is great for home use. If you run your home server on a separate box, like a RaspPi or NUC, docker-machine via SSH can make provisioning and updating a tad easier.
- dkryptr 6y ago
  I've used Nomad by HashiCorp and it's been really nice to use. Though I haven't needed to scale it up a ton so I can't comment on that, but the day-to-day deploying of updated apps is nice and works well.
- castillar76 6y ago
  I've plugged it before, but Portainer (portainer.io) is what finally got me into running containers for apps. It presents a really nice interface for running individual containers or stacks including docker-compose files, and it extends the nerd knobs for doing things like custom networking or filesystem mounts. I paired it with Ouroboros for doing automated container upgrades in the background, and the result is that I have a solid set of container apps running at all times that I never have to touch. Were I learning k8s for work, it wouldn't be a good solution, but for simple home apps that you want to fire and forget, it definitely beats doing a hand-made installation and periodically care-and-feeding them.

by rb808 6y ago

StreamBright 6y ago

I do not blame you. I try to avoid it as long as it is possible. The amount of time, energy and effort wasted on this is just insane.

brianwawok 6y ago
Hosted k8s has changed my life. Let me focus on my business and not my infra.
- StreamBright 6y ago
  Have you had any serious yet? If you are on the happy path does not really matter what you use.
  - brianwawok 6y ago
    Any serious what? I am running a mildly successful SaaS business on hosted kubernetes, yes.
    - StreamBright 6y ago
      Sorry, HN made me retry couple of times. Any serious troubles with Kubernetes. Have you had any of those?
      - brianwawok 6y ago
        No I have not. Going on 3 years now?
        StreamBright 6y ago
        Great. So you do not have any experience with the worst-case scenario. That is exactly my point. I would like to hear your opinion once you fixed an outage that is caused by K8s.
        brianwawok 6y ago
        Well what is this outage caused by? Not understanding how k8s work, or actual provider screwing up? if the former - I try to read carefully and test my workloads in a test cluster before I do it in prod. if the latter, I think google has excellent operational excellence. This is like saying you wouldn't take driving advice from me until I total a few cars. Perhaps I didn't total a few cars because I make good driving decisions?
        StreamBright 6y ago
        An outage that is caused by k8s. You can pick a random from this list. https://github.com/hjacobs/kubernetes-failure-stories https://github.com/hjacobs/kubernetes-failure-stories >>This is like saying you wouldn't take driving advice from me until I total a few cars No, I am saying you do not have enough experience with k8s. That is all. Totaling a car implying you are driving, using k8s implying that whoever wrote that particular part of k8s is driving. You have some control but ultimately you need to understand what is going on. As that list shows many companies who put k8s into production learned it in a hard way.
        brianwawok 6y ago
        Sorry I’m not following Should I compile a list of failure stores for companies that ran Linux because Linux is complicated or has had various bugs? How about a list of failure stories of how Ruby causes problems? You can get errors with any system. You would need to prove to me that k8s has say more errors per day than a competitor (lambda? VMs). I just don’t understand...
- rb808 6y ago
  expensive though?
  - brianwawok 6y ago
    Not really, total cloud costs are under 5% of my revenue. If I want to shave pennies, I have many places to shave before the very thing that is the heart of my business.

tuananh 6y ago

what do you end up using instead of k8s?

StavrosK 6y ago
docker run -d --restart=unless-stopped
0xEFF 6y ago
Plain old systemd unit files work great. If you really need containers for some reason and love UID and GID mapping then podman in systemd is great. The only reason I’ve found to run k8s at home is if your full time job is operating k8s at work.
Bombthecat 6y ago
I mostly use docker compose. Perfect fit for tinkering but not over complicated and huge ram eater. Also:k3s might be an option.
- bradgessler 6y ago
  Compose is great for home use. If you run your home server on a separate box, like a RaspPi or NUC, docker-machine via SSH can make provisioning and updating a tad easier.
dkryptr 6y ago
I've used Nomad by HashiCorp and it's been really nice to use. Though I haven't needed to scale it up a ton so I can't comment on that, but the day-to-day deploying of updated apps is nice and works well.
castillar76 6y ago
I've plugged it before, but Portainer (portainer.io) is what finally got me into running containers for apps. It presents a really nice interface for running individual containers or stacks including docker-compose files, and it extends the nerd knobs for doing things like custom networking or filesystem mounts. I paired it with Ouroboros for doing automated container upgrades in the background, and the result is that I have a solid set of container apps running at all times that I never have to touch. Were I learning k8s for work, it wouldn't be a good solution, but for simple home apps that you want to fire and forget, it definitely beats doing a hand-made installation and periodically care-and-feeding them.