4 ms·

Firstly tell me more about your 1-person full-stack venture, but second how comes you, with barely any time for sitting down can use k8s happily but it falls ov

Firstly tell me more about your 1-person full-stack venture, but second how comes you, with barely any time for sitting down can use k8s happily but it falls over for others. I am struggling to see truth amount the comments here :-(

dashwav 6y ago
TBF with a completely greenfield project and managed K8s (GKE or EKS) you can absolutely get a pretty well set up infra very quickly if you are willing to learn how to do so. I often get the feeling a lot of the negativity comes (rightfully so) from trying to replicate a current existing project into kubernetes. This is true of almost any paradigm - try replicating a Java EE monolith into Erlang and you are going to have a lot of problems. The big thing to note is that starting a project on Erlang very well might solve the problems that a Java EE project ran into, but that is because they were able to solve them at the ground floor, and just popping a Java EE project with all of it's architecture into an Erlang project will probably end up in a worse spot. I think that this is what often happens with k8s as well - if you or your company have a currently working implementation that isn't on k8s, of course you won't be able to just easily plop it into a k8s cluster and everything be all well and good, but I think the problem is that people are equating that issue with k8s itself, which is a completely different paradigm.
- solidasparagus 6y ago
  > managed K8s (GKE or EKS) you can absolutely get a pretty well set up infra very quickly And then tear your hair out when something doesn't work for some reason and root causing it requires learning a stupid number of layers. k8s is easy until it goes wrong.
  - harpratap 6y ago
    Isn't this the same for every software? How is debugging issues with Linux, NGINX, any complex framework any easier?
    - Aeolun 6y ago
      Well, for one thing, all logs are in /var/log. If it doesn’t fail just right in kubernetes there might be no logs at all. I’m thinking on particular about trying to mount filesystems into pods.
    - lenkite 6y ago
      Debugging k8s issues is like debugging a vast distributed monolith with a vague guesses on where the problem occurred.
snuxoll 6y ago
Not the GP, but I honestly couldn't tell you. A lot probably comes down to tooling, the applications you are deploying, security requirements, etc., as well as how familiar you are with k8s itself. I migrated PCGamingWiki from running on some Hetzner boxes to DigitalOcean Kubernetes in a few days of work creating Dockerfiles and k8s manifests. I run a Kubernetes cluster at work fairly hands-free that hosts applications critical to our billing operations, and developers on the team deploy new applications with little or no support. Any of the issues I've hit are an artifact of migrating legacy applications not designed to run in more-or-less stateless environments, which is why the PCGW Community site still runs on its own server (Invision Community sucks). I really don't see all the issues people have that aren't due to a mismatch of application design vs target environment (and no, it's not monolith vs microservice - monoliths run just fine on k8s; but you should be designing your application with a 12-factor environment in mind) or a misguided notion that you will be drowning in YAML hell (it's real, but you can manage it - and it's directly related to the complexity of the services you are deploying).
- ramraj07 6y ago
  Seems like you are a thoughtful engineer who can also make sure you don't make any fundamental design flaws while building these systems well. Whenever I have seen kubernetes fail it's often also because the engineer(s) who built it were not thoughtful at all and often didn't fully understand what they were doing. Perhaps k8s's failing is it makes people who dont know enough think that they do.
  - snuxoll 6y ago
    I mean, I’m far from perfect - the trick has always been to KISS. I don’t use istio, it’s absolute overkill for my needs. I use nginx-ingress because it fits the bill, I know nginx as do enough other people that they could exec into a pod to debug it. I don’t run stateful applications that aren’t prepared to have servers randomly vanishing because it take a LOT of work to get these running in-cluster. I don’t use public helm charts because they often suck and making your own container is something you can do quickly if you were able to deploy the software on a traditional server. Every choice I make is done with the day 2 operations in mind, not what is hot, what gets initially deployed fastest - but what makes it so I can touch the thing as frequently as possible. PCGW is a great example - installing a new Mediawiki extension, changing a config file, upgrading to a newer MW release is just updating a file or a git submodule and committing. I don’t get paid thousands a month to manage the site for the owner, so I make my time spent as efficient as can be done.
banifo 6y ago
Might also be the amount of experience a person has. I have done Softwaredevelopment for ~15 years, swiched now to infrastructure and build a gke cluster. It was awesome, very logical, nice and easy to use. Now i read stories from coinbase and don't get it.
lmm 6y ago
People who don't like - or don't "get" - declarative systems tend to spend an inordinate amount of time and effort fighting them. I've seen the same thing with a delcarative build system (maven), or with adopting an ORM - if you're willing to work with the tool then it will save you a lot of effort, but if you're determined to do things your own way then you can make it almost arbitrarily difficult.
- antonvs 6y ago
  > if you're determined to do things your own way then you can make it almost arbitrarily difficult. This is true of software development in general, if not life itself! But you're right that declarative systems amplify this issue.
antonvs 6y ago
This is pretty typical when a product is still climbing the adoption curve. I've helped small companies set up (managed) k8s clusters and migrate their apps to them, and when you know what you're doing, it's a super smooth experience that's basically all upside. But, if you're approaching it for the first time with no assistance, there are lots of things that can trip you up, and lots to learn. That's not a reflection on k8s, it's just the nature of the large set of problems it's solving. K8s is succeeding because it's very well designed, has a large and diverse ecosystem, and solves set of important problems that very few other tools even try to tackle. Apache Mesos perhaps comes closest, but it's not quite as pragmatic, and its adoption level reflects that. Also, because of k8s' scope, many people may not fully appreciate the range of problems it's solving, seeing it through the lens of their own background and focus.

dashwav 6y ago

TBF with a completely greenfield project and managed K8s (GKE or EKS) you can absolutely get a pretty well set up infra very quickly if you are willing to learn how to do so. I often get the feeling a lot of the negativity comes (rightfully so) from trying to replicate a current existing project into kubernetes. This is true of almost any paradigm - try replicating a Java EE monolith into Erlang and you are going to have a lot of problems. The big thing to note is that starting a project on Erlang very well might solve the problems that a Java EE project ran into, but that is because they were able to solve them at the ground floor, and just popping a Java EE project with all of it's architecture into an Erlang project will probably end up in a worse spot. I think that this is what often happens with k8s as well - if you or your company have a currently working implementation that isn't on k8s, of course you won't be able to just easily plop it into a k8s cluster and everything be all well and good, but I think the problem is that people are equating that issue with k8s itself, which is a completely different paradigm.

solidasparagus 6y ago
> managed K8s (GKE or EKS) you can absolutely get a pretty well set up infra very quickly And then tear your hair out when something doesn't work for some reason and root causing it requires learning a stupid number of layers. k8s is easy until it goes wrong.
- harpratap 6y ago
  Isn't this the same for every software? How is debugging issues with Linux, NGINX, any complex framework any easier?
  - Aeolun 6y ago
    Well, for one thing, all logs are in /var/log. If it doesn’t fail just right in kubernetes there might be no logs at all. I’m thinking on particular about trying to mount filesystems into pods.
  - lenkite 6y ago
    Debugging k8s issues is like debugging a vast distributed monolith with a vague guesses on where the problem occurred.

snuxoll 6y ago

Not the GP, but I honestly couldn't tell you. A lot probably comes down to tooling, the applications you are deploying, security requirements, etc., as well as how familiar you are with k8s itself. I migrated PCGamingWiki from running on some Hetzner boxes to DigitalOcean Kubernetes in a few days of work creating Dockerfiles and k8s manifests. I run a Kubernetes cluster at work fairly hands-free that hosts applications critical to our billing operations, and developers on the team deploy new applications with little or no support. Any of the issues I've hit are an artifact of migrating legacy applications not designed to run in more-or-less stateless environments, which is why the PCGW Community site still runs on its own server (Invision Community sucks). I really don't see all the issues people have that aren't due to a mismatch of application design vs target environment (and no, it's not monolith vs microservice - monoliths run just fine on k8s; but you should be designing your application with a 12-factor environment in mind) or a misguided notion that you will be drowning in YAML hell (it's real, but you can manage it - and it's directly related to the complexity of the services you are deploying).

ramraj07 6y ago
Seems like you are a thoughtful engineer who can also make sure you don't make any fundamental design flaws while building these systems well. Whenever I have seen kubernetes fail it's often also because the engineer(s) who built it were not thoughtful at all and often didn't fully understand what they were doing. Perhaps k8s's failing is it makes people who dont know enough think that they do.
- snuxoll 6y ago
  I mean, I’m far from perfect - the trick has always been to KISS. I don’t use istio, it’s absolute overkill for my needs. I use nginx-ingress because it fits the bill, I know nginx as do enough other people that they could exec into a pod to debug it. I don’t run stateful applications that aren’t prepared to have servers randomly vanishing because it take a LOT of work to get these running in-cluster. I don’t use public helm charts because they often suck and making your own container is something you can do quickly if you were able to deploy the software on a traditional server. Every choice I make is done with the day 2 operations in mind, not what is hot, what gets initially deployed fastest - but what makes it so I can touch the thing as frequently as possible. PCGW is a great example - installing a new Mediawiki extension, changing a config file, upgrading to a newer MW release is just updating a file or a git submodule and committing. I don’t get paid thousands a month to manage the site for the owner, so I make my time spent as efficient as can be done.

banifo 6y ago

Might also be the amount of experience a person has. I have done Softwaredevelopment for ~15 years, swiched now to infrastructure and build a gke cluster. It was awesome, very logical, nice and easy to use. Now i read stories from coinbase and don't get it.

lmm 6y ago

People who don't like - or don't "get" - declarative systems tend to spend an inordinate amount of time and effort fighting them. I've seen the same thing with a delcarative build system (maven), or with adopting an ORM - if you're willing to work with the tool then it will save you a lot of effort, but if you're determined to do things your own way then you can make it almost arbitrarily difficult.

antonvs 6y ago
> if you're determined to do things your own way then you can make it almost arbitrarily difficult. This is true of software development in general, if not life itself! But you're right that declarative systems amplify this issue.

antonvs 6y ago

This is pretty typical when a product is still climbing the adoption curve. I've helped small companies set up (managed) k8s clusters and migrate their apps to them, and when you know what you're doing, it's a super smooth experience that's basically all upside. But, if you're approaching it for the first time with no assistance, there are lots of things that can trip you up, and lots to learn. That's not a reflection on k8s, it's just the nature of the large set of problems it's solving. K8s is succeeding because it's very well designed, has a large and diverse ecosystem, and solves set of important problems that very few other tools even try to tackle. Apache Mesos perhaps comes closest, but it's not quite as pragmatic, and its adoption level reflects that. Also, because of k8s' scope, many people may not fully appreciate the range of problems it's solving, seeing it through the lens of their own background and focus.