4 ms·

I’m somewhat struggling through getting a service working on Kubernetes. So much configuration to go through. I’m trying to build a product, not be a systems en

by geuis 7y ago

I’m somewhat struggling through getting a service working on Kubernetes. So much configuration to go through. I’m trying to build a product, not be a systems engineer. It occurred to me that Kubernetes might really benefit from some kind of nodal editor. Think iOS Interface Builder but nodal. It would let us visually see the network configurations, make connecting different bits easier, etc. The editor would also show a visual representation of our current clusters, nodes, pods etc along with health and usage etc. Man if that existed right now I’d pay some cash to use it.

haolez 7y ago
Helm might help you installing complicated services like PostgreSQL. But keep in mind that Kubernetes is complex and Helm adds magic on top of that.
- geuis 7y ago
  I’ll take a look. Thanks for the tip. Spent the last couple days going through one blog after another and pages of k8s docs where the directions seem to fall to “copy this command that downloads a script from github and then follow the next step”. I just want to improve my service’s reliability by adding auto scaling (Linode is my host) and k8s seems the way to go. Wasn’t expecting it to be so complicated though.
  - haolez 7y ago
    If you are not using managed Kubernetes, then Nomad is probably a saner choice. https://www.nomadproject.io/ https://www.nomadproject.io/
tonyarkles 7y ago
I'm smiling at your comment, and please don't interpret this as me making fun of you; I'm laughing along with you at the absurdity. > I’m somewhat struggling through getting a service working on Kubernetes. So much configuration to go through. I’m trying to build a product, not be a systems engineer. I'm a systems engineer through-and-through. I've resisted k8s for a long time, but recently inherited a project that was already very k8s-native (for example, it's an Elixir-based app that auto-joins other OTP nodes based on k8s discovery) and figured it was time to dive in. From my understanding, the whole point of this is to abstract away a lot of the pieces that were fundamental to "sysadmin" skills. I'm running this on a DigitalOcean Managed K8s cluster, so I haven't had to do any configuration on the nodes at all, but there's a ton of things that feel dramatically more complicated, even though they're nominally more "abstract" than they would be on bare metal. What's missing, though, are the systems power tools that help you debug what's happening! I have no doubt that this is an exceptionally powerful platform, but... when I start getting errors about a process not being able to make a TCP connection to a publicly-accessible IPv4 address, I can't just connect and look at the iptables configuration on the node to see if there's any kind of rule that might be blocking it. In a lot of ways it feels like we're throwing out the baby with the bath water.
- geuis 7y ago
  Yeah I’m setting this up on Linode’s beta k8s product. Funny thing is that I know how to do mid-level server admin. Most of that comes from building and maintaining the servers for https://jsonip.com https://jsonip.com for years. iptables, nginx configs, etc. Building up a new Ubuntu server from scratch isn’t second nature, but I have a run book and enough familiarity that it isn’t hard. I’ve even worked with Kubernetes a little over the last few years at a couple jobs. Dockerize my app, use some kubectl commands to get it deployed. Luckily I wasn’t responsible for maintaining the infrastructure. Last night I had this realization. The last company I worked at is now publicly traded and has hundreds of millions of users. When I left they were just starting to experiment with k8s but we weren’t using it in production anywhere. There was an entire devops team working on it for at least a year and it still wasn’t ready for production. So I realized that I don’t actually need Kubernetes. I’m super small scale and working on this project on my own. They scaled up to be a billion dollar company without it. But then it leaves me in the position of either hacking together my own limited auto scaling infrastructure or struggling through with Kubernetes. I’m not really sure which direction to move tonight. I have enough personal funds for maybe 6 months to a year and this is just eating up time. Do I struggle through a few more days trying to get this working with Kubernetes and “do it the right way” or give up and hack together a nodejs app that monitors system performance and auto scales for me as needed? At least with the “hacky” app I’ll understand at a base level what’s going on. It’ll take the same time to implement, but when problems eventually crop up I’ll know what’s going on and how to fix it. But with Kubernetes I’ll forget most of the details after it’s working and fixing the inevitable issues will mean re-educating myself during a downtime. Neither is great, but it makes me lean towards DIY. And DIY is what systems like Kubernetes were built to purportedly replace. Help!
  - miscPerson 7y ago
    There’s an expression at Amazon, adopted from our Indian colleagues: do the needful. It means do what’s necessary to accomplish your objective and don’t worry so much about what the “right” approach is — that you should feel empowered to solve problems, not get caught up in the never ending stream of people yelling about what is “correct”. Your goal is to release a product, and you have a tight deadline, so use the tools that let you accomplish that — what you’re familiar with, what you understand, and what enables you to work quickly. If Kubernetes authors have opinions about how you work, they should show some customer obsession and release a product you can effectively use. That’s not your problem. Your problem is releasing your app. Do the needful. There’s no shame in that. For some meme fun: https://www.youtube.com/watch?v=N0fMHt2edZc https://www.youtube.com/watch?v=N0fMHt2edZc
thinkingkong 7y ago
k8s is interesting because its an enabling technology that also wraps tons of other complexity under its name. The execution model is straightforward enough but the networking model is complex for anyone not already familiar with ipvsadm, iptables, routing, layer 2 systems, overlay networks, proxies, dns, etc. Getting it running is actually easy enough to lull people into a sense of accomplishment. But if something goes wrong most teams wont know how to respond other than by restarting something. I really hope we can agree on a simpler abstraction where we trade ease of use for less flexibility.
- streetcat1 7y ago
  Kubernetes was not intended to be managed by humans. The missing piece here is a set of operators that will automatically manage networking, storage. Those operators probably exist in GCP but were not open-sourced.

by geuis 7y ago

haolez 7y ago

Helm might help you installing complicated services like PostgreSQL. But keep in mind that Kubernetes is complex and Helm adds magic on top of that.

geuis 7y ago
I’ll take a look. Thanks for the tip. Spent the last couple days going through one blog after another and pages of k8s docs where the directions seem to fall to “copy this command that downloads a script from github and then follow the next step”. I just want to improve my service’s reliability by adding auto scaling (Linode is my host) and k8s seems the way to go. Wasn’t expecting it to be so complicated though.
- haolez 7y ago
  If you are not using managed Kubernetes, then Nomad is probably a saner choice. https://www.nomadproject.io/ https://www.nomadproject.io/

tonyarkles 7y ago

I'm smiling at your comment, and please don't interpret this as me making fun of you; I'm laughing along with you at the absurdity. > I’m somewhat struggling through getting a service working on Kubernetes. So much configuration to go through. I’m trying to build a product, not be a systems engineer. I'm a systems engineer through-and-through. I've resisted k8s for a long time, but recently inherited a project that was already very k8s-native (for example, it's an Elixir-based app that auto-joins other OTP nodes based on k8s discovery) and figured it was time to dive in. From my understanding, the whole point of this is to abstract away a lot of the pieces that were fundamental to "sysadmin" skills. I'm running this on a DigitalOcean Managed K8s cluster, so I haven't had to do any configuration on the nodes at all, but there's a ton of things that feel dramatically more complicated, even though they're nominally more "abstract" than they would be on bare metal. What's missing, though, are the systems power tools that help you debug what's happening! I have no doubt that this is an exceptionally powerful platform, but... when I start getting errors about a process not being able to make a TCP connection to a publicly-accessible IPv4 address, I can't just connect and look at the iptables configuration on the node to see if there's any kind of rule that might be blocking it. In a lot of ways it feels like we're throwing out the baby with the bath water.

geuis 7y ago
Yeah I’m setting this up on Linode’s beta k8s product. Funny thing is that I know how to do mid-level server admin. Most of that comes from building and maintaining the servers for https://jsonip.com https://jsonip.com for years. iptables, nginx configs, etc. Building up a new Ubuntu server from scratch isn’t second nature, but I have a run book and enough familiarity that it isn’t hard. I’ve even worked with Kubernetes a little over the last few years at a couple jobs. Dockerize my app, use some kubectl commands to get it deployed. Luckily I wasn’t responsible for maintaining the infrastructure. Last night I had this realization. The last company I worked at is now publicly traded and has hundreds of millions of users. When I left they were just starting to experiment with k8s but we weren’t using it in production anywhere. There was an entire devops team working on it for at least a year and it still wasn’t ready for production. So I realized that I don’t actually need Kubernetes. I’m super small scale and working on this project on my own. They scaled up to be a billion dollar company without it. But then it leaves me in the position of either hacking together my own limited auto scaling infrastructure or struggling through with Kubernetes. I’m not really sure which direction to move tonight. I have enough personal funds for maybe 6 months to a year and this is just eating up time. Do I struggle through a few more days trying to get this working with Kubernetes and “do it the right way” or give up and hack together a nodejs app that monitors system performance and auto scales for me as needed? At least with the “hacky” app I’ll understand at a base level what’s going on. It’ll take the same time to implement, but when problems eventually crop up I’ll know what’s going on and how to fix it. But with Kubernetes I’ll forget most of the details after it’s working and fixing the inevitable issues will mean re-educating myself during a downtime. Neither is great, but it makes me lean towards DIY. And DIY is what systems like Kubernetes were built to purportedly replace. Help!
- miscPerson 7y ago
  There’s an expression at Amazon, adopted from our Indian colleagues: do the needful. It means do what’s necessary to accomplish your objective and don’t worry so much about what the “right” approach is — that you should feel empowered to solve problems, not get caught up in the never ending stream of people yelling about what is “correct”. Your goal is to release a product, and you have a tight deadline, so use the tools that let you accomplish that — what you’re familiar with, what you understand, and what enables you to work quickly. If Kubernetes authors have opinions about how you work, they should show some customer obsession and release a product you can effectively use. That’s not your problem. Your problem is releasing your app. Do the needful. There’s no shame in that. For some meme fun: https://www.youtube.com/watch?v=N0fMHt2edZc https://www.youtube.com/watch?v=N0fMHt2edZc

thinkingkong 7y ago

k8s is interesting because its an enabling technology that also wraps tons of other complexity under its name. The execution model is straightforward enough but the networking model is complex for anyone not already familiar with ipvsadm, iptables, routing, layer 2 systems, overlay networks, proxies, dns, etc. Getting it running is actually easy enough to lull people into a sense of accomplishment. But if something goes wrong most teams wont know how to respond other than by restarting something. I really hope we can agree on a simpler abstraction where we trade ease of use for less flexibility.

streetcat1 7y ago
Kubernetes was not intended to be managed by humans. The missing piece here is a set of operators that will automatically manage networking, storage. Those operators probably exist in GCP but were not open-sourced.