5 ms·

> look forward to the day that we never have to hear about Kubernetes again. How do you expect to deploy services in a cluster of multiple nodes communicating

> look forward to the day that we never have to hear about Kubernetes again. How do you expect to deploy services in a cluster of multiple nodes communicating over a private software-defined network, then?

coding123 4y ago
There ARE other ways, but mesos was a freaking mess. I actually liked swarm, but it lost. k8s also brings in pretty novel ideas that destroy proprietary IaaS, such as the nginx (or other) front end ingress routers that I haven't seen implemented meaningfully in a IaaS neutral way. Sure, you may have a few properties in your k8s yaml that change from AWS vs Google vs Azure vs other k8s hosts. That's not something that takes more than a debugging session to get right. That's HUUUGE progress.
- arinlen 4y ago
  > I actually liked swarm, but it lost. "lost" is too strong of a word. To me Docker swarm mode still holds the upper hand in terms of deployment ease and simplicity. If anyone just wants to bolt together a couple of nodes without much concern regarding performance then Docker swarm mode is perhaps the best option out there at the moment. However, this is changing very fast. Ubuntu now ships with microk8s which just works and outperforms Docker swarm mode. Perhaps Docker swarm's worst enemy is how the project was managed and promoted. They had a tool to trivially setup a cluster of nodes with close to zero challenge, but somehow Kubernetes managed to be pushed as the only option.
ok123456 4y ago
docker compose. Start it as a systemd service if you want.
- majewsky 4y ago
  docker-compose only deploys on individual nodes though.
  - ok123456 4y ago
    That's shifting the goal posts. And, that covers 99.9% of cases anyway. If you want to deploy across multiple nodes run docker compose on multiple nodes and expose the nodes to each other over a vlan that is managed as part of your actual infrastructure where it has a chance of being actually documented and put into a configuration management system.
    - coding123 4y ago
      It may not be shifting the goal posts much because the typical situations that k8s comes up is when you need multiple nodes, so yeah we're IN the 0.1% of k8s users in here.
    - orf 4y ago
      > If you want to deploy across multiple nodes run docker compose on multiple nodes and expose the nodes to each other over a vlan that is managed as part of your actual infrastructure Indeed, and for a Linux user, you can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem. It doesn’t mean Kubernetes isn’t a useful turnkey solution, especially if it’s managed.
      - ok123456 4y ago
        What's the actual savings if you now just have additional complexity that isn't really managed all that well when you use it as a turn key solution?
        orf 4y ago
        Because you press 4 buttons on a cloud console, or write a bit of terraform, and you have a consistent management plane for all of this, one that you can plug into any tool the very large ecosystem. Then your entire engineering org just deploys things. Need per-application network security policies? Done. Per app IAM roles? Done. Secrets management? Done. Role based access control? Done. Isolate services to different pools of machines? Done. Per-app live-updating configuration files across your entire fleet? Done. CPU, memory or arbitrary per-app metric based auto scaling? Done. Want to run all this on cheap spot instances? Done. Want path based HTTP routing within your cluster? Done. Want an easy deployment mechanism with versioning and rollbacks? Done. Want simple and consistent service discovery across all your servers? Done. Want to get advanced and build custom resources to manage groups of other resources whilst taking advantage of k8s management plane in a consistent manner that just works with all the other tooling? Done. Want to do all this fairly consistently on any cloud vendor, or even on prem? Done. Want a large pool of candidates familiar with this stack? Done and done. All those lead to non trivial savings, so you can actually focus on your product and not building a crappy version of Kubernetes by stringing together docker compose instances. But yes, if you’ve got a single developer and you’re deploying by ssh-ing into a single machine and running git pull, it’s definitely overkill.
ki_ 4y ago
foreach serviceX.servers as server { ssh user@ip "cd service-x-project-dir && git pull && ./deploy.sh && service serviceX reload" } there u go. As for SDN, why do u even need that? Some simple scripts that manage your server count and their configurations over ssh is all u need.
- ok123456 4y ago
  This also works.
- arinlen 4y ago
  > foreach serviceX.servers as server { ssh user@ip "cd service-x-project-dir && git pull && ./deploy.sh && service serviceX reload" } Do you honestly think this sort of thing, hand-waving included, is a better alternative to $ kubectl apply -k <kustomize script> I mean, what are your plans to react to one of your serviceX crapping out? How do you know? How do you recover? How do you roll back? Do you expect to throw more of your ad hoc shell scripts to the mix? Your example is as good as presenting your deployment strategy as $ load ""
  - ki_ 4y ago
    Software is not really random. Mostly, they all fail or none of them fail. But in case only 1 server has a problem. You'd just use a script that removes that server from your load balancer so you can inspect it, fix it, etc.. I dont do rollbacks. If you deploy code that bugs out, you already failed. Because clearly the coders & reviewers did NOT understand the code. Proper testing helps also. And if doom day comes, just deploy a previous commit. It's pretty much the same as what u do with kubernetes. That being said. Kubernetes is indeed much easier for this. It all comes down to: "do you actually know what you are doing?". If you know what you are doing, all these problems dont happen. If you feel it's important to be able to spin up new servers from scratch every time you deploy. i guess kubernetes is great for that. it's not for me. As for being able to know what is happening, i do understand that can be impossible sometimes in large companies. Too many developers, too much code. Still, it shouldnt be used as an execuse for breaking the live servers.

coding123 4y ago

There ARE other ways, but mesos was a freaking mess. I actually liked swarm, but it lost. k8s also brings in pretty novel ideas that destroy proprietary IaaS, such as the nginx (or other) front end ingress routers that I haven't seen implemented meaningfully in a IaaS neutral way. Sure, you may have a few properties in your k8s yaml that change from AWS vs Google vs Azure vs other k8s hosts. That's not something that takes more than a debugging session to get right. That's HUUUGE progress.

arinlen 4y ago
> I actually liked swarm, but it lost. "lost" is too strong of a word. To me Docker swarm mode still holds the upper hand in terms of deployment ease and simplicity. If anyone just wants to bolt together a couple of nodes without much concern regarding performance then Docker swarm mode is perhaps the best option out there at the moment. However, this is changing very fast. Ubuntu now ships with microk8s which just works and outperforms Docker swarm mode. Perhaps Docker swarm's worst enemy is how the project was managed and promoted. They had a tool to trivially setup a cluster of nodes with close to zero challenge, but somehow Kubernetes managed to be pushed as the only option.

ok123456 4y ago

docker compose. Start it as a systemd service if you want.

majewsky 4y ago
docker-compose only deploys on individual nodes though.
- ok123456 4y ago
  That's shifting the goal posts. And, that covers 99.9% of cases anyway. If you want to deploy across multiple nodes run docker compose on multiple nodes and expose the nodes to each other over a vlan that is managed as part of your actual infrastructure where it has a chance of being actually documented and put into a configuration management system.
  - coding123 4y ago
    It may not be shifting the goal posts much because the typical situations that k8s comes up is when you need multiple nodes, so yeah we're IN the 0.1% of k8s users in here.
  - orf 4y ago
    > If you want to deploy across multiple nodes run docker compose on multiple nodes and expose the nodes to each other over a vlan that is managed as part of your actual infrastructure Indeed, and for a Linux user, you can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem. It doesn’t mean Kubernetes isn’t a useful turnkey solution, especially if it’s managed.
    - ok123456 4y ago
      What's the actual savings if you now just have additional complexity that isn't really managed all that well when you use it as a turn key solution?
      - orf 4y ago
        Because you press 4 buttons on a cloud console, or write a bit of terraform, and you have a consistent management plane for all of this, one that you can plug into any tool the very large ecosystem. Then your entire engineering org just deploys things. Need per-application network security policies? Done. Per app IAM roles? Done. Secrets management? Done. Role based access control? Done. Isolate services to different pools of machines? Done. Per-app live-updating configuration files across your entire fleet? Done. CPU, memory or arbitrary per-app metric based auto scaling? Done. Want to run all this on cheap spot instances? Done. Want path based HTTP routing within your cluster? Done. Want an easy deployment mechanism with versioning and rollbacks? Done. Want simple and consistent service discovery across all your servers? Done. Want to get advanced and build custom resources to manage groups of other resources whilst taking advantage of k8s management plane in a consistent manner that just works with all the other tooling? Done. Want to do all this fairly consistently on any cloud vendor, or even on prem? Done. Want a large pool of candidates familiar with this stack? Done and done. All those lead to non trivial savings, so you can actually focus on your product and not building a crappy version of Kubernetes by stringing together docker compose instances. But yes, if you’ve got a single developer and you’re deploying by ssh-ing into a single machine and running git pull, it’s definitely overkill.

ki_ 4y ago

foreach serviceX.servers as server { ssh user@ip "cd service-x-project-dir && git pull && ./deploy.sh && service serviceX reload" } there u go. As for SDN, why do u even need that? Some simple scripts that manage your server count and their configurations over ssh is all u need.

ok123456 4y ago
This also works.
arinlen 4y ago
> foreach serviceX.servers as server { ssh user@ip "cd service-x-project-dir && git pull && ./deploy.sh && service serviceX reload" } Do you honestly think this sort of thing, hand-waving included, is a better alternative to $ kubectl apply -k <kustomize script> I mean, what are your plans to react to one of your serviceX crapping out? How do you know? How do you recover? How do you roll back? Do you expect to throw more of your ad hoc shell scripts to the mix? Your example is as good as presenting your deployment strategy as $ load ""
- ki_ 4y ago
  Software is not really random. Mostly, they all fail or none of them fail. But in case only 1 server has a problem. You'd just use a script that removes that server from your load balancer so you can inspect it, fix it, etc.. I dont do rollbacks. If you deploy code that bugs out, you already failed. Because clearly the coders & reviewers did NOT understand the code. Proper testing helps also. And if doom day comes, just deploy a previous commit. It's pretty much the same as what u do with kubernetes. That being said. Kubernetes is indeed much easier for this. It all comes down to: "do you actually know what you are doing?". If you know what you are doing, all these problems dont happen. If you feel it's important to be able to spin up new servers from scratch every time you deploy. i guess kubernetes is great for that. it's not for me. As for being able to know what is happening, i do understand that can be impossible sometimes in large companies. Too many developers, too much code. Still, it shouldnt be used as an execuse for breaking the live servers.