3 ms·

Do you really find k8s useful for a single node deployment or were you just making an example? I haven't used but I have used DC/OS, Mesos, and Marathon extens

Do you really find k8s useful for a single node deployment or were you just making an example? I haven't used but I have used DC/OS, Mesos, and Marathon extensively, which is not a setup I'd do for a small number of nodes personally.

linkmotif 9y ago
Yes. Though I should emphasize that I'm using GKE. So it's zero config, zero ops. I am running a single n1-standard-2 node on GKE for a smallish app right now. Comes to 30-40 dollars a month all in (egress traffic, cloud storage, other services). I am working hard on getting people to use this app, and it's great to know that scaling will be best-in-class if I succeed. But I stress it's not about the scaling. It's about the features even at a single node. I wouldn't be spending time writing this stuff had it not been revelatory for me. PS. If you're not able to try it on GKE, there's also https://github.com/kubernetes/kops https://github.com/kubernetes/kops. GKE is great for trying, at least, though. Just get a cluster running and get rolling with the concepts. Then you'll know what it's all about. There isn't that much to it, and what there is to it is great and well documented with a great community.
- Axsuul 9y ago
  Yet with a Docker Swarm setup, you could get away with probably 10 dollars/month. K8s likely taking more resources than your app containers is why k8s may not be the best option for single node deployments. I've personally tried both orchestration options and much prefer Swarm to smaller deployments. Sure k8s could prepare you better for the future if your app takes off to the moon but that's just premature optimization.
  - linkmotif 9y ago
    This might all very well be true, but Swarm doesn’t let you connect cloud disks directly to your containers, to my knowledge. Also I challenge the $10 because what cloud instances plus ancillary services add up to $10? I’m talking 2 CPUs and 7.5gb RAM.
    - Bgd1 9y ago
      It does. Check out Docker volume plugins like: https://docs.docker.com/docker-for-aws/persistent-data-volumes/ https://docs.docker.com/docker-for-aws/persistent-data-volum... https://github.com/thecodeteam/rexray https://github.com/thecodeteam/rexray
    - Axsuul 9y ago
      A Digital Ocean droplet @ 1 GB runs at $10/mo which I have Docker Swarm + my app containers running on.
LaGrange 9y ago
I've found that both Swarm and k8s - but _especially_ the latter - makes the day-to-day application deployment really smooth compared to almost anything I used before (git-deploy, fabric, rsyncing things around). Docker containers are a large part of that, of course, but the tooling and configuration/secret handling help as well.
dominotw 9y ago
>I have used DC/OS, Mesos, and Marathon extensively Would you use DC/OS-marathon (vs k8s) now if you were to make that decision now. I've heard that mesos stack is good for machine learning/big data stacks but how does marathon compare for deploying webapps.
- nemothekid 9y ago
  I've run 2 Mesos stacks in production and have experience setting up a k8s stack (on prem). First off in my experience k8s ops is way more complex that the DC/OS stack. I recently setup a new DC/OS deployment (80% of the cluster resources was Spark, which works natively with Mesos and I'd rather run the ancillary services on Marathon, then spend another 80% of my time on k8s). If I didn't have the Spark requirement I would have went k8s. Despite going with mesos I really had to contend with the fact that k8s just has way, way more developer support - there are so many rich applications in the k8s sphere. Meanwhile I can probably name all the well supported Mesos frameworks offhand. Next, marathon "feels" dead. They recently killed their UI interface as I imagine that they are having trouble giving resources to marathon. 3 years ago I wanted a reverse proxy solution that integrated with mesos as well as non-mesos services so I hacked Caddy to make that work [1]. 3 years later, I was looking for a similar solution and found traefik. It claimed to work with mesos/marathon, but the marathon integration was broken and the mesos integration required polling even though mesos had an events API, so I hacked traefik to make that work [2]. On the other side of the fence, you have companies like Buoyant who rewrote a core piece of their tech (Linkerd) just to support K8s (and only K8s). This has a compounding effect, where over the years things will just become more reliant on assuming you are running k8s. That "cost" you pay to setup Mesos/k8s is usually a one time cost on the order of a month. I feel however, that k8s is going to give you a better ROI (Unless, you are managing 100s of nodes with Spark/YARN/HDFS, then Mesos continues to be the clear winner). [1] https://github.com/mholt/caddy/pull/40 https://github.com/mholt/caddy/pull/40 [1] https://github.com/containous/traefik/pull/2617 https://github.com/containous/traefik/pull/2617
  - mholt 9y ago
    From me, an eternal thank-you for upgrading Caddy's proxy middleware!
chrisgaun 9y ago
As a discloser, I work at Mesosphere who engineers DC/OS. I would like to note that DC/OS now has Kubernetes [Beta]. You can manage Kubernetes resources on your DC/OS cluster now. Here are the instructions on Github to get started: https://github.com/mesosphere/dcos-kubernetes-quickstart https://github.com/mesosphere/dcos-kubernetes-quickstart You can also join the Slack to help you on you out: http://chat.dcos.io/ http://chat.dcos.io/ (the #kubernetes channel)

linkmotif 9y ago

Yes. Though I should emphasize that I'm using GKE. So it's zero config, zero ops. I am running a single n1-standard-2 node on GKE for a smallish app right now. Comes to 30-40 dollars a month all in (egress traffic, cloud storage, other services). I am working hard on getting people to use this app, and it's great to know that scaling will be best-in-class if I succeed. But I stress it's not about the scaling. It's about the features even at a single node. I wouldn't be spending time writing this stuff had it not been revelatory for me. PS. If you're not able to try it on GKE, there's also https://github.com/kubernetes/kops https://github.com/kubernetes/kops. GKE is great for trying, at least, though. Just get a cluster running and get rolling with the concepts. Then you'll know what it's all about. There isn't that much to it, and what there is to it is great and well documented with a great community.

Axsuul 9y ago
Yet with a Docker Swarm setup, you could get away with probably 10 dollars/month. K8s likely taking more resources than your app containers is why k8s may not be the best option for single node deployments. I've personally tried both orchestration options and much prefer Swarm to smaller deployments. Sure k8s could prepare you better for the future if your app takes off to the moon but that's just premature optimization.
- linkmotif 9y ago
  This might all very well be true, but Swarm doesn’t let you connect cloud disks directly to your containers, to my knowledge. Also I challenge the $10 because what cloud instances plus ancillary services add up to $10? I’m talking 2 CPUs and 7.5gb RAM.
  - Bgd1 9y ago
    It does. Check out Docker volume plugins like: https://docs.docker.com/docker-for-aws/persistent-data-volumes/ https://docs.docker.com/docker-for-aws/persistent-data-volum... https://github.com/thecodeteam/rexray https://github.com/thecodeteam/rexray
  - Axsuul 9y ago
    A Digital Ocean droplet @ 1 GB runs at $10/mo which I have Docker Swarm + my app containers running on.

LaGrange 9y ago

I've found that both Swarm and k8s - but _especially_ the latter - makes the day-to-day application deployment really smooth compared to almost anything I used before (git-deploy, fabric, rsyncing things around). Docker containers are a large part of that, of course, but the tooling and configuration/secret handling help as well.

dominotw 9y ago

>I have used DC/OS, Mesos, and Marathon extensively Would you use DC/OS-marathon (vs k8s) now if you were to make that decision now. I've heard that mesos stack is good for machine learning/big data stacks but how does marathon compare for deploying webapps.

nemothekid 9y ago
I've run 2 Mesos stacks in production and have experience setting up a k8s stack (on prem). First off in my experience k8s ops is way more complex that the DC/OS stack. I recently setup a new DC/OS deployment (80% of the cluster resources was Spark, which works natively with Mesos and I'd rather run the ancillary services on Marathon, then spend another 80% of my time on k8s). If I didn't have the Spark requirement I would have went k8s. Despite going with mesos I really had to contend with the fact that k8s just has way, way more developer support - there are so many rich applications in the k8s sphere. Meanwhile I can probably name all the well supported Mesos frameworks offhand. Next, marathon "feels" dead. They recently killed their UI interface as I imagine that they are having trouble giving resources to marathon. 3 years ago I wanted a reverse proxy solution that integrated with mesos as well as non-mesos services so I hacked Caddy to make that work [1]. 3 years later, I was looking for a similar solution and found traefik. It claimed to work with mesos/marathon, but the marathon integration was broken and the mesos integration required polling even though mesos had an events API, so I hacked traefik to make that work [2]. On the other side of the fence, you have companies like Buoyant who rewrote a core piece of their tech (Linkerd) just to support K8s (and only K8s). This has a compounding effect, where over the years things will just become more reliant on assuming you are running k8s. That "cost" you pay to setup Mesos/k8s is usually a one time cost on the order of a month. I feel however, that k8s is going to give you a better ROI (Unless, you are managing 100s of nodes with Spark/YARN/HDFS, then Mesos continues to be the clear winner). [1] https://github.com/mholt/caddy/pull/40 https://github.com/mholt/caddy/pull/40 [1] https://github.com/containous/traefik/pull/2617 https://github.com/containous/traefik/pull/2617
- mholt 9y ago
  From me, an eternal thank-you for upgrading Caddy's proxy middleware!

chrisgaun 9y ago

As a discloser, I work at Mesosphere who engineers DC/OS. I would like to note that DC/OS now has Kubernetes [Beta]. You can manage Kubernetes resources on your DC/OS cluster now. Here are the instructions on Github to get started: https://github.com/mesosphere/dcos-kubernetes-quickstart https://github.com/mesosphere/dcos-kubernetes-quickstart You can also join the Slack to help you on you out: http://chat.dcos.io/ http://chat.dcos.io/ (the #kubernetes channel)