5 ms·

Maybe it's just me, but I fail to see how all those project make the developer/devops person's life less complex. I don't have the experience to have managed a

Maybe it's just me, but I fail to see how all those project make the developer/devops person's life less complex. I don't have the experience to have managed a cluster of 1000s of machines but I HAVE managed a cluster of ~50 nodes with MySQL, ElasticSearch, RabbitMQ, ZeroMQ, PHP, Clojure and the whole gang at various points in time. I am yet to see a single project (Mesos, Kubernetes, Docker, whatever) that would DRAMATICALLY make my life so much better. Not trying to be overly negative here, just looking for answers and better solutions. EDIT I also continuously wonder how do things like Ansible fit into the picture. Are they competitors? Are they supplemental to the likes of Mesos/Kubernetes? Are they orthogonal?

sre_ops 12y ago
People like solutions in search of a problem especially if with those "solutions" come with VC money from the same kind of clue challenged VCs that invest into the likes of Meerkat.
brendandburns 12y ago
Disclaimer: lead engineer on kubernetes here... It really depends on how you do deployment. Containers provide deployment (and more important rollback) that is better than other deployment tools like Puppet/Chef/... because they are atomic (they either work, or they fail, they don't get stuck in the middle) and they package up all of their dependencies within them, so that they don't have the "well it worked on my machine" problems. Systems like Mesos and Kuberenetes, decouple applications from the individual machines (and the operating system on those machines), and are online systems with self-healing properties (so that they will fix themselves rather than waking you up in the middle of the night) k8s and mesos turn containers into an API that spans an entire fleet of machines, and enables you to dynamically use (and re-use) the fleet of machines for multiple different applications. No more dedicated boxes for mysql, mongo, etc. This in turn enables you to have an easier ops experience, because every single machine in your fleet is homogenous (same OS, same patches, etc) OS management is abstracted away from Application management, so that they don't interfere with each other. Since things in the API are expressed in terms of applications, it's easy to add health checks and automatic restart to the system, and provide self-healing properties as well. Both kubernetes and Mesos also make replication a first-order primitive so that it is easy to scale in response to load. Ansible is sort of orthogonal to systems like Kubernetes and Mesos. Kubernetes and Mesos are designed to be online, self-repairing systems. Ansible is a way to easily execute commands on a bunch of machines. I can see collaborative use cases, where you generally use Kubernetes for deployments, but use Ansible for querying some data while debugging, or somesuch. Anyway, sorry for the extended response. There actually is way more that I could say about the topic ;)
- raspasov 12y ago
  Thanks for the detailed response. The point about atomic deploys is a good one. However, if we all agree that immutable deployments are a good thing, I've been wondering how is that fundamentally different from launching new instances via AWS/Rackspace/Google API? Is there really a fundamental difference between shipping a Docker container to a set of servers, or just relaunching/rebuilding servers? You also mention the "no dedicated Mysql, Mongo boxes" etc. I am 100% for that. However, how can you really make that work with databases/systems like MySQL which were fundamentally designed to work on one machine, and scaling them across machines is usually very painful, or at least, let's say, not very "idiomatic" (if I can use that word here lol). I can see the auto scaling part working with distributed databases like Riak/Cassandra, but even there the solution is not clear-cut and "out of the box". It still feels like some manual orchestration work is needed - correct me if I'm wrong. I can totally see the "online, self-repairing point" but only for application servers that were designed from the scratch to be easily scalable by just adding servers. Which is the case for most scripting languages PHP/Ruby/Python et al and for well designed JVM/CLR/native/Go applications as well. However, I would argue scaling the app boxes/containers is the EASY part. Again, you can always scale up by "just" copying machines (I know, it's always harder that that). The hard part comes with managing the database servers or your cluster of messaging queues, or some other stateful thing that has to persist data SAFELY. Does Kubernetes/Docker REALLY make my life easier with those kinds of things? Is the answer to use DynamoDB/BigTable/RDS/managed queues and forget about that hassle of managing a database or a queueing cluster? Looking for answers :) . Thanks!
  - m_mueller 12y ago
    > I've been wondering how is that fundamentally different from launching new instances via AWS/Rackspace/Google API? Is there really a fundamental difference between shipping a Docker container to a set of servers, or just relaunching/rebuilding servers? There's two perspectives to this: * From a user's perspective, ignoring performance implications, containers should be the same as VMs. * From a hardware perspective however, containers are much more lightweight in terms of CPU, Ram and Disk space, since they all share at least the kernel. Why should you care? Well if something is less resource consuming, it means that (a) you could run it for less than what you pay for VMs or (b) it can be sold to you for less. There's some additional benefits like fast "boots", strict decoupling of persistant and non-persistant storage (which I find an advantage, restarting a container cleans up whatever you don't care for), but at the end I think it comes down to money.
benjaminwootton 12y ago
A few ways in which this stack can simplify your life: - Abstract away elements of failover, disaster recovery, clustering, load balancing - Abstract your application services cleanly from underlying servers - Give you more consistency across and within development, test and production environments - Allow for immutable deployments - Allow for canary releasing, A/B releasing, rollback etc - Better isolation of processes and services - Move away from general purpose operating systems to lighter weight single purpose OS such as CoreOS Configuration management still has a role to play in this stack, but somewhat smaller than the old world
leef 12y ago
It doesn't. As an average developer it's my public/private cloud providers job to take my images/dockers and schedule them correctly on whatever underlying fleet of hardware they run. I can only assume Tectonic and related tech are aimed at those developers who run hybrid or private clouds for the rest of their company.
- brendandburns 12y ago
  This is also very true. The purpose of systems like Kubernetes/Mesos is to be set up once by a company, and for most developers to just interact with the CLI tool to run containers. Tools like Google Container Engine, make this into a software as a service product, where the cloud provider provides the API. In that world, as a developer, you never really think about the OS, just your app. But unlike in traditional PaaS, there is no framework that restricts what you can code (language, libraries, etc).
  - jacques_chester 12y ago
    > But unlike in traditional PaaS, there is no framework that restricts what you can code (language, libraries, etc). I'm not sure how traditional PaaSes are limited in that sense. Heroku accepts a wide variety of software through the buildpacks mechanism. Cloud Foundry supports both buildpacks and docker images, with clean extension points to add further mechanisms -- for example, .NET-on-Windows deployments, which is under development. Disclaimer: I've worked on Cloud Foundry and I'm employed by Pivotal.
  - ajessup 12y ago
    The main difference between an PaaS like Heroku and a general purpose cluster manager service like GKE is that the former can simplify my making certain assumptions about your workload. For example if a service knows your container is serving a web application then it can sensibly provision load balancers, DNS, HTTPS, auto-scaling, static content caching, automatic QPS monitoring etc. to support your app with little explicit configuration. And with a commercial service you get an SLA for those things. You can get much of this with a general purpose cluster as well, but of course you need to configure it yourself, and more importantly - debug it when it goes awry. Disclaimer: I work for Google
cmelbye 12y ago
There are already a couple pretty good responses, but I would add that all of the technologies you mentioned (along with others in this space) are still evolving and maturing. Docker popularized containers, and now we're seeing the supporting technologies that will help fulfill some of the promises of container-based deployment. Big companies have had this technology internally for years, and now it's starting to get productized. I'm referring to things like schedulers, networking, monitoring, etcd/consul, and more. I personally don't think all of the pieces are there yet to make it as seamless and easy as it could be, but it's coming.

sre_ops 12y ago

People like solutions in search of a problem especially if with those "solutions" come with VC money from the same kind of clue challenged VCs that invest into the likes of Meerkat.

brendandburns 12y ago

Disclaimer: lead engineer on kubernetes here... It really depends on how you do deployment. Containers provide deployment (and more important rollback) that is better than other deployment tools like Puppet/Chef/... because they are atomic (they either work, or they fail, they don't get stuck in the middle) and they package up all of their dependencies within them, so that they don't have the "well it worked on my machine" problems. Systems like Mesos and Kuberenetes, decouple applications from the individual machines (and the operating system on those machines), and are online systems with self-healing properties (so that they will fix themselves rather than waking you up in the middle of the night) k8s and mesos turn containers into an API that spans an entire fleet of machines, and enables you to dynamically use (and re-use) the fleet of machines for multiple different applications. No more dedicated boxes for mysql, mongo, etc. This in turn enables you to have an easier ops experience, because every single machine in your fleet is homogenous (same OS, same patches, etc) OS management is abstracted away from Application management, so that they don't interfere with each other. Since things in the API are expressed in terms of applications, it's easy to add health checks and automatic restart to the system, and provide self-healing properties as well. Both kubernetes and Mesos also make replication a first-order primitive so that it is easy to scale in response to load. Ansible is sort of orthogonal to systems like Kubernetes and Mesos. Kubernetes and Mesos are designed to be online, self-repairing systems. Ansible is a way to easily execute commands on a bunch of machines. I can see collaborative use cases, where you generally use Kubernetes for deployments, but use Ansible for querying some data while debugging, or somesuch. Anyway, sorry for the extended response. There actually is way more that I could say about the topic ;)

raspasov 12y ago
Thanks for the detailed response. The point about atomic deploys is a good one. However, if we all agree that immutable deployments are a good thing, I've been wondering how is that fundamentally different from launching new instances via AWS/Rackspace/Google API? Is there really a fundamental difference between shipping a Docker container to a set of servers, or just relaunching/rebuilding servers? You also mention the "no dedicated Mysql, Mongo boxes" etc. I am 100% for that. However, how can you really make that work with databases/systems like MySQL which were fundamentally designed to work on one machine, and scaling them across machines is usually very painful, or at least, let's say, not very "idiomatic" (if I can use that word here lol). I can see the auto scaling part working with distributed databases like Riak/Cassandra, but even there the solution is not clear-cut and "out of the box". It still feels like some manual orchestration work is needed - correct me if I'm wrong. I can totally see the "online, self-repairing point" but only for application servers that were designed from the scratch to be easily scalable by just adding servers. Which is the case for most scripting languages PHP/Ruby/Python et al and for well designed JVM/CLR/native/Go applications as well. However, I would argue scaling the app boxes/containers is the EASY part. Again, you can always scale up by "just" copying machines (I know, it's always harder that that). The hard part comes with managing the database servers or your cluster of messaging queues, or some other stateful thing that has to persist data SAFELY. Does Kubernetes/Docker REALLY make my life easier with those kinds of things? Is the answer to use DynamoDB/BigTable/RDS/managed queues and forget about that hassle of managing a database or a queueing cluster? Looking for answers :) . Thanks!
- m_mueller 12y ago
  > I've been wondering how is that fundamentally different from launching new instances via AWS/Rackspace/Google API? Is there really a fundamental difference between shipping a Docker container to a set of servers, or just relaunching/rebuilding servers? There's two perspectives to this: * From a user's perspective, ignoring performance implications, containers should be the same as VMs. * From a hardware perspective however, containers are much more lightweight in terms of CPU, Ram and Disk space, since they all share at least the kernel. Why should you care? Well if something is less resource consuming, it means that (a) you could run it for less than what you pay for VMs or (b) it can be sold to you for less. There's some additional benefits like fast "boots", strict decoupling of persistant and non-persistant storage (which I find an advantage, restarting a container cleans up whatever you don't care for), but at the end I think it comes down to money.

benjaminwootton 12y ago

A few ways in which this stack can simplify your life: - Abstract away elements of failover, disaster recovery, clustering, load balancing - Abstract your application services cleanly from underlying servers - Give you more consistency across and within development, test and production environments - Allow for immutable deployments - Allow for canary releasing, A/B releasing, rollback etc - Better isolation of processes and services - Move away from general purpose operating systems to lighter weight single purpose OS such as CoreOS Configuration management still has a role to play in this stack, but somewhat smaller than the old world

leef 12y ago

It doesn't. As an average developer it's my public/private cloud providers job to take my images/dockers and schedule them correctly on whatever underlying fleet of hardware they run. I can only assume Tectonic and related tech are aimed at those developers who run hybrid or private clouds for the rest of their company.

brendandburns 12y ago
This is also very true. The purpose of systems like Kubernetes/Mesos is to be set up once by a company, and for most developers to just interact with the CLI tool to run containers. Tools like Google Container Engine, make this into a software as a service product, where the cloud provider provides the API. In that world, as a developer, you never really think about the OS, just your app. But unlike in traditional PaaS, there is no framework that restricts what you can code (language, libraries, etc).
- jacques_chester 12y ago
  > But unlike in traditional PaaS, there is no framework that restricts what you can code (language, libraries, etc). I'm not sure how traditional PaaSes are limited in that sense. Heroku accepts a wide variety of software through the buildpacks mechanism. Cloud Foundry supports both buildpacks and docker images, with clean extension points to add further mechanisms -- for example, .NET-on-Windows deployments, which is under development. Disclaimer: I've worked on Cloud Foundry and I'm employed by Pivotal.
- ajessup 12y ago
  The main difference between an PaaS like Heroku and a general purpose cluster manager service like GKE is that the former can simplify my making certain assumptions about your workload. For example if a service knows your container is serving a web application then it can sensibly provision load balancers, DNS, HTTPS, auto-scaling, static content caching, automatic QPS monitoring etc. to support your app with little explicit configuration. And with a commercial service you get an SLA for those things. You can get much of this with a general purpose cluster as well, but of course you need to configure it yourself, and more importantly - debug it when it goes awry. Disclaimer: I work for Google

cmelbye 12y ago

There are already a couple pretty good responses, but I would add that all of the technologies you mentioned (along with others in this space) are still evolving and maturing. Docker popularized containers, and now we're seeing the supporting technologies that will help fulfill some of the promises of container-based deployment. Big companies have had this technology internally for years, and now it's starting to get productized. I'm referring to things like schedulers, networking, monitoring, etcd/consul, and more. I personally don't think all of the pieces are there yet to make it as seamless and easy as it could be, but it's coming.