4 ms·

@jacques_chester - after your last comment, I did check CF out. It is orders of magnitude harder than k8s for example. To deploy CF on AWS - i need 20 instance

@jacques_chester - after your last comment, I did check CF out. It is orders of magnitude harder than k8s for example. To deploy CF on AWS - i need 20 instances. k8s can happen with a master and slave. I know that you probably do much much more.... but it will be nice to gradually scale out the components. Second .. well BOSH. I see that there are 13K commits to the BOSH repo, so its probably set in stone anyway... but it would be nice to have something more standard (like Chef, Salt or Ansible. Ansible would be the closest thing to BOSH). These are extremely flexible tools... but far more accessible. Third - the big one : Docker. I know that you have the Garden repository to run Docker.... but I dont think its production ready. Cloud Foundry is an incredible piece of technology. But the problem is not that its unsexy... its inaccessible.

jacques_chester 10y ago
Thanks for your frankness. > I know that you probably do much much more.... but it will be nice to gradually scale out the components. Agreed. There are intermediate options. To try a full Cloud Foundry that fits into a laptop, you can install PCFDev[0]. As an intermediate position, you can install BOSH-lite in the cloud. You lose a major advantage of Cloud Foundry, though, which is that it's a distributed system which BOSH can keep alive. Similarly, the default installation for Cloud Foundry uses 20 VMs. You can, if you wish, consolidate these to fewer. It's a matter of copying and pasting some YAML. But not recommended in the interests of robustness. > but it would be nice to have something more standard (like Chef, Salt or Ansible. Ansible would be the closest thing to BOSH). These are extremely flexible tools... but far more accessible. I have not used Ansible or Salt. I did spend time with cfengine, Puppet and Chef a few years back. BOSH's concept of operations is a complete inversion of those tools. cfengine, Puppet or Chef take a system as it is and transform it, through declared goals, into the system you want it to be. This actually means they have to do with a great deal of incidental complexity. Different distros, different versions of distros, different packaging tools and so on and so forth. The BOSH paradigm is: wait, why are you trying to automate the construction of pets? You want cattle. So it starts with a fixed OS base image and installs packages in a fixed, non-negotiable way. At the low level, very inflexible. But it allows BOSH to focus on the business of installing, upgrading, repairing and reconfiguring distributed systems as single systems. The single distributed system is where BOSH starts, not where it was extended to. It's an important distinction. > Docker. I know that you have the Garden repository to run Docker.... but I dont think its production ready. Strictly, Garden can run Docker images. It's an API with driver backends. The original Garden-Linux backend is being replaced by one which uses runC instead. No point replicating engineering effort if we can share it with Docker. But the problem is not that its unsexy... its inaccessible. You're right. We're in the classic position of other heavy, robust tech. Hard to tinker with. I've been jawboning anyone inside Pivotal about this for years. We're making headway -- PCFDev is a start, bosh-bootloader is under active development and BOSH itself is going to get major love. But compared to docker swarm or k8s, it's harder to set up an experimental instance. [0] https://docs.pivotal.io/pcf-dev/ https://docs.pivotal.io/pcf-dev/
- sandGorgon 10y ago
  > The BOSH paradigm is: wait, why are you trying to automate the construction of pets? You want cattle. So it starts with a fixed OS base image and installs packages in a fixed, non-negotiable way. At the low level, very inflexible. But it allows BOSH to focus on the business of installing, upgrading, repairing and reconfiguring distributed systems as single systems. The single distributed system is where BOSH starts, not where it was extended to. It's an important distinction. This is a problem compared to how we are reasoning about systems in the Docker world. You either use an orchestration tool - which doesnt care what the base system is, and only cares whether it is discoverable/alive/dead/etc - or you use a construction tool . The construction tool could be a Dockerfile .. which you massage in shape for the particular OS/config you have... or you use ansible/salt to "transform" the system. For example, k8s uses Salt for orchestration and Docker for construction. BOSH is a merger of both.. and is therefore extremely, extremely opinionated and incestuous. If I may overstep the limits of my intelligence - I would say, it is a product of times when Docker was not invented yet. It is also very hard to work with as a consequence. Just wanted to let you know that.
  - jacques_chester 10y ago
    > If I may overstep the limits of my intelligence - I would say, it is a product of times when Docker was not invented yet. Well, it was, so that makes sense. I actually said ... a year ago now, I think? ... that tools like Puppet and Chef would morph from "discipline my pets" into straight up image builders. They're so much nicer than Dockerfiles -- the latter is severely constrained by the nature stackable filesystems. > It is also very hard to work with as a consequence. Just wanted to let you know that. Out of curiousity, where do you see the seams to cut it open?
    - sandGorgon 10y ago
      I think Docker or runC is pretty much mandatory. Not because it is great tech or anything.. but because of https://hub.docker.com/explore/ https://hub.docker.com/explore/ So BOSH needs to work with my Dockerfile. That's the first step. Second, build a minimal cluster (1 node?) for me with my Dockerfile. All that the cluster is doing is restarting my application if it dies and helping me deploy. And let me scale out gradually from there. Keep BOSH if you want ;)
wlamartin 10y ago
@sandGordon - Hey, I work for Pivotal and am the anchor of the Garden team. Is there anything in particular that has you concerned about Garden? I see jacques mentioned we're moving (very nearly have moved!) everything over to runC backend which is certainly a big step in the right direction but I'd love to get a feel for what else you feel we need to get right before you would consider it "production ready". Feel free to reach me here, in the #garden channel on the Cloud Foundry slack, or any other medium that works for you. Thanks.
- sandGorgon 10y ago
  hi @wlamartin - thanks for replying. It will be great if you can follow the (rather long) parallel thread on this topic. You should have a fairly good idea of how I personally look at this. I wouldnt generalize that this is the problem that a small startup would always see ... but I think it is fairly close. I would say this comment (and its reply) captures the gist very accurately - https://news.ycombinator.com/item?id=12378554 https://news.ycombinator.com/item?id=12378554
  - wlamartin 10y ago
    I did read it, and believe I understood, I just wasn't sure whether to apply your criticisms of CF as a whole to Garden individually, or whether there were additional distinct technical issues that you had concerns about. The good news from a small startup/lone developer/tinkerer is that we have some work coming up in our backlog to make that much easier, including a single downloadable binary with sensible pre-configured defaults so people who don't want to wrestle with BOSH don't have to. (https://www.pivotaltracker.com/story/show/119174859 https://www.pivotaltracker.com/story/show/119174859) While we grew to meet the containerization needs of CF first and foremost, I'd definitely love to see more projects like Concourse (https://concourse.ci https://concourse.ci) make use of Garden. Seeing first hand the dance they do to consume it, we definitely need to up our game if this is to be an attainable goal. Thanks for your feedback!

jacques_chester 10y ago

Thanks for your frankness. > I know that you probably do much much more.... but it will be nice to gradually scale out the components. Agreed. There are intermediate options. To try a full Cloud Foundry that fits into a laptop, you can install PCFDev[0]. As an intermediate position, you can install BOSH-lite in the cloud. You lose a major advantage of Cloud Foundry, though, which is that it's a distributed system which BOSH can keep alive. Similarly, the default installation for Cloud Foundry uses 20 VMs. You can, if you wish, consolidate these to fewer. It's a matter of copying and pasting some YAML. But not recommended in the interests of robustness. > but it would be nice to have something more standard (like Chef, Salt or Ansible. Ansible would be the closest thing to BOSH). These are extremely flexible tools... but far more accessible. I have not used Ansible or Salt. I did spend time with cfengine, Puppet and Chef a few years back. BOSH's concept of operations is a complete inversion of those tools. cfengine, Puppet or Chef take a system as it is and transform it, through declared goals, into the system you want it to be. This actually means they have to do with a great deal of incidental complexity. Different distros, different versions of distros, different packaging tools and so on and so forth. The BOSH paradigm is: wait, why are you trying to automate the construction of pets? You want cattle. So it starts with a fixed OS base image and installs packages in a fixed, non-negotiable way. At the low level, very inflexible. But it allows BOSH to focus on the business of installing, upgrading, repairing and reconfiguring distributed systems as single systems. The single distributed system is where BOSH starts, not where it was extended to. It's an important distinction. > Docker. I know that you have the Garden repository to run Docker.... but I dont think its production ready. Strictly, Garden can run Docker images. It's an API with driver backends. The original Garden-Linux backend is being replaced by one which uses runC instead. No point replicating engineering effort if we can share it with Docker. But the problem is not that its unsexy... its inaccessible. You're right. We're in the classic position of other heavy, robust tech. Hard to tinker with. I've been jawboning anyone inside Pivotal about this for years. We're making headway -- PCFDev is a start, bosh-bootloader is under active development and BOSH itself is going to get major love. But compared to docker swarm or k8s, it's harder to set up an experimental instance. [0] https://docs.pivotal.io/pcf-dev/ https://docs.pivotal.io/pcf-dev/

sandGorgon 10y ago
> The BOSH paradigm is: wait, why are you trying to automate the construction of pets? You want cattle. So it starts with a fixed OS base image and installs packages in a fixed, non-negotiable way. At the low level, very inflexible. But it allows BOSH to focus on the business of installing, upgrading, repairing and reconfiguring distributed systems as single systems. The single distributed system is where BOSH starts, not where it was extended to. It's an important distinction. This is a problem compared to how we are reasoning about systems in the Docker world. You either use an orchestration tool - which doesnt care what the base system is, and only cares whether it is discoverable/alive/dead/etc - or you use a construction tool . The construction tool could be a Dockerfile .. which you massage in shape for the particular OS/config you have... or you use ansible/salt to "transform" the system. For example, k8s uses Salt for orchestration and Docker for construction. BOSH is a merger of both.. and is therefore extremely, extremely opinionated and incestuous. If I may overstep the limits of my intelligence - I would say, it is a product of times when Docker was not invented yet. It is also very hard to work with as a consequence. Just wanted to let you know that.
- jacques_chester 10y ago
  > If I may overstep the limits of my intelligence - I would say, it is a product of times when Docker was not invented yet. Well, it was, so that makes sense. I actually said ... a year ago now, I think? ... that tools like Puppet and Chef would morph from "discipline my pets" into straight up image builders. They're so much nicer than Dockerfiles -- the latter is severely constrained by the nature stackable filesystems. > It is also very hard to work with as a consequence. Just wanted to let you know that. Out of curiousity, where do you see the seams to cut it open?
  - sandGorgon 10y ago
    I think Docker or runC is pretty much mandatory. Not because it is great tech or anything.. but because of https://hub.docker.com/explore/ https://hub.docker.com/explore/ So BOSH needs to work with my Dockerfile. That's the first step. Second, build a minimal cluster (1 node?) for me with my Dockerfile. All that the cluster is doing is restarting my application if it dies and helping me deploy. And let me scale out gradually from there. Keep BOSH if you want ;)

wlamartin 10y ago

@sandGordon - Hey, I work for Pivotal and am the anchor of the Garden team. Is there anything in particular that has you concerned about Garden? I see jacques mentioned we're moving (very nearly have moved!) everything over to runC backend which is certainly a big step in the right direction but I'd love to get a feel for what else you feel we need to get right before you would consider it "production ready". Feel free to reach me here, in the #garden channel on the Cloud Foundry slack, or any other medium that works for you. Thanks.

sandGorgon 10y ago
hi @wlamartin - thanks for replying. It will be great if you can follow the (rather long) parallel thread on this topic. You should have a fairly good idea of how I personally look at this. I wouldnt generalize that this is the problem that a small startup would always see ... but I think it is fairly close. I would say this comment (and its reply) captures the gist very accurately - https://news.ycombinator.com/item?id=12378554 https://news.ycombinator.com/item?id=12378554
- wlamartin 10y ago
  I did read it, and believe I understood, I just wasn't sure whether to apply your criticisms of CF as a whole to Garden individually, or whether there were additional distinct technical issues that you had concerns about. The good news from a small startup/lone developer/tinkerer is that we have some work coming up in our backlog to make that much easier, including a single downloadable binary with sensible pre-configured defaults so people who don't want to wrestle with BOSH don't have to. (https://www.pivotaltracker.com/story/show/119174859 https://www.pivotaltracker.com/story/show/119174859) While we grew to meet the containerization needs of CF first and foremost, I'd definitely love to see more projects like Concourse (https://concourse.ci https://concourse.ci) make use of Garden. Seeing first hand the dance they do to consume it, we definitely need to up our game if this is to be an attainable goal. Thanks for your feedback!