4 ms·

Bosh: Tool for deploying and monitoring distributed systems

66 points by phantom_oracle 10y ago

jungturk 10y ago
This is a component of Pivotal's CloudFoundry stack, no? Good to see more of those components being open sourced. Unclear until further review how much of the secret sauce has been retained from the project.
- jacques_chester 10y ago
  A few corrections: 1. It was originally developed to deploy and update Cloud Foundry, but it can be used to deploy and update any stateful distributed system. At Pivotal we're using it to build out Cloud Foundry services for Redis, Cassandra, RabbitMQ, Gemfire, MySQL and some others I'm not sure if I am allowed to talk about. 2. BOSH doesn't belong to Pivotal. It belongs to the Cloud Foundry foundation. We don't keep any secret sauce back from it. We use stock-standard BOSH to update our own public cloud, Pivotal Web Services[0]. Most of the time nobody notices that we upgraded an entire PaaS underneath them. Disclosure: as you've probably guessed, I work for Pivotal. [0] http://run.pivotal.io/ http://run.pivotal.io/
i_have_to_speak 10y ago
Interesting. The tutorial [1] is a easier place to start. Too many abstractions to learn though. Also, can it deploy/setup standard software like StackDeploy [2] can? [1] http://mariash.github.io/learn-bosh http://mariash.github.io/learn-bosh [2] https://stackdeploy.com/ https://stackdeploy.com/
- jacques_chester 10y ago
  > Also, can it deploy/setup standard software like StackDeploy [2] can? Insofar as it can drive an underlying IaaS, yes. It looks as though StackDeploy doesn't do anything past the (admittedly very cool) GUI and first application. BOSH is driven out of two kinds of document: "releases", which define a catalog of software components, and "deployments", which describe a mapping of release components to systems and networks. BOSH can then take those and apply them to various IaaSes through "Cloud Provider Interfaces", or CPIs. There are mature CPIs for OpenStack, AWS and vSphere. Google have contributed a CPI for GCP and Microsoft have contributed one for Azure. We have more CPIs on the radar. Here's the other thing. Once you've deployed, that's not the end of the story. BOSH gives you a range of tools for inspecting and repairing the deployment. You can ask BOSH "are all my components running?", you can ask after the status of a particular component and so on. You can also restart various components or re-roll the whole deployment. There are mechanisms to make this process automatic, but they're not commonly used -- most operators like to make that decision themselves. Upgrading is essentially based on that compare-and-repair mechanism. You upload a new release, then redeploy. BOSH will detect the changes and apply them progressively, including waiting on canary copies and rolling out the changes in batches (all configurable). Similarly, if you want to scale up or down, or change various topologies or settings, you modify the manifest and ask BOSH to do its thing. BOSH also has first-class support in Concourse[0][1,2,3]. The practical upshot is that a lot of teams at Pivotal have fully versioned histories of their deployments, how those deployments went, what was in them etc. Disclosure: I work for Pivotal as an engineer. [0] https://concourse.ci https://concourse.ci [1] https://github.com/concourse/bosh-deployment-resource https://github.com/concourse/bosh-deployment-resource [2] https://github.com/concourse/bosh-io-release-resource https://github.com/concourse/bosh-io-release-resource [3] https://github.com/concourse/bosh-io-stemcell-resource https://github.com/concourse/bosh-io-stemcell-resource

This is a component of Pivotal's CloudFoundry stack, no? Good to see more of those components being open sourced. Unclear until further review how much of the secret sauce has been retained from the project.

jacques_chester 10y ago
A few corrections: 1. It was originally developed to deploy and update Cloud Foundry, but it can be used to deploy and update any stateful distributed system. At Pivotal we're using it to build out Cloud Foundry services for Redis, Cassandra, RabbitMQ, Gemfire, MySQL and some others I'm not sure if I am allowed to talk about. 2. BOSH doesn't belong to Pivotal. It belongs to the Cloud Foundry foundation. We don't keep any secret sauce back from it. We use stock-standard BOSH to update our own public cloud, Pivotal Web Services[0]. Most of the time nobody notices that we upgraded an entire PaaS underneath them. Disclosure: as you've probably guessed, I work for Pivotal. [0] http://run.pivotal.io/ http://run.pivotal.io/

i_have_to_speak 10y ago

Interesting. The tutorial [1] is a easier place to start. Too many abstractions to learn though. Also, can it deploy/setup standard software like StackDeploy [2] can? [1] http://mariash.github.io/learn-bosh http://mariash.github.io/learn-bosh [2] https://stackdeploy.com/ https://stackdeploy.com/

jacques_chester 10y ago
> Also, can it deploy/setup standard software like StackDeploy [2] can? Insofar as it can drive an underlying IaaS, yes. It looks as though StackDeploy doesn't do anything past the (admittedly very cool) GUI and first application. BOSH is driven out of two kinds of document: "releases", which define a catalog of software components, and "deployments", which describe a mapping of release components to systems and networks. BOSH can then take those and apply them to various IaaSes through "Cloud Provider Interfaces", or CPIs. There are mature CPIs for OpenStack, AWS and vSphere. Google have contributed a CPI for GCP and Microsoft have contributed one for Azure. We have more CPIs on the radar. Here's the other thing. Once you've deployed, that's not the end of the story. BOSH gives you a range of tools for inspecting and repairing the deployment. You can ask BOSH "are all my components running?", you can ask after the status of a particular component and so on. You can also restart various components or re-roll the whole deployment. There are mechanisms to make this process automatic, but they're not commonly used -- most operators like to make that decision themselves. Upgrading is essentially based on that compare-and-repair mechanism. You upload a new release, then redeploy. BOSH will detect the changes and apply them progressively, including waiting on canary copies and rolling out the changes in batches (all configurable). Similarly, if you want to scale up or down, or change various topologies or settings, you modify the manifest and ask BOSH to do its thing. BOSH also has first-class support in Concourse[0][1,2,3]. The practical upshot is that a lot of teams at Pivotal have fully versioned histories of their deployments, how those deployments went, what was in them etc. Disclosure: I work for Pivotal as an engineer. [0] https://concourse.ci https://concourse.ci [1] https://github.com/concourse/bosh-deployment-resource https://github.com/concourse/bosh-deployment-resource [2] https://github.com/concourse/bosh-io-release-resource https://github.com/concourse/bosh-io-release-resource [3] https://github.com/concourse/bosh-io-stemcell-resource https://github.com/concourse/bosh-io-stemcell-resource