8 ms·
Habitat — A new approach to automation
- otoburb 10y agoThe article link points directly to the Habitat demo. For those of us that don't already know what Habitat is, a better place to start might be the about page: https://www.habitat.sh/about/why-habitat/ https://www.habitat.sh/about/why-habitat/ Habitat seems like an entirely new Chef product to enable what they're calling Application Automation. Relevant quote: "[...] automation must travel with the application, rather than be provided by the platform. Everything the application needs, from build dependencies, run-time dependencies, configuration, dynamic topologies, deployment strategies, secrets management, security auditing, and so on belongs with the application, because we're doing it for the application. Runtime and infrastructure layers exist to support the application, but the efficient building, deployment, and management of the application are decoupled from those layers." Incredibly ambitious.
- evgen 10y agoContainerization is disrupting a lot of their current market, so they need to adapt in some fashion. Given the fact that the only hammer they have at hand is ruby systems automation tooling it is not a big surprise they are aiming at these targets. Unfortunately, IMHO, the scope is a mile wide and an inch deep in terms of what chef brings to the table here. I can walk down that list and either think immediately of another tool that is dedicated to the specific task or the needs are so slight that a few ansible or bash scripts take care of the need. Sounds like they are aiming to occupy some space in the general "tooling & orchestration" arena, but it will be an uphill climb.
- deleted 10y ago[deleted]
- 2461001642 10y agoThis is an excellent point - kind of goes against my earlier comment. Maybe they have to move this way because that's their only choice - if they don't move into automating this, containers will disrupt them (or already has)
- jacques_chester 10y agoComments based on the most trivial skimming of the docs, so I'm probably wrong about everything. > Incredibly ambitious. It is, but ambition is necessary if you want to build distributed production systems. BOSH[0] does something similar in providing an all-or-nothing approach to configuring distributed systems built out of heterogenous software components spread across multiple machines. Note here that the key is the heterogenous part -- if you have an undifferentiated flock of software, you can rely on a different abstraction to deploy them (anything from PaaS to a hand-rolled thing, though I don't recommend hand-rolling this stuff). But very few distributed systems are designed to rely on emergent behaviour. In fact that's usually what we're trying to avoid. Where they appear to differ is that BOSH is quite strict about lifecycle -- you can't "truly" update configurations on-the-fly. You update the relevant YAML and then ask BOSH to update the world for you. It seems like a hassle, but if you couple it with git and Concourse pipelines, you get a complete, externalised history of every system configuration change you've ever made. We already know that this top-down model of heterogenous distributed systems configuration management works better than trying to extend it from the machine up (as Cfengine and its various successors tried to do). BOSH has been deploying large systems for years and years. It was originally designed to deploy and upgrade Cloud Foundry without downtime, something it does with aplomb. What I really like about Habitat over BOSH is the way it exposes the available knobs and dials. That kind of interactive discoverability is super helpful sometimes. I was, however, struck by this claim: > This architecture allows us to avoid the production cliff problem: the closer you get to production, the more difficult it is. This is literally the problem that PaaSes solve for you. I've seen companies without PaaSes, with home-rolled PaaSes and with Cloud Foundry. In the latter case, the biggest obstacle is usually getting VPN access. And there are other PaaSes apart from CF. Habitat seems to be doing a kinda-sorta PaaS here (note the "Supervisor"), but without most of the feature plumbing that makes PaaSes operationally useful: routing, logging, service injection and the like. Disclaimer: I work for Pivotal, we do the majority of the engineering on both BOSH and Cloud Foundry. [0] https://bosh.io/ https://bosh.io/
- willejs 10y agoThis looks really interesting. Im waiting for 8:30PDT for the full announcement though! Im curious about how it works under the hood, it seems to solve 'all the problems' ;) My skepticism comes from some of chefs past product releases, which have seemed rather, half baked. Products like chef metal (provisioning), pushy (push jobs) or delivery... This is in no way negative as I am a long time chef user and fan of chef.
- onion2k 10y agoAs nice as the 'console' on the page is, the fact the window gets cleared when you click to go to the second step makes it look like it lost what you did on the first page. To me that makes it look like it's broken - the steps should maintain the state and build on each other, not start over each time.
- sctb 10y agoWe updated the link from the live demo page (https://www.habitat.sh/try/ https://www.habitat.sh/try/) to the homepage.
- 2461001642 10y agoI love the Chef ppl, but... Is it just me or are there a ton of other folks doing this? The idea of introducing yet another layer of stuff is scary, and there's lots of work left to do on Chef. In a lot of ways, they feel just like Nomad (coincidentally also on the front page)... I wish they would spend some time on their core products versus expanding into an overfilled market.
- rmoriz 10y agoThat's what I thought, too. Chef (software/ecosystem) needs a bit more love IMHO. However it shows that Chef (Company) is still able to design, build and ship a new product, as an organization. That's a very positive sign. Until recently I thought when comparing Chef with Hashicorp: Hashicorp is so much faster in building and releasing usable stuff but they seem to feel the pain in growing too fast and too broad in terms of products. So, in the end: It's a great thing, just also don't forget to put more love into Chef (software) and don't dissipate developers/time/budget :)
- _greim_ 10y agoI'm not sure what I'm looking at. What is being "automated" here? The headline and home page don't give much context.
- otoburb 10y agoThe main points around automation seem to be[1]: * Flexible versioning and configuration rollout * Ability to deploy anywhere * Ease of management Concretely, they're touting their gossip-based configuration and topology management layer as a way to send configuration updates to service groups, and then letting the service groups apply the changes to all the group members. Contrast this with a more traditional way of propagating changes across a network by enumerating every single host/VM/container. I'd imagine another large component of the application automation would come in the form of tooling allowing the ability to deploy anywhere (bare metal vs. VM vs. container vs. IaaS/PaaS providers), but haven't seen that just yet. Maybe it's being covered in the live announcement. [1] https://www.habitat.sh/about/why-package-automation-with-app/ https://www.habitat.sh/about/why-package-automation-with-app...
- _greim_ 10y agoSo but am I automating devices in my home/building IoT-style? Or am I automating deployments of software? Or hardware? "gossip-based configuration and topology management" means nothing to me unless I know actually what the overall context is here.
- otoburb 10y agoFrom what I'm reading, I believe this is primarily geared towards trying to automate software deployments.
- blakeyrat 10y agoThe two-word answer seems to be "software builds". I am always amazed at people's capability to give an extremely complicated answer to an extremely simple question. "What is being automated here?" "Software builds."
- willejs 10y agoTheres a live announcement now https://bignews.chef.io/ https://bignews.chef.io/
- Cshelton 10y agoFound this elsewhere, interesting to note, Habitat looks to be built mostly with Rust https://github.com/habitat-sh/habitat https://github.com/habitat-sh/habitat
- holoway 10y agoThis is Adam - Rust has been amazing to work with, and was the prefect choice for this platform. It's great.
- Cshelton 10y agoYou should reach out to the Rust team if you haven't already. I'm sure many would love a blog post on using Rust. I was thinking about using protocol buffers for something in Rust, it's cool to see an example of a real app using it, thanks!
- holoway 10y agoWe're totally going to do it.
- bjz_ 10y agoCould you elaborate a bit more on why you chose Rust, and why you think it was the right choice for this project?
- jonaf 10y agoOn the heels of this, I've been exploring Rust a bit lately but haven't been able to find any solid patterns / style guides for idiomatic Rust. Do any such resources exist? For example, the official rust book describes Arc, Mutex, etc but.. Are there more of these in the standard library? What purpose do they serve? What are some good idioms to apply when using them? I can surely make them work, but maybe I should be using some other thing?
- bluejekyll 10y ago
- the_arun 10y agoIs this like Docker?
- radhikadesai 10y agoI had the same question
- holoway 10y agoNope! You can use it with docker, and we think it makes the experience better. We love docker.
- SEJeff 10y agoIs there anywhere that actually lays out the tech that powers habitat? ie: This thing uses kubernetes or this thing uses mesos This other thing uses docker/rkt/containerd/lxd This is the set of things we wrote in rust, etc, etc.
- ungzd 10y agoRather like nix + consul.
- vishvananda 10y agoI've been waiting for this announcement for a few weeks. Since the home page is quite lean on an actual explanation, let me give a rough summary of interesting points as i understand it. I apologize in advance if I have anything wrong. 1. It is a from-scratch source build system written in rust that borrows a huge amount from nix. This means repeatable software builds with isolated dependencies. 2. It attempts to move some configuration management primitives into the build pipeline so that the deployment config has enough flexibility for real-world use. 3. It intends to replace single host process supervision with a distributed model based on gossip. This project is most definitely ambitious, and I'm not sure the whole goal is realizable but some aspects are particularly interesting. 1. The build process seems quite useful for building containers, especially if it gets some traction. The main drawback of nix is that no one knows how to use it. If this gets the backing of the hordes of chef-aware system administrators, we may have a good solution for isolated repeatable builds, especially if they focus on deterministic reproducible builds[1] as well. 2. Distributed systems are notoriously hard to configure in a general way. It seems like every company has their own magic combination of config management, monitoring, and scripting for keeping systems like zookeeper running. Its pretty clear that we don't have the right primitives for sharing this work. I don't think a container management system has the right primiteves for this either, because I don't think you can solve for all distributed systems in a generic way. You really need custom logic for the particular software you are trying to distribute. I'm not convinced that a gossip-based process-supervisor is the best approach for this, but it could give us a place to start collaborating on these primitives. [1] https://wiki.debian.org/ReproducibleBuilds https://wiki.debian.org/ReproducibleBuilds
- davexunit 10y ago>1. It is a from-scratch source build system written in rust that borrows a huge amount from nix. This means repeatable software builds with isolated dependencies. Except the builds aren't at all repeatable in a meaningful way, judging by the example code on the home page. The 'do_build' function runs 'npm install'. This means that 1) the build container has network access (i.e. it's nondeterministic by definition) and 2) the declared dependencies do not fully describe the dependency graph. So, I really don't see how much it could possibly borrow from Nix if it threw out the most important part of it.
- canterburry 10y agoHmm, I very much like the concept but when I see the need for supervisors and a habitat service, it makes me wonder if this is really what is says it is. If I need to pre-install all these services on my hosts before any of this will work, then how is that really different from uniformly deploying all my applications on an application server such as WebSphere? An EAR too contains all my configuration data, my bindings, my security settings etc. So...if you uniformly run WebSphere everywhere in your enterprise, it sort of accomplishers the same thing. HOWEVER, if the habitat configuration is capable of talking to my F5 load balancer to create a resource pool, self register my app with my corporate DNS, create a DB schema and populate it with my tables, talk to my networking equipment and open all relevant firewall ports and do so when the application is deployed to a new environment, as well as undo all these actions if it's ever moved elsewhere, then you have piqued my interest...
- GrinningFool 10y agoThere's no pre-installation - the supervisor is bundled with the application deployment itself.
- HerpDerpLerp 10y agoCan I also buy a sofa? http://www.habitat.co.uk/ http://www.habitat.co.uk/
- albasha 10y agoAh, another limiting abstraction layer to learn and to fight. Bash is plenty good.
- davexunit 10y agoBut all of the package recipes are just Bash scripts, which I think is just terrible, btw. Using a real programming language would have been a much better choice.
- holoway 10y agoWe struggled with this! I think we found the right balance, but time will tell.
- zeveb 10y ago> Using a real programming language would have been a much better choice. The problem is that everyone disagrees on what a tolerable real programming language is. Some refuse to use Python; others Ruby; others Scheme; others Lisp. At least every can tolerate shell more or less.
- hrayr 10y agoHome Automation is what first came to mind when I read the title. With this in mind, reading the blurb on the website sounded like, oh they're talking about building home automation software... "build automation, IT automation, deployment automation, etc" would have saved me some confusion. Or simply compare yourself to other solutions out there will help me understand what category this new piece of utility and name fall into.
- reitanqild 10y agoYep. Would made perfect sense for the name as well.
- johnbellone 10y agoThe GitHub project: https://github.com/habitat-sh/habitat https://github.com/habitat-sh/habitat
- mwcampbell 10y agoI wonder if this project is doing too many things. It includes a package builder and a supervisor. Is there any benefit to integrating the two? For a project that's smaller in scope but addresses the part about packaging the automation with the application, Joyent's ContainerPilot [1] looks interesting. [1]: https://www.joyent.com/containerpilot https://www.joyent.com/containerpilot
- usaphp 10y agoThat jumping header is quote annoying, I guess the developer did not calculate a proper padding to accomodate for a sticky menu once it becomes fixed positioned
- BHSPitMonkey 10y agoHow does this compare with Heroku's buildpack system?
- rmcdermo 10y agoI watched the videos, did the tutorial, read the docs, read the tech site stories but I still don't see why I would use Habitat over the current Docker workflow and ecosystem (including third party) for building, shipping and deploying applications. Both are a type of packaging that puts the application, its dependencies and configuration in a bundle that can run exactly the same everywhere. Docker has Dockerfiles and docker-compose.yml files, Habitat has plan files (seems like a shell script, Dockerfiles and compose files are much simpler). Docker has a central repository/registry "docker hub", habitat has depots. "docker run redis" == "hab start core/redis", etc... etc... The docs/articles say habitat apps can run in containers, but why bother if your app can already be run in a container that achieves the same thing with even greater isolation (cgroups). Why put a self contained app in a self contained container application? I'm not trying to put down Habitat, I'm trying to understand its role. As I currently see it, why would I choose this over the more mature Docker environment with a massive established ecosystem and user base? Again I'm just trying to understand what I'm missing, if I'm missing something please point me in the right direction.
- rmcdermo 10y agoI watched the videos, did the tutorial, read the docs, read the tech site stories but I still don't see why I would use Habitat over the current Docker workflow and ecosystem (including third party) for building, shipping and deploying applications. Both are a type of packaging that puts the application, its dependencies and configuration in a bundle that can run exactly the same everywhere. Docker has Dockerfiles and docker-compose.yml files, Habitat has plan files (seems like a shell script, Dockerfiles and compose files are much simpler). Docker has a central repository/registry "docker hub", habitat has depots. "docker run redis" == "hab start core/redis", etc... etc... The docs/articles say habitat apps can run in containers, but why bother if your app can already be run in a container that achieves the same thing with even greater isolation (cgroups). Why put a self contained app in a self contained container application? I'm not trying to put down Habitat, I'm trying to understand its role. As I currently see it, why would I choose this over the more mature Docker environment with a massive established ecosystem and user base? Again I'm just trying to understand what I'm missing, if I'm missing something please point me in the right direction.