16 ms·

Decommissioning Otto

151 points by zhenjl 10y ago

_bpo 10y ago
It's too bad (and uncharacteristic of mitchellh) that this post is so light on specifics. Were the "previously unknown challenges" simply that not enough people adopted Otto? Or were there actual technical hurdles? The premise of Otto isn't clearly flawed, so it would be interesting to see specific challenges - even if it's just "the problem space is way too big and not enough people wanted it"
- dkarapetyan 10y ago
  The selling point was you just do the bare minimum and otto figures out what other specific things you need to get your project up and running. That's not scope, that's magic.
  - _bpo 10y ago
    Lots of useful, successful things appear to be magic (especially in their selling points). Early Heroku is a great example in this space. I didn't see anything in the initial premise of otto that was technically untenable. We could speculate about the "challenges" - the scope was too wide/unbounded, it was open source, it was a distraction from other company goals, it didn't gain enough early traction - but that's simply speculation without details from the creators.
- mitchellh 10y ago
  I'm happy to answer myself. Previously unknown challenges are just the various facets of building and deploying an application. Its not so much that they were unknown problems so much as the abstraction we designed for doing so proved challenging to solve those problems. Ultimately, Otto was trying to be a masterless PaaS (Heroku, etc.). When you frame it that way and think about all the things you'll have to solve it becomes challenging. On top of that, we always wanted Otto itself to be fairly "thin" and delegate its difficult duties to the rest of the stack. This required us to build a bunch of features we weren't ready to build into our other products OR risk bloating Otto itself. Overall, it was too early for us to do.
  - newsat13 10y ago
    Thanks for your comment/insight. I understand what a PaaS is but what does the 'masterless' qualifier mean?
    - lstamour 10y ago
      Well, in Heroku proper, you feed its git repos an app, it figures out what type of app it is, and applies the right build pack and hosting environment for it. Keeping build packs up to date, keeping all the scripts running, and making sure an app has associated dependencies, etc--I imagine that's the difference between an independent setup you can self-host quickly and easily and one that's very dependent on an ecosystem of Heroku maintainers, tooling and existing server infrastructure... It's a very hard problem to solve, and one which will likely only catch on as devops tooling improves and becomes expected for apps, and as app runtimes standardize. Alternatively, you could look at the myriad ways operating systems package applications, and the ways applications allow themselves to be packaged, let alone store data in production, and ... basically give up on this ever happening in an easy, hands-free automated way.
      - jacques_chester 10y ago
        > Keeping build packs up to date, keeping all the scripts running, and making sure an app has associated dependencies, etc--I imagine that's the difference between an independent setup you can self-host quickly and easily and one that's very dependent on an ecosystem of Heroku maintainers, tooling and existing server infrastructure... I work for Pivotal on the Cloud Foundry buildpacks team. 4 of our buildpacks (Ruby, Python, Go, NodeJS) are downstream forks of Heroku's. We merge from upstream approximately weekly, but the pace has definitely dropped. We build all the runtime binaries we ship with our buildpacks. We also build the rootfs it all runs on. Some of these pipelines are now fully automated. For example, when a NodeJS tag lands, our pipeline will build the binary, add it to a buildpack and put it through our battery of test suites. Our product manager can make a release with a few keystrokes and a button press. The difficulty of engineering really comes down to the nature of the ecosystem you're turning into a buildpack. We did an article on writing buildpacks[0], taking Rust as our example. It was a doddle, because of Cargo. Meanwhile our PHP buildpack performs incredible gymnastics to make a 12-factor cloud platform look like a shared host circa 1999. [0] http://engineering.pivotal.io/post/creating-a-custom-buildpack/ http://engineering.pivotal.io/post/creating-a-custom-buildpa...
ploxiln 10y ago
Otto, the successor to Vagrant 772 points agonzalezro a year ago 177 comments (https://www.hashicorp.com/blog/otto.html) Me at the time: Are you kidding?! Me now: Oh, I even forgot about that, the world of software engineering isn't always insane after all, I should lighten up ...
- yeukhon 10y ago
  I don't know why people are downvoting you. I seriously think this should be at the top. The trend of complete replacement but discontinue about a year later is very discouraging trying anything new.
manojlds 10y ago
Considering the "progress" they were making, and the very lofty goals, this was expected.
rdtsc 10y ago
They did a good job at trying something, it didn't work as expected and they were honest and decided to focus on something else. Those are all very good qualities. I think that's commendable.
Mizza 10y ago
Does anybody _actually_ use the HashiCorp stack, besides Vagrant, for serious work? I tried and honestly found their products sorely, sorely lacking. Very shiny documentation, very incomplete, un-battled-tested tools, no examples given, little response from their devs other than the PR team. For a small example, I _still_ get +1 notifications on this critical issue nearly every day: https://github.com/mitchellh/packer/issues/409 https://github.com/mitchellh/packer/issues/409 - no response from dev team whatsoever.
- code_research 10y ago
  unfortunately even vagrant is buggy and very slow and I would like to see a better alternative. consul and vault are very nice for devops.
- morgante 10y ago
  I use and love Terraform. Even for small projects, I find it a lot better for provisioning AWS resources than any of the alternatives.
  - Mizza 10y ago
    Have you tried Troposphere? I compared the two and found I liked Tropo much much more.
  - nixgeek 10y ago
    Seemingly every couple of days, someone on Twitter whom I follow will exclaim "Oh noes, Terraform did something bad". Seems like a 'principle of least surprise' violation. Is that something others have observed?
    - morgante 10y ago
      I've never run into that myself. I do always inspect the plan before applying, but fortunately Terraform makes that easy. Personally, I prefer it to alternatives like Boto/Troposphere because it's fully declarative and not coupled to a single cloud.
fn 10y ago
I still use Vagrant every day. It's great for keeping separate projects really truly separated.
adamb 10y ago
Kudos to HashiCorp for realizing that the complexity of the project was getting away from them and for having the guts to pull the plug in such a public way.
- jacques_chester 10y ago
  Hear hear. I work for Pivotal on the fringes of Cloud Foundry. Between us and other Cloud Foundry Foundation members there are around 200 full time engineers working on it. Featuresome, robust, industrial-grade cloud platforms are hard.
deleted 10y ago
[deleted]
je42 10y ago
Anybody using vault ?
- otterley 10y ago
  Yes.
apeace 10y ago
Question for mitchellh or others at Hashicorp: Will "zero configuration" be a feature/goal in your next attempts at this abstraction? Was this one of the things that made building Otto so challenging? A few months ago when I tried Otto I found the "zero configuration" idea off-putting. In fact, I couldn't even get a basic Python app to work because there was no way for me to install libmysqlclient[0]. There really was no way to configure anything in Otto. FWIW, we ended up using Convox[1] and loved it. [0] https://webcache.googleusercontent.com/search?q=cache:9QfCNXO52GsJ:https://github.com/hashicorp/otto/issues/400+&cd=1&hl=en&ct=clnk&gl=us https://webcache.googleusercontent.com/search?q=cache:9QfCNX... [1] https://convox.com/ https://convox.com/
- mitchellh 10y ago
  I'm still a big fan of minimal configuration through sane defaults, but not advocating "zero configuration" in that sense that you have no power to change those defaults. One area I think we've done really well with this in the past year is the "-dev" flag on Vault, Consul, Nomad. It is a zero-configuration way to get a fully functional dev server up and running in one command though you can still specify a config if you wanted. For non-dev, I just don't want to set dozens of options just to get going, so we'll continue to strive for defaults that work where we can. All that being said, they are defaults, so you can always change them.

151 points by zhenjl 10y ago

_bpo 10y ago

It's too bad (and uncharacteristic of mitchellh) that this post is so light on specifics. Were the "previously unknown challenges" simply that not enough people adopted Otto? Or were there actual technical hurdles? The premise of Otto isn't clearly flawed, so it would be interesting to see specific challenges - even if it's just "the problem space is way too big and not enough people wanted it"

dkarapetyan 10y ago
The selling point was you just do the bare minimum and otto figures out what other specific things you need to get your project up and running. That's not scope, that's magic.
- _bpo 10y ago
  Lots of useful, successful things appear to be magic (especially in their selling points). Early Heroku is a great example in this space. I didn't see anything in the initial premise of otto that was technically untenable. We could speculate about the "challenges" - the scope was too wide/unbounded, it was open source, it was a distraction from other company goals, it didn't gain enough early traction - but that's simply speculation without details from the creators.
mitchellh 10y ago
I'm happy to answer myself. Previously unknown challenges are just the various facets of building and deploying an application. Its not so much that they were unknown problems so much as the abstraction we designed for doing so proved challenging to solve those problems. Ultimately, Otto was trying to be a masterless PaaS (Heroku, etc.). When you frame it that way and think about all the things you'll have to solve it becomes challenging. On top of that, we always wanted Otto itself to be fairly "thin" and delegate its difficult duties to the rest of the stack. This required us to build a bunch of features we weren't ready to build into our other products OR risk bloating Otto itself. Overall, it was too early for us to do.
- newsat13 10y ago
  Thanks for your comment/insight. I understand what a PaaS is but what does the 'masterless' qualifier mean?
  - lstamour 10y ago
    Well, in Heroku proper, you feed its git repos an app, it figures out what type of app it is, and applies the right build pack and hosting environment for it. Keeping build packs up to date, keeping all the scripts running, and making sure an app has associated dependencies, etc--I imagine that's the difference between an independent setup you can self-host quickly and easily and one that's very dependent on an ecosystem of Heroku maintainers, tooling and existing server infrastructure... It's a very hard problem to solve, and one which will likely only catch on as devops tooling improves and becomes expected for apps, and as app runtimes standardize. Alternatively, you could look at the myriad ways operating systems package applications, and the ways applications allow themselves to be packaged, let alone store data in production, and ... basically give up on this ever happening in an easy, hands-free automated way.
    - jacques_chester 10y ago
      > Keeping build packs up to date, keeping all the scripts running, and making sure an app has associated dependencies, etc--I imagine that's the difference between an independent setup you can self-host quickly and easily and one that's very dependent on an ecosystem of Heroku maintainers, tooling and existing server infrastructure... I work for Pivotal on the Cloud Foundry buildpacks team. 4 of our buildpacks (Ruby, Python, Go, NodeJS) are downstream forks of Heroku's. We merge from upstream approximately weekly, but the pace has definitely dropped. We build all the runtime binaries we ship with our buildpacks. We also build the rootfs it all runs on. Some of these pipelines are now fully automated. For example, when a NodeJS tag lands, our pipeline will build the binary, add it to a buildpack and put it through our battery of test suites. Our product manager can make a release with a few keystrokes and a button press. The difficulty of engineering really comes down to the nature of the ecosystem you're turning into a buildpack. We did an article on writing buildpacks[0], taking Rust as our example. It was a doddle, because of Cargo. Meanwhile our PHP buildpack performs incredible gymnastics to make a 12-factor cloud platform look like a shared host circa 1999. [0] http://engineering.pivotal.io/post/creating-a-custom-buildpack/ http://engineering.pivotal.io/post/creating-a-custom-buildpa...

ploxiln 10y ago

Otto, the successor to Vagrant 772 points agonzalezro a year ago 177 comments (https://www.hashicorp.com/blog/otto.html) Me at the time: Are you kidding?! Me now: Oh, I even forgot about that, the world of software engineering isn't always insane after all, I should lighten up ...

yeukhon 10y ago
I don't know why people are downvoting you. I seriously think this should be at the top. The trend of complete replacement but discontinue about a year later is very discouraging trying anything new.

manojlds 10y ago

Considering the "progress" they were making, and the very lofty goals, this was expected.

rdtsc 10y ago

They did a good job at trying something, it didn't work as expected and they were honest and decided to focus on something else. Those are all very good qualities. I think that's commendable.

Mizza 10y ago

Does anybody _actually_ use the HashiCorp stack, besides Vagrant, for serious work? I tried and honestly found their products sorely, sorely lacking. Very shiny documentation, very incomplete, un-battled-tested tools, no examples given, little response from their devs other than the PR team. For a small example, I _still_ get +1 notifications on this critical issue nearly every day: https://github.com/mitchellh/packer/issues/409 https://github.com/mitchellh/packer/issues/409 - no response from dev team whatsoever.

code_research 10y ago
unfortunately even vagrant is buggy and very slow and I would like to see a better alternative. consul and vault are very nice for devops.
morgante 10y ago
I use and love Terraform. Even for small projects, I find it a lot better for provisioning AWS resources than any of the alternatives.
- Mizza 10y ago
  Have you tried Troposphere? I compared the two and found I liked Tropo much much more.
- nixgeek 10y ago
  Seemingly every couple of days, someone on Twitter whom I follow will exclaim "Oh noes, Terraform did something bad". Seems like a 'principle of least surprise' violation. Is that something others have observed?
  - morgante 10y ago
    I've never run into that myself. I do always inspect the plan before applying, but fortunately Terraform makes that easy. Personally, I prefer it to alternatives like Boto/Troposphere because it's fully declarative and not coupled to a single cloud.

fn 10y ago

I still use Vagrant every day. It's great for keeping separate projects really truly separated.

adamb 10y ago

Kudos to HashiCorp for realizing that the complexity of the project was getting away from them and for having the guts to pull the plug in such a public way.

jacques_chester 10y ago
Hear hear. I work for Pivotal on the fringes of Cloud Foundry. Between us and other Cloud Foundry Foundation members there are around 200 full time engineers working on it. Featuresome, robust, industrial-grade cloud platforms are hard.

deleted 10y ago

[deleted]

je42 10y ago

Anybody using vault ?

otterley 10y ago
Yes.

apeace 10y ago

Question for mitchellh or others at Hashicorp: Will "zero configuration" be a feature/goal in your next attempts at this abstraction? Was this one of the things that made building Otto so challenging? A few months ago when I tried Otto I found the "zero configuration" idea off-putting. In fact, I couldn't even get a basic Python app to work because there was no way for me to install libmysqlclient[0]. There really was no way to configure anything in Otto. FWIW, we ended up using Convox[1] and loved it. [0] https://webcache.googleusercontent.com/search?q=cache:9QfCNXO52GsJ:https://github.com/hashicorp/otto/issues/400+&cd=1&hl=en&ct=clnk&gl=us https://webcache.googleusercontent.com/search?q=cache:9QfCNX... [1] https://convox.com/ https://convox.com/

mitchellh 10y ago
I'm still a big fan of minimal configuration through sane defaults, but not advocating "zero configuration" in that sense that you have no power to change those defaults. One area I think we've done really well with this in the past year is the "-dev" flag on Vault, Consul, Nomad. It is a zero-configuration way to get a fully functional dev server up and running in one command though you can still specify a config if you wanted. For non-dev, I just don't want to set dozens of options just to get going, so we'll continue to strive for defaults that work where we can. All that being said, they are defaults, so you can always change them.