21 ms·

Flynn - Open Source PaaS powered by Docker

339 points by danielsiders 13y ago

silasb 13y ago
Just read the FAQ and saw that Flynn is “Super Dokku” Awesome job.
- jahewson 13y ago
  Here's a link: http://progrium.com/blog/2013/06/19/dokku-the-smallest-paas-implementation-youve-ever-seen/ http://progrium.com/blog/2013/06/19/dokku-the-smallest-paas-...
amccloud 13y ago
So it's not open source yet?
- Titanous 13y ago
  It's still in the early stages of development. We're currently fundraising so that we can work on it full time. All code will be pushed to GitHub as we write it.
  - _alex4nder 13y ago
    I don't see this in the FAQ: the magic that makes all of this happen (which you're integrating) is technology built by other developers. Why should we be supporting you to do this work rather than the other developers who broke ground and already did a majority of the heavy lifting?
    - progrium 13y ago
      Docker was a big piece of the puzzle. I worked with DotCloud to make Docker happen and we're involved in that community quite heavily. There's still a lot more work to do, otherwise we would not be bothering with a campaign like this.
      - _alex4nder 13y ago
        Not to discount your work, but the way I see it: the kernel (e.g. cgroups, namespaces) and the corresponding lxc work dwarfs anything docker brought to the table. This project smells like a campaign to get a product funded.
        progrium 13y ago
        Considering LXC support will soon be a module to Docker and not core, I'm not sure you understand what value Docker adds. Just as there is value added all the way up the stack. Regarding the product funding remark, that's not the intention, though with the license anybody could build a product with this project. I'm not convinced that's a bad thing.
  - johnbellone 13y ago
    I'eve been sketching out something that is beginning to sound similar to me. Looking forward to seeing what you've already produced.
  - X4 13y ago
    @titanous Jonathan Rudenberg Are you the only developer on this project currently? Sorry, just found your FAQ is at the bottom of the page and the question I had was on the bottom right corner ;) Just in case someone is too lazy to look it up himself: He's not alone and teamed up with Jeff Lindsay.
    - Titanous 13y ago
      Nope, Jeff Lindsay (http://progrium.com http://progrium.com) is the other half of the team.
      - lifty 13y ago
        Tent and now Flynn. You are building some foundational pillars that have the potential to empower a lot of people. Great work!
        danielsiders 13y ago
        thanks!
- benatkin 13y ago
  I'd like the headline changed. Nothing in the headline or the url makes it sound like it's a funding page.
- progrium 13y ago
  Some of the components or some of the early validation of the types of components are already open source in the project Dokku.
chatmasta 13y ago
And this is why you shouldn't open source the core part of your business (looking at you DotCloud). Sure, Docker is good for the community. But it's also a key competitive advantage for DotCloud. By giving away that advantage, they create opportunities for viable competitors.
- druiid 13y ago
  Indeed. I'm a bit curious as to why DotCloud released Docker? I mean, sure, someone was going to have an easy-to-use container system sooner or later, but for now that is one of the few defining features that DotCloud has going for them... although one of their competitors is successfully using a modified version of Cloud Foundry (AppFog).
  - danielsiders 13y ago
    We're been working with the DotCloud folks on the container problem since long before Docker was public. They've been very supportive of our efforts to build a full featured PaaS based on Docker and knew about it long before they released Docker publicly. They have their own strategy for their company and product, releasing Docker was a very intentional part of that.
- rglullis 13y ago
  Because it's better to have a small slice of a market that you can compete on than nothing on a market that is dominated by other players. This supposed competitive advantage that they would have would mean nothing if they still had to work on top of other IaaS-providers. They would be undercut by the IaaS themselves. By changing the game, they get another angle to work from.
- shykes 13y ago
  Hi, I'm the founder of dotCloud and author of Docker. Projects like Flynn are the reason we opened Docker in the first place. They are a good thing for the docker ecosystem, and they are definitely good for dotCloud. The dirty little secret of PaaS is that there is not, and will never be a single PaaS that fits everyone's needs. With docker we offer a foundation for many PaaSes to engage in a healthy competition while preserving interoperability. So Shopify will always be free to mix and match Flynn's lego bricks with the bricks of other offerings - because they are all Docker-compatible. The authors of Flynn get that, which is why I believe their product will be good and I will be personally sponsoring it :)
jpadilla_ 13y ago
Seems like there's a CSS issue with the sponsor modal http://d.pr/i/Pgko http://d.pr/i/Pgko
- danielsiders 13y ago
  thanks-- fixed a bug where part of the modal was inaccessible on smaller screens
  - jpadilla_ 13y ago
    Awesome, great job! Really looking forward to this.
druiid 13y ago
Very cool! Been looking for something like this since Docker was announced. Basically Cloud Foundry is far too complex for what I/we need but this would fit the bill nicely. One question though, what is going to be 'the cluster' (The site says it distributes work across 'the cluster')? Basically, will it be capable of interfacing with cloud API's or will it just be you manually setting up a group of instances to host the machines to run Docker on top of?
- Titanous 13y ago
  The 'cluster' is a group of nodes running the Docker + Flynn software. It can be anything from a single server to a fleet of EC2 instances to a rack of servers. It would be easy to make a tool that spawns instances running Flynn on your choice of public/private cloud.
  - druiid 13y ago
    Okay, thanks for clarifying. The next question then would be, if you do reach your stretch goals and implement auto-scaling, would you at that point have cloud/MaaS API integration?
    - Titanous 13y ago
      It's possible, we want to make Flynn really easy to use.
  - meskyanichi 13y ago
    So integrating with VPS providers such as Linode and Digital Ocean will be possible? If so, would this be easy to do for end-users? Never got in to Cloud Foundry because it's overly complex and very limiting. Always went with Heroku, but wished I could have a Heroku-style infrastructure on the above mentioned VPS providers for the benefits of running better hardware, being more affordable, being multi-region, etc, etc.
    - Titanous 13y ago
jahewson 13y ago
My main problem with Docker is that container IP addresses and MACs are changed after every container reboot. This means I can't have a database in one container and a web app in another because I have no way for the web app to discover the address of the database server. Seems like this would be an issue for the host-only (private) network?
- cr4zy 13y ago
  This could be solved on AWS with Elastic IP's or OpenStack environments with Floating IP's. Adding this to Docker would be a nice way to allow new machines to be created but to keep the same IP with automatic switchover.
  - shykes 13y ago
    There has been some work around integrating docker iwth Openstack floating IPs already. Check out http://github.com/dotcloud/openstack-docker http://github.com/dotcloud/openstack-docker
- progrium 13y ago
  Docker is in very active development and issues like this get resolved every week. That said, it's not a problem because of the service discovery and routing that would be involved in the Flynn system.
  - jahewson 13y ago
    Yes, Docker is moving quickly but one barrier to resolving this is philosophical: static IPs and private networks are un-Dockerish. Right now it's not obvious what a clean solution would look like. I'd be keen to see how Flynn's discovery and routing develops, it sounds very promising.
    - ascendantlogic 13y ago
      Sounds like a job for some dynamic DNS. That's exactly the sort of thing it's supposed to solve for.
slajax 13y ago
So it's a git post-receive hook that parses a Procfile and starts your app?
- coenhyde 13y ago
  It's probably a pre-receive hook. That's how Dokku works. It's easier to abort the deploy this way.
  - progrium 13y ago
    I don't think they were being serious. :)
  - slajax 13y ago
    Oh really? I've always used post. Can you elaborate or link to something that exemplifies the pre method?
julien421 13y ago
Dokku on Steroids! gg guys!
jonny_eh 13y ago
This looks amazing! I wish I could contribute $20 or so, but looks like $1500 is the minimum?
- danielsiders 13y ago
  Thanks! We're experimenting with a new funding model for open source: publish a spec for a project that's useful to (and can save lots of money for) companies and see who will chip in. We don't think there are enough personal donors for a project like this to raise significant cash (and the sponsorship levels are affordable for most companies) so we want to try sponsorship first. Once that market is saturated, we'll experiment with individual donation options, maybe after the MVP ships
oellegaard 13y ago
I was trying to built something similar, already had a proof of concept ready built on top of components from Dokku. I can't wait to follow this project - I might even pull myself together and learn go!
- coenhyde 13y ago
  So am I. What's your project?
  - oellegaard 13y ago
    https://github.com/KristianOellegaard/puppet-dokku https://github.com/KristianOellegaard/puppet-dokku Lots of uncomitted changs locally. With this announcement, not sure if its worth continuing down this path.
    - josegonzalez 13y ago
      Nope, keep going. Always worth working on your side project!
    - druiid 13y ago
      Puppet-lint ;). But seriously, cool to see a puppet module for dokku as I imagine the author will continue to work on it even with this new project. I'd say keep working on it if you have time!
knodi 13y ago
Heroku I'm looking at you, lower your prices.
- progrium 13y ago
  They give a lot away for free. Too much even. Really it's the addons that get expensive.
  - ricardobeat 13y ago
    Too much?
- methehack 13y ago
  Heroku has been able to charge a premium for ease of use. It's hard to imagine, though, that ease of use will be a competitive advantage much longer. Much of their creativity (and it's substantial) has been in showing the way and it's only a matter of time, it seems to me, before that will be imitated (in a flattering way) by open source projects. Ultimately, it seems, heroku will need to find an additional axis (besides ease of use) to differentiate on. Maybe I'm wrong, though. Apple sure has been able to turn ease of use into a sustainable competitive advantage. In heroku's case, though, there are fewer UI decisions and its audience is super-technical, so I'd be surprised if ease of use alone is sustainable in this domain. We'll see. I want to see them win, to be honest, because I feel like they've done so much good in a providing such an accessible and (at first at least) cheap way for a developer to get his or her stuff out there.
  - progrium 13y ago
    Yeah. And by the way, I love Heroku, if it's not obvious. I'm friends with their engineers and visit their offices occasionally. It's been one of the most inspiring platforms I've seen.
- hcarvalhoalves 13y ago
  I doubt. The prices they command seem attractive to developers with zero infrastructure experience.
jedahan 13y ago
I'd love to see a comparison between this and [deliver](https://github.com/gerhard/deliver https://github.com/gerhard/deliver)
avgp 13y ago
What are the advantages/disadvantages compared to Dokku (http://progrium.com/blog/2013/06/19/dokku-the-smallest-paas-implementation-youve-ever-seen/ http://progrium.com/blog/2013/06/19/dokku-the-smallest-paas-...)? It is also based on Docker, super easy to set up on your own server and: Available today.
- ricardobeat 13y ago
  The Dokku author is one of the guys behind this project.
- progrium 13y ago
  Dokku is sort of a proto-Flynn. It has different, more reachable goals because it's intended for a single host. It solves some of the same problems, just in a non-distributed way. Breaking Dokku down into its components starts providing part of the plan for Flynn. They may even share code in the end, but Dokku will remain focused on simple, single host deployments. Flynn is about a real, distributed system that could be used in production. Since Flynn is really just a packaged collection of components (similar to Dokku), Dokku may just end up being a single host version of Flynn someday.
soapdog 13y ago
I wanted to contribute but I am a single freelance developer, I don't have the amount of money shown on the sponsorship options. I would be great to have a USD 50 option there.
- danielsiders 13y ago
  Thanks-- we definitely plan to explore ways for individuals to support the project.
  - yapcguy 13y ago
    If open source projects had always required $50,000+ to get started, we might not have GNU/Linux today. Why can't you start the project first and then ask for donations later? Are you saying that unless you guys get paid you're not going to work on the project?
    - FooBarWidget 13y ago
      You will not nearly get as much money if you ask for donations later. Certainly not enough to pay for your cost of living. People have the tendency of thinking "hey I can get this for free, and I can get this TODAY. hm those developers are cool. maybe I'll donate some day" Then they download and use the thing. 97% forget about donating and never return. Of the remaining people, they donate maybe $100 or something.
      - thejosh 13y ago
        proof?
        joshcrowder 13y ago
        Human observation. How many open source applications do you use that you have paid for?
        13y ago
mwcampbell 13y ago
I like the fact that Flynn will be written primarily in Go. That should mean very little overhead for this PaaS. I wonder if using Heroku buildpacks is a good idea, though. Something that can more fully take advantage of Docker layers, and in some cases APT packages for language runtimes, would be better IMO.
- shykes 13y ago
  They are not mutually exclusive. An ideal system would honor the Dockerfile when it is present, and attempt to auto-detect the build method when it is not. Buildpacks are great defaults when auto-detecting. That way, you get a nice out-of-box experience (push your code, something magic happens), and when you need more control over your build and dependencies, you can add a Dockerfile. I'm hoping this is what Flynn will do :)
- progrium 13y ago
  Flynn will be designed so Heroku buildpack support is optional.
sciurus 13y ago
"Unlike existing open source PaaS products, Flynn focuses on providing a set of modular components that can be replaced and reused." I wonder if there is anyone here who works on CloudFoundry or OpenShift and could comment on this, or comment on the proposed capabilities of Flynn (https://github.com/flynn/flynn-spec https://github.com/flynn/flynn-spec) compared to the capabilities of their systems.
- druiid 13y ago
  Well I can say that the closest alternative to this would be Cloud Foundry. Their Warden system is what manages the containers: https://github.com/cloudfoundry/warden/tree/master/warden https://github.com/cloudfoundry/warden/tree/master/warden They actually don't use LXC though and implement resource control with Cgroups. That said, Cloud Foundry is extremely difficult to get setup and running in my experience (even with the v2.0 recently released). It sounds like one big differentiating factor is that Flynn would have fewer moving pieces (Cloud Foundry is at least: BOSH, Warden, VCAP?, CF itself, etc.).
  - oellegaard 13y ago
    +1 on the fact CF installation is tough! They recently releases a suite of scripts to help installing it on AWS. Forget about installing it manually on Open Stack (talking from experience). I can only hope Flynn gets the easy install right, so people will be able to try it out. CF is actually very plugable as well - there is a well defined internal API, which you could just add components that listens/publishes to - however, I don't think its encouraged, neither have I seen a single example of it.
    - druiid 13y ago
      Yeah, CF is a very powerful toolkit (as you mention with the API), but part of the power seems to come coupled with the complexity. I've seen some scripts/documentation to install on Openstack (this is the best so far http://www.pistoncloud.com/2013/06/deploying-bosh-cloud-foundry-openstack/ http://www.pistoncloud.com/2013/06/deploying-bosh-cloud-foun...), but still, it's a huge, huge pain.
polvi 13y ago
I'm curious what type of deployment APIs people want. It is the docker rest API + git push for your own code? Should we be using heroku APIs? Are we going to invent something new like OpenStack did?
- cmelbye 13y ago
  Supporting Heroku APIs somehow would be really great.
- progrium 13y ago
  I know why you're asking. :D
  - polvi 13y ago
    I feel like we are early enough into all of this that we might have a chance at standardization... vs the IaaS APIs which have all diverged at this point.
danielsiders 13y ago
Join the discussion in #flynn on freenode
alpb 13y ago
Can't I contribute with my $10? I'll use it personally.
jcastro 13y ago
This is great guys, good to see more people bringing PaaS-like workflow without the baggage. We've been working in this space with Juju (http://juju.ubuntu.com http://juju.ubuntu.com) but we're using standard LXC for containers. Using LXC allows us to support applications that run on multiple processes, and since Juju is also written in Go, It seems like it might be possible for us to work together in some interesting ways. We already have charms for Rack, node.js, Django, etc. already working and we can replicate the example in the video, just on raw LXC, EC2, and OpenStack. And we’ve put a lot of time and energy into managing the relationships between services, which is the big complicated part of a PaaS So, there seems to be pretty good overlap. Have you looked at Juju? Perhaps there are places where you can leverage the heavy lifting that the juju team has already done in creating a “bring your own” kind of PaaS.
- progrium 13y ago
  I really liked Juju. I think we operate on slightly different paradigms / abstractions. Would love to talk more though.
  - markramm 13y ago
    I agree, we have very much approached the problem from that of service administration/management rather than the "build a PaaS"approach. But, I think the "big problem" behind building a heroku style PaaS, is not getting simple scripted deployment to a container -- it is managing relationships between applications, databases, message queues, and other connected services, and I think that is where Juju could be useful to your project. I'd also be happy to talk more about this any time.
- stevvooe 13y ago
  Juju is a great project and you guys are way ahead of the game but the AGPL licensing is problematic for proprietary services built on that platform.
  - girvo 13y ago
    Do you mind explaining that? Would the fact that they use AGPL for Juju matter if you're not hacking on Juju itself? I don't see how it would be an issue, it wouldn't extend to the proprietary software you run on a Juju-setup service, would it? I could be completely wrong, mind you.
opdemand 13y ago
We at OpDemand wish the Flynn crew the best of luck. I expect more Docker-powered PaaS's to sprout up, so I'm glad Solomon and the dotCloud crew are focused so heavily on interoperability. OpDemand will be releasing a very similar open-source project called Deis in the next few weeks. It's an open-source PaaS powered by Docker & Chef, with a Heroku-inspired workflow. We're busy ironing out a few final issues, putting together documentation, quick-start guides, tutorials, etc.. but we are very close to launching our public 0.1.0, which will allow you to deploy your own private PaaS providing you complete control over hosting, backends, proxies and more.
- druiid 13y ago
  Any chance you'll be adding/have Puppet support as well? We have our cart hitched pretty firmly to it at this point. Congrats on the upcoming release!
  - opdemand 13y ago
    Thanks. We're excited to share our take on private PaaS with the open source community. Re: Puppet -- A good chunk of the "orchestration" is done by updating data bags on a Chef Server, and force-converging Chef nodes over SSH. However I know Puppet has similar constructs, so it's definitely possible to fork the Deis controller and Puppetize it. We'd encourage that sort of contribution.
- julien421 13y ago
  sounds good ! Looking fw to test it!
chrisfarms 13y ago
I'd love to know more about how the cluster will organized. Is there going to be some kind of master controller node, or will it be distributed and self organizing somehow? It almost feels as tho Flynn (or maybe something build on top of Flynn) could/should be a whole minimal linux distribution (SmartOS-ish) that could just boot up into RAM, be told to read a cluster config from somewhere and then start working for the cluster accepting Docker services.
- progrium 13y ago
  Yes, you're on to something. :)
echohack 13y ago
Does anyone know anything like this (a self hosted PaaS) for .NET? I'd love to use docker or something similar for .NET applications.
- Titanous 13y ago
  If your app works with Mono, there's no reason why you can't use Docker/Flynn: https://github.com/friism/heroku-buildpack-mono https://github.com/friism/heroku-buildpack-mono
  - echohack 13y ago
    That's a good option. Thank you for your insight there. I can't help but think that this won't really work for many Microsoft shops because the Microsoft CLI is very different from the Mono CLI. The reason I ask is that there's a number of providers of private PaaS solutions that are willing to install their stuff in your organization for Microsoft stack(HP Matrix, I'm looking at you), but they are full of holes and aren't nearly as robust as what Docker/Dokku/Flynn can offer in terms of plugability and modularity. Also, free is a pretty good price.
baq 13y ago
$ wtf is paas
- julien421 13y ago
  http://en.wikipedia.org/wiki/Platform_as_a_service http://en.wikipedia.org/wiki/Platform_as_a_service
vhodges 13y ago
Re: 1500 Minimum, definitely understand the model you're going for, but some of us ARE companies, just really small. I'd be happy to contribute at a lower level (at least $100, possibly up to $500) as I do see the value in this. Dokku is more than sufficient for my needs at the moment, but see Flynn as a natural extension to that.
- danielsiders 13y ago
  Thanks for your interest. Of course we're happy to accept small contributions from companies. I just sent you an email with details. for anyone else who's interested in contributing on behalf of an organization, please email: contact@flynn.io and we'll set you up at whatever amount you feel comfortable with. We'll start accepting personal contributions next week as well.
cabalamat 13y ago
Is there an explain-like-I'm-5 explanation of what this is and why it is useful? Because I've only a very vague idea of what it's about.
- joeblau 13y ago
  Think of a Platform-as-a-service like a web server that is already configured for multiple languages and all you have to do is install your web application and start/restart it. The idea is that you can turn on and off web servers really quickly to do tests and also that it should scale as the amount of work that your web application needs scales.
danielsiders 13y ago
Just added a mailing list form at the bottom of the page for anyone who's interested.
simonpantzare 13y ago
I got interested in building something like this when I saw Docker issue 410[1]. If my understanding is correct Mesos uses Zookeeper to keep a Mesos master available. Zab, Zookeeper's distributed consensus algorithm, is similar to Raft which has a popular Go implementation[2] written by Ben Johnson. Perhaps Flynn agents can use this implementation. 1. https://github.com/dotcloud/docker/issues/410 https://github.com/dotcloud/docker/issues/410 2. https://github.com/benbjohnson/go-raft https://github.com/benbjohnson/go-raft

339 points by danielsiders 13y ago

silasb 13y ago

Just read the FAQ and saw that Flynn is “Super Dokku” Awesome job.

jahewson 13y ago
Here's a link: http://progrium.com/blog/2013/06/19/dokku-the-smallest-paas-implementation-youve-ever-seen/ http://progrium.com/blog/2013/06/19/dokku-the-smallest-paas-...

amccloud 13y ago

So it's not open source yet?

Titanous 13y ago
It's still in the early stages of development. We're currently fundraising so that we can work on it full time. All code will be pushed to GitHub as we write it.
- _alex4nder 13y ago
  I don't see this in the FAQ: the magic that makes all of this happen (which you're integrating) is technology built by other developers. Why should we be supporting you to do this work rather than the other developers who broke ground and already did a majority of the heavy lifting?
  - progrium 13y ago
    Docker was a big piece of the puzzle. I worked with DotCloud to make Docker happen and we're involved in that community quite heavily. There's still a lot more work to do, otherwise we would not be bothering with a campaign like this.
    - _alex4nder 13y ago
      Not to discount your work, but the way I see it: the kernel (e.g. cgroups, namespaces) and the corresponding lxc work dwarfs anything docker brought to the table. This project smells like a campaign to get a product funded.
      - progrium 13y ago
        Considering LXC support will soon be a module to Docker and not core, I'm not sure you understand what value Docker adds. Just as there is value added all the way up the stack. Regarding the product funding remark, that's not the intention, though with the license anybody could build a product with this project. I'm not convinced that's a bad thing.
- johnbellone 13y ago
  I'eve been sketching out something that is beginning to sound similar to me. Looking forward to seeing what you've already produced.
- X4 13y ago
  @titanous Jonathan Rudenberg Are you the only developer on this project currently? Sorry, just found your FAQ is at the bottom of the page and the question I had was on the bottom right corner ;) Just in case someone is too lazy to look it up himself: He's not alone and teamed up with Jeff Lindsay.
  - Titanous 13y ago
    Nope, Jeff Lindsay (http://progrium.com http://progrium.com) is the other half of the team.
    - lifty 13y ago
      Tent and now Flynn. You are building some foundational pillars that have the potential to empower a lot of people. Great work!
      - danielsiders 13y ago
        thanks!
benatkin 13y ago
I'd like the headline changed. Nothing in the headline or the url makes it sound like it's a funding page.
progrium 13y ago
Some of the components or some of the early validation of the types of components are already open source in the project Dokku.

chatmasta 13y ago

And this is why you shouldn't open source the core part of your business (looking at you DotCloud). Sure, Docker is good for the community. But it's also a key competitive advantage for DotCloud. By giving away that advantage, they create opportunities for viable competitors.

druiid 13y ago
Indeed. I'm a bit curious as to why DotCloud released Docker? I mean, sure, someone was going to have an easy-to-use container system sooner or later, but for now that is one of the few defining features that DotCloud has going for them... although one of their competitors is successfully using a modified version of Cloud Foundry (AppFog).
- danielsiders 13y ago
  We're been working with the DotCloud folks on the container problem since long before Docker was public. They've been very supportive of our efforts to build a full featured PaaS based on Docker and knew about it long before they released Docker publicly. They have their own strategy for their company and product, releasing Docker was a very intentional part of that.
rglullis 13y ago
Because it's better to have a small slice of a market that you can compete on than nothing on a market that is dominated by other players. This supposed competitive advantage that they would have would mean nothing if they still had to work on top of other IaaS-providers. They would be undercut by the IaaS themselves. By changing the game, they get another angle to work from.
shykes 13y ago
Hi, I'm the founder of dotCloud and author of Docker. Projects like Flynn are the reason we opened Docker in the first place. They are a good thing for the docker ecosystem, and they are definitely good for dotCloud. The dirty little secret of PaaS is that there is not, and will never be a single PaaS that fits everyone's needs. With docker we offer a foundation for many PaaSes to engage in a healthy competition while preserving interoperability. So Shopify will always be free to mix and match Flynn's lego bricks with the bricks of other offerings - because they are all Docker-compatible. The authors of Flynn get that, which is why I believe their product will be good and I will be personally sponsoring it :)

jpadilla_ 13y ago

Seems like there's a CSS issue with the sponsor modal http://d.pr/i/Pgko http://d.pr/i/Pgko

danielsiders 13y ago
thanks-- fixed a bug where part of the modal was inaccessible on smaller screens
- jpadilla_ 13y ago
  Awesome, great job! Really looking forward to this.

druiid 13y ago

Very cool! Been looking for something like this since Docker was announced. Basically Cloud Foundry is far too complex for what I/we need but this would fit the bill nicely. One question though, what is going to be 'the cluster' (The site says it distributes work across 'the cluster')? Basically, will it be capable of interfacing with cloud API's or will it just be you manually setting up a group of instances to host the machines to run Docker on top of?

Titanous 13y ago
The 'cluster' is a group of nodes running the Docker + Flynn software. It can be anything from a single server to a fleet of EC2 instances to a rack of servers. It would be easy to make a tool that spawns instances running Flynn on your choice of public/private cloud.
- druiid 13y ago
  Okay, thanks for clarifying. The next question then would be, if you do reach your stretch goals and implement auto-scaling, would you at that point have cloud/MaaS API integration?
  - Titanous 13y ago
    It's possible, we want to make Flynn really easy to use.
- meskyanichi 13y ago
  So integrating with VPS providers such as Linode and Digital Ocean will be possible? If so, would this be easy to do for end-users? Never got in to Cloud Foundry because it's overly complex and very limiting. Always went with Heroku, but wished I could have a Heroku-style infrastructure on the above mentioned VPS providers for the benefits of running better hardware, being more affordable, being multi-region, etc, etc.
  - Titanous 13y ago

jahewson 13y ago

My main problem with Docker is that container IP addresses and MACs are changed after every container reboot. This means I can't have a database in one container and a web app in another because I have no way for the web app to discover the address of the database server. Seems like this would be an issue for the host-only (private) network?

cr4zy 13y ago
This could be solved on AWS with Elastic IP's or OpenStack environments with Floating IP's. Adding this to Docker would be a nice way to allow new machines to be created but to keep the same IP with automatic switchover.
- shykes 13y ago
  There has been some work around integrating docker iwth Openstack floating IPs already. Check out http://github.com/dotcloud/openstack-docker http://github.com/dotcloud/openstack-docker
progrium 13y ago
Docker is in very active development and issues like this get resolved every week. That said, it's not a problem because of the service discovery and routing that would be involved in the Flynn system.
- jahewson 13y ago
  Yes, Docker is moving quickly but one barrier to resolving this is philosophical: static IPs and private networks are un-Dockerish. Right now it's not obvious what a clean solution would look like. I'd be keen to see how Flynn's discovery and routing develops, it sounds very promising.
  - ascendantlogic 13y ago
    Sounds like a job for some dynamic DNS. That's exactly the sort of thing it's supposed to solve for.

slajax 13y ago

So it's a git post-receive hook that parses a Procfile and starts your app?

coenhyde 13y ago
It's probably a pre-receive hook. That's how Dokku works. It's easier to abort the deploy this way.
- progrium 13y ago
  I don't think they were being serious. :)
- slajax 13y ago
  Oh really? I've always used post. Can you elaborate or link to something that exemplifies the pre method?

julien421 13y ago

Dokku on Steroids! gg guys!

jonny_eh 13y ago

This looks amazing! I wish I could contribute $20 or so, but looks like $1500 is the minimum?

danielsiders 13y ago
Thanks! We're experimenting with a new funding model for open source: publish a spec for a project that's useful to (and can save lots of money for) companies and see who will chip in. We don't think there are enough personal donors for a project like this to raise significant cash (and the sponsorship levels are affordable for most companies) so we want to try sponsorship first. Once that market is saturated, we'll experiment with individual donation options, maybe after the MVP ships

oellegaard 13y ago

I was trying to built something similar, already had a proof of concept ready built on top of components from Dokku. I can't wait to follow this project - I might even pull myself together and learn go!

coenhyde 13y ago
So am I. What's your project?
- oellegaard 13y ago
  https://github.com/KristianOellegaard/puppet-dokku https://github.com/KristianOellegaard/puppet-dokku Lots of uncomitted changs locally. With this announcement, not sure if its worth continuing down this path.
  - josegonzalez 13y ago
    Nope, keep going. Always worth working on your side project!
  - druiid 13y ago
    Puppet-lint ;). But seriously, cool to see a puppet module for dokku as I imagine the author will continue to work on it even with this new project. I'd say keep working on it if you have time!

knodi 13y ago

Heroku I'm looking at you, lower your prices.

progrium 13y ago
They give a lot away for free. Too much even. Really it's the addons that get expensive.
- ricardobeat 13y ago
  Too much?
methehack 13y ago
Heroku has been able to charge a premium for ease of use. It's hard to imagine, though, that ease of use will be a competitive advantage much longer. Much of their creativity (and it's substantial) has been in showing the way and it's only a matter of time, it seems to me, before that will be imitated (in a flattering way) by open source projects. Ultimately, it seems, heroku will need to find an additional axis (besides ease of use) to differentiate on. Maybe I'm wrong, though. Apple sure has been able to turn ease of use into a sustainable competitive advantage. In heroku's case, though, there are fewer UI decisions and its audience is super-technical, so I'd be surprised if ease of use alone is sustainable in this domain. We'll see. I want to see them win, to be honest, because I feel like they've done so much good in a providing such an accessible and (at first at least) cheap way for a developer to get his or her stuff out there.
- progrium 13y ago
  Yeah. And by the way, I love Heroku, if it's not obvious. I'm friends with their engineers and visit their offices occasionally. It's been one of the most inspiring platforms I've seen.
hcarvalhoalves 13y ago
I doubt. The prices they command seem attractive to developers with zero infrastructure experience.

jedahan 13y ago

I'd love to see a comparison between this and [deliver](https://github.com/gerhard/deliver https://github.com/gerhard/deliver)

avgp 13y ago

What are the advantages/disadvantages compared to Dokku (http://progrium.com/blog/2013/06/19/dokku-the-smallest-paas-implementation-youve-ever-seen/ http://progrium.com/blog/2013/06/19/dokku-the-smallest-paas-...)? It is also based on Docker, super easy to set up on your own server and: Available today.

ricardobeat 13y ago
The Dokku author is one of the guys behind this project.
progrium 13y ago
Dokku is sort of a proto-Flynn. It has different, more reachable goals because it's intended for a single host. It solves some of the same problems, just in a non-distributed way. Breaking Dokku down into its components starts providing part of the plan for Flynn. They may even share code in the end, but Dokku will remain focused on simple, single host deployments. Flynn is about a real, distributed system that could be used in production. Since Flynn is really just a packaged collection of components (similar to Dokku), Dokku may just end up being a single host version of Flynn someday.

soapdog 13y ago

I wanted to contribute but I am a single freelance developer, I don't have the amount of money shown on the sponsorship options. I would be great to have a USD 50 option there.

danielsiders 13y ago
Thanks-- we definitely plan to explore ways for individuals to support the project.
- yapcguy 13y ago
  If open source projects had always required $50,000+ to get started, we might not have GNU/Linux today. Why can't you start the project first and then ask for donations later? Are you saying that unless you guys get paid you're not going to work on the project?
  - FooBarWidget 13y ago
    You will not nearly get as much money if you ask for donations later. Certainly not enough to pay for your cost of living. People have the tendency of thinking "hey I can get this for free, and I can get this TODAY. hm those developers are cool. maybe I'll donate some day" Then they download and use the thing. 97% forget about donating and never return. Of the remaining people, they donate maybe $100 or something.
    - thejosh 13y ago
      proof?
      - joshcrowder 13y ago
        Human observation. How many open source applications do you use that you have paid for?
        13y ago

mwcampbell 13y ago

I like the fact that Flynn will be written primarily in Go. That should mean very little overhead for this PaaS. I wonder if using Heroku buildpacks is a good idea, though. Something that can more fully take advantage of Docker layers, and in some cases APT packages for language runtimes, would be better IMO.

shykes 13y ago
They are not mutually exclusive. An ideal system would honor the Dockerfile when it is present, and attempt to auto-detect the build method when it is not. Buildpacks are great defaults when auto-detecting. That way, you get a nice out-of-box experience (push your code, something magic happens), and when you need more control over your build and dependencies, you can add a Dockerfile. I'm hoping this is what Flynn will do :)
progrium 13y ago
Flynn will be designed so Heroku buildpack support is optional.

sciurus 13y ago

"Unlike existing open source PaaS products, Flynn focuses on providing a set of modular components that can be replaced and reused." I wonder if there is anyone here who works on CloudFoundry or OpenShift and could comment on this, or comment on the proposed capabilities of Flynn (https://github.com/flynn/flynn-spec https://github.com/flynn/flynn-spec) compared to the capabilities of their systems.

druiid 13y ago
Well I can say that the closest alternative to this would be Cloud Foundry. Their Warden system is what manages the containers: https://github.com/cloudfoundry/warden/tree/master/warden https://github.com/cloudfoundry/warden/tree/master/warden They actually don't use LXC though and implement resource control with Cgroups. That said, Cloud Foundry is extremely difficult to get setup and running in my experience (even with the v2.0 recently released). It sounds like one big differentiating factor is that Flynn would have fewer moving pieces (Cloud Foundry is at least: BOSH, Warden, VCAP?, CF itself, etc.).
- oellegaard 13y ago
  +1 on the fact CF installation is tough! They recently releases a suite of scripts to help installing it on AWS. Forget about installing it manually on Open Stack (talking from experience). I can only hope Flynn gets the easy install right, so people will be able to try it out. CF is actually very plugable as well - there is a well defined internal API, which you could just add components that listens/publishes to - however, I don't think its encouraged, neither have I seen a single example of it.
  - druiid 13y ago
    Yeah, CF is a very powerful toolkit (as you mention with the API), but part of the power seems to come coupled with the complexity. I've seen some scripts/documentation to install on Openstack (this is the best so far http://www.pistoncloud.com/2013/06/deploying-bosh-cloud-foundry-openstack/ http://www.pistoncloud.com/2013/06/deploying-bosh-cloud-foun...), but still, it's a huge, huge pain.

polvi 13y ago

I'm curious what type of deployment APIs people want. It is the docker rest API + git push for your own code? Should we be using heroku APIs? Are we going to invent something new like OpenStack did?

cmelbye 13y ago
Supporting Heroku APIs somehow would be really great.
progrium 13y ago
I know why you're asking. :D
- polvi 13y ago
  I feel like we are early enough into all of this that we might have a chance at standardization... vs the IaaS APIs which have all diverged at this point.

danielsiders 13y ago

Join the discussion in #flynn on freenode

alpb 13y ago

Can't I contribute with my $10? I'll use it personally.

jcastro 13y ago

This is great guys, good to see more people bringing PaaS-like workflow without the baggage. We've been working in this space with Juju (http://juju.ubuntu.com http://juju.ubuntu.com) but we're using standard LXC for containers. Using LXC allows us to support applications that run on multiple processes, and since Juju is also written in Go, It seems like it might be possible for us to work together in some interesting ways. We already have charms for Rack, node.js, Django, etc. already working and we can replicate the example in the video, just on raw LXC, EC2, and OpenStack. And we’ve put a lot of time and energy into managing the relationships between services, which is the big complicated part of a PaaS So, there seems to be pretty good overlap. Have you looked at Juju? Perhaps there are places where you can leverage the heavy lifting that the juju team has already done in creating a “bring your own” kind of PaaS.

progrium 13y ago
I really liked Juju. I think we operate on slightly different paradigms / abstractions. Would love to talk more though.
- markramm 13y ago
  I agree, we have very much approached the problem from that of service administration/management rather than the "build a PaaS"approach. But, I think the "big problem" behind building a heroku style PaaS, is not getting simple scripted deployment to a container -- it is managing relationships between applications, databases, message queues, and other connected services, and I think that is where Juju could be useful to your project. I'd also be happy to talk more about this any time.
stevvooe 13y ago
Juju is a great project and you guys are way ahead of the game but the AGPL licensing is problematic for proprietary services built on that platform.
- girvo 13y ago
  Do you mind explaining that? Would the fact that they use AGPL for Juju matter if you're not hacking on Juju itself? I don't see how it would be an issue, it wouldn't extend to the proprietary software you run on a Juju-setup service, would it? I could be completely wrong, mind you.

opdemand 13y ago

We at OpDemand wish the Flynn crew the best of luck. I expect more Docker-powered PaaS's to sprout up, so I'm glad Solomon and the dotCloud crew are focused so heavily on interoperability. OpDemand will be releasing a very similar open-source project called Deis in the next few weeks. It's an open-source PaaS powered by Docker & Chef, with a Heroku-inspired workflow. We're busy ironing out a few final issues, putting together documentation, quick-start guides, tutorials, etc.. but we are very close to launching our public 0.1.0, which will allow you to deploy your own private PaaS providing you complete control over hosting, backends, proxies and more.

druiid 13y ago
Any chance you'll be adding/have Puppet support as well? We have our cart hitched pretty firmly to it at this point. Congrats on the upcoming release!
- opdemand 13y ago
  Thanks. We're excited to share our take on private PaaS with the open source community. Re: Puppet -- A good chunk of the "orchestration" is done by updating data bags on a Chef Server, and force-converging Chef nodes over SSH. However I know Puppet has similar constructs, so it's definitely possible to fork the Deis controller and Puppetize it. We'd encourage that sort of contribution.
julien421 13y ago
sounds good ! Looking fw to test it!

chrisfarms 13y ago

I'd love to know more about how the cluster will organized. Is there going to be some kind of master controller node, or will it be distributed and self organizing somehow? It almost feels as tho Flynn (or maybe something build on top of Flynn) could/should be a whole minimal linux distribution (SmartOS-ish) that could just boot up into RAM, be told to read a cluster config from somewhere and then start working for the cluster accepting Docker services.

progrium 13y ago
Yes, you're on to something. :)

echohack 13y ago

Does anyone know anything like this (a self hosted PaaS) for .NET? I'd love to use docker or something similar for .NET applications.

Titanous 13y ago
If your app works with Mono, there's no reason why you can't use Docker/Flynn: https://github.com/friism/heroku-buildpack-mono https://github.com/friism/heroku-buildpack-mono
- echohack 13y ago
  That's a good option. Thank you for your insight there. I can't help but think that this won't really work for many Microsoft shops because the Microsoft CLI is very different from the Mono CLI. The reason I ask is that there's a number of providers of private PaaS solutions that are willing to install their stuff in your organization for Microsoft stack(HP Matrix, I'm looking at you), but they are full of holes and aren't nearly as robust as what Docker/Dokku/Flynn can offer in terms of plugability and modularity. Also, free is a pretty good price.

baq 13y ago

$ wtf is paas

julien421 13y ago
http://en.wikipedia.org/wiki/Platform_as_a_service http://en.wikipedia.org/wiki/Platform_as_a_service

vhodges 13y ago

Re: 1500 Minimum, definitely understand the model you're going for, but some of us ARE companies, just really small. I'd be happy to contribute at a lower level (at least $100, possibly up to $500) as I do see the value in this. Dokku is more than sufficient for my needs at the moment, but see Flynn as a natural extension to that.

danielsiders 13y ago
Thanks for your interest. Of course we're happy to accept small contributions from companies. I just sent you an email with details. for anyone else who's interested in contributing on behalf of an organization, please email: contact@flynn.io and we'll set you up at whatever amount you feel comfortable with. We'll start accepting personal contributions next week as well.

cabalamat 13y ago

Is there an explain-like-I'm-5 explanation of what this is and why it is useful? Because I've only a very vague idea of what it's about.

joeblau 13y ago
Think of a Platform-as-a-service like a web server that is already configured for multiple languages and all you have to do is install your web application and start/restart it. The idea is that you can turn on and off web servers really quickly to do tests and also that it should scale as the amount of work that your web application needs scales.

danielsiders 13y ago

Just added a mailing list form at the bottom of the page for anyone who's interested.

simonpantzare 13y ago

I got interested in building something like this when I saw Docker issue 410[1]. If my understanding is correct Mesos uses Zookeeper to keep a Mesos master available. Zab, Zookeeper's distributed consensus algorithm, is similar to Raft which has a popular Go implementation[2] written by Ben Johnson. Perhaps Flynn agents can use this implementation. 1. https://github.com/dotcloud/docker/issues/410 https://github.com/dotcloud/docker/issues/410 2. https://github.com/benbjohnson/go-raft https://github.com/benbjohnson/go-raft