4 ms·

I'll seed what will quickly be an enthusiastic crowd of spruikers for various platforms: have you looked at Cloud Foundry? It's mature, has been scaled to 10,0

I'll seed what will quickly be an enthusiastic crowd of spruikers for various platforms: have you looked at Cloud Foundry? It's mature, has been scaled to 10,000 real app containers in a single install with a 250,000-container milestone coming soon, belongs to an independent foundation instead of a single company, runs on AWS, Azure, GCP, OpenStack, vSphere with more coming. It handles routing, service injection, logging, container orchestration and a whole bunch of other bits and bobs. You can get started by just pushing your sourcecode, or you can provide container images, whichever works better for you. We even made a version you can test drive on your laptop[0] and we've been working on a "bootloader" for spinning up a full AWS version quickly[1]. Disclaimer: by "we" I mean Pivotal, my employer, which donates the majority of engineering effort to Cloud Foundry. [0] http://pivotal.io/pcf-dev http://pivotal.io/pcf-dev [1] https://github.com/pivotal-cf-experimental/bosh-bootloader https://github.com/pivotal-cf-experimental/bosh-bootloader

Veratyr 10y ago
Back when I looked at CloudFoundry (years ago now), it didn't work out because there was no middle ground deployment between a single PC and a full blown production deployment. You either ran it on one machine or you spun up what would have costed ~$1200 in servers at the time. It doesn't appear this has changed: http://docs.pivotal.io/pivotalcf/1-7/customizing/openstack.html http://docs.pivotal.io/pivotalcf/1-7/customizing/openstack.h... Even at a cheap provider like DigitalOcean, this is 16x10 + 6x160 = $1120/month in __overhead__, without even running any useful applications.
- jacques_chester 10y ago
  I can see your point, insofar at CF is biased to provide HA for all its foundational components. It's actually possible to consolidate it down to fewer machines if you want. But it requires BOSH knowledge and the willingness to surrender the HA guarantees. If you only need to deploy a handful of apps, it'd be easier to start with a hosted service and go from there. We have one in Pivotal Web Services[0], pricing is based on RAM usage. IBM have BlueMix, there are also service installations managed by CenturyLink, AnyNines and Swisscom. [0] https://run.pivotal.io/ https://run.pivotal.io/
  - Veratyr 10y ago
    BOSH knowledge isn't easy to come by unfortunately and I don't think the problem is HA for the foundational components so much as there being so many foundational components. Look at a HA Kubernetes cluster. You've got kubelets and 2-3 masters and you're done. Look at CF and you need 22 servers total just to get started. It may be easier/cheaper to use a hosted service compared to setting up your own CloudFoundry cluster but it's still extremely expensive and doesn't compete with alternatives like hosted Kubernetes. Say I want 32GB total of apps with 8 CPUs for a few development environments. For CF I pay ~$1600, for Pivotal I pay ~$700, for self hosted Kubernetes I pay ~$400 (4x8GB DigitalOcean instances + 3x2GB for masters) and for managed Kubernetes I pay ~$205 (4x n1-standard-2 Google Compute Engine instances + free cluster management since under 6 nodes).
    - jacques_chester 10y ago
      I'm not sure it's apples to oranges, though. Out of the box Kubernetes provides some, but not all, of what Cloud Foundry provides. When you say "so many foundational components", you're right: CF is featuresome and opinionated. That's the whole idea. With Kubernetes you can absolutely assemble all the other parts yourself and wire them together to taste, but that will chew up some of your 32Gb. And our calculations should expand to include engineering time, which is, last time I checked, quite expensive. I'll definitely give Google due credit for competitive pricing though. For that price, and given their expertise, it'd be silly to self-host Kubernetes if you could use GCP.
hosh 10y ago
One of the things I liked about CF was how it encoded application dependencies into the app package. Chef has just recently announced Habitat, which apparently brings that kind of an idea into play. The thing is though, Kubernetes is a set of lego blocks from which you can compose into CF, but you couldn't create Kubernetes from CF. That makes Kubernetes more interesting to me.
- jacques_chester 10y ago
  I think that's a fair call. CF is an already assembled kit and we're constantly looking for ways to have better and better fit-and-finish. The whole point is to allow operators and developers to have smooth "Day 0", "Day 1" and "Day 2" experiences. For those who really want a prebuilt Kubernete PaaS, Red Hat built OpenShift 3. That said, Diego (our scheduler tech) was deliberately architected to be ignorant of Cloud Foundry. Right now it's pretty much serving the historical bias of Cloud Foundry's initial architecture -- applications -- but as the v3 API arrives we're breaking it down into finer grained concepts, some familiar to Kubernetes folk. At one point, Diego was extracted out into a Kubernetes-like package with the Lattice project, which Pivotal supported for a while. But once we went and talked to developers, they said "well I don't really trust it because it's not a real Cloud Foundry". Hence we pivoted to PCFDev. If Docker and Kubernetes or Mesos had emerged earlier, it's likely we'd have used them as our building blocks (as we've adopted other components). But when Cloud Foundry was born, none of these components really existed outside of private systems owned by Google or Facebook, or in some respects at Heroku. Disclaimer: I was not responsible for any of these decisions, or even near to them, so take it all of the above with a grain of salt.
  - hosh 10y ago
    Yep, I remember when CF first came out. It was really interesting tech.
    - jacques_chester 10y ago
      It still is! :)

Veratyr 10y ago

Back when I looked at CloudFoundry (years ago now), it didn't work out because there was no middle ground deployment between a single PC and a full blown production deployment. You either ran it on one machine or you spun up what would have costed ~$1200 in servers at the time. It doesn't appear this has changed: http://docs.pivotal.io/pivotalcf/1-7/customizing/openstack.html http://docs.pivotal.io/pivotalcf/1-7/customizing/openstack.h... Even at a cheap provider like DigitalOcean, this is 16x10 + 6x160 = $1120/month in __overhead__, without even running any useful applications.

jacques_chester 10y ago
I can see your point, insofar at CF is biased to provide HA for all its foundational components. It's actually possible to consolidate it down to fewer machines if you want. But it requires BOSH knowledge and the willingness to surrender the HA guarantees. If you only need to deploy a handful of apps, it'd be easier to start with a hosted service and go from there. We have one in Pivotal Web Services[0], pricing is based on RAM usage. IBM have BlueMix, there are also service installations managed by CenturyLink, AnyNines and Swisscom. [0] https://run.pivotal.io/ https://run.pivotal.io/
- Veratyr 10y ago
  BOSH knowledge isn't easy to come by unfortunately and I don't think the problem is HA for the foundational components so much as there being so many foundational components. Look at a HA Kubernetes cluster. You've got kubelets and 2-3 masters and you're done. Look at CF and you need 22 servers total just to get started. It may be easier/cheaper to use a hosted service compared to setting up your own CloudFoundry cluster but it's still extremely expensive and doesn't compete with alternatives like hosted Kubernetes. Say I want 32GB total of apps with 8 CPUs for a few development environments. For CF I pay ~$1600, for Pivotal I pay ~$700, for self hosted Kubernetes I pay ~$400 (4x8GB DigitalOcean instances + 3x2GB for masters) and for managed Kubernetes I pay ~$205 (4x n1-standard-2 Google Compute Engine instances + free cluster management since under 6 nodes).
  - jacques_chester 10y ago
    I'm not sure it's apples to oranges, though. Out of the box Kubernetes provides some, but not all, of what Cloud Foundry provides. When you say "so many foundational components", you're right: CF is featuresome and opinionated. That's the whole idea. With Kubernetes you can absolutely assemble all the other parts yourself and wire them together to taste, but that will chew up some of your 32Gb. And our calculations should expand to include engineering time, which is, last time I checked, quite expensive. I'll definitely give Google due credit for competitive pricing though. For that price, and given their expertise, it'd be silly to self-host Kubernetes if you could use GCP.

hosh 10y ago

One of the things I liked about CF was how it encoded application dependencies into the app package. Chef has just recently announced Habitat, which apparently brings that kind of an idea into play. The thing is though, Kubernetes is a set of lego blocks from which you can compose into CF, but you couldn't create Kubernetes from CF. That makes Kubernetes more interesting to me.

jacques_chester 10y ago
I think that's a fair call. CF is an already assembled kit and we're constantly looking for ways to have better and better fit-and-finish. The whole point is to allow operators and developers to have smooth "Day 0", "Day 1" and "Day 2" experiences. For those who really want a prebuilt Kubernete PaaS, Red Hat built OpenShift 3. That said, Diego (our scheduler tech) was deliberately architected to be ignorant of Cloud Foundry. Right now it's pretty much serving the historical bias of Cloud Foundry's initial architecture -- applications -- but as the v3 API arrives we're breaking it down into finer grained concepts, some familiar to Kubernetes folk. At one point, Diego was extracted out into a Kubernetes-like package with the Lattice project, which Pivotal supported for a while. But once we went and talked to developers, they said "well I don't really trust it because it's not a real Cloud Foundry". Hence we pivoted to PCFDev. If Docker and Kubernetes or Mesos had emerged earlier, it's likely we'd have used them as our building blocks (as we've adopted other components). But when Cloud Foundry was born, none of these components really existed outside of private systems owned by Google or Facebook, or in some respects at Heroku. Disclaimer: I was not responsible for any of these decisions, or even near to them, so take it all of the above with a grain of salt.
- hosh 10y ago
  Yep, I remember when CF first came out. It was really interesting tech.
  - jacques_chester 10y ago
    It still is! :)