6 ms·

Pivotal Cloud Foundry is terrible. I am having a terrible time working with PAS. The use case of PCF is weak especially when Kubernetes does everything it can

Pivotal Cloud Foundry is terrible. I am having a terrible time working with PAS. The use case of PCF is weak especially when Kubernetes does everything it can do. It just sells enterprise fluff to "architects" in suits who haven't touched code in years and are so far from the reality of Devs working day to day. The time it took me to migrate a Java spring boot service from AWS to GCP was around a day. Getting it to work with Pivotals way of doing things was more like a week. We are now starting to do some of our infrastructure on Kubernetes, much less headache. If anyone wants more info feel free to ask. My post might seem angry but as a developer I know better than the execs.

codingminds 8y ago
That's a very interesting point for me. Our company is in discussion with Pivotal to use PAS or PKS on prem. Do you have a specific example or something similar what was the problem ? I've only seen the slides which should sell the products to OPS guys and the idea looked promising.
- mothsonasloth 8y ago
  I can only comment on PAS but the main issues I've experienced are: I have to login to PCF to check environments or add services to bind to an app. The CF cli is not very easy to understand compared to aws eb cli or herokus toolbelt The login screen says "email" but you actually have to enter your user ID Devops become your enemies, especially if they lock down PCF environments. I don't want to beg for a redis tile. PCF eats a lot of ram from our virtual servers. The PCF support is layered with different urgency levels, most of the time they ignore you or are slow to respond Basically I think it's a tool that gets in the way of developers and gives headaches to the platform team. However my company signed a contract with them, so we're tied in for a year.... I might quit before then
  - codingminds 8y ago
    thanks for the answer
  - twic 8y ago
    I remember being in a meeting with a client, with a Pivotal salesman alongside me, when the question of PCF's overhead came up. We managed to work out that a PCF installation required about 10 separate VMs before you even got to the ones that would run applications.
    - latchkey 8y ago
      The point being that those VM can be distributed across multiple hosts for scalability and reliability. I'm sure that if you wanted to really package it all up into a single VM, you could... but that wouldn't make much sense when you're building an onpremis deployment cloud.
      - twic 8y ago
        No, the ten VMs are not there for scalability or reliability, they're there because there are ten teams building the essential components of Cloud Foundry, and the BOSH model is that each component gets its own VM. You could squeeze them onto one VM, but that would require that CF teams cooperate on manifests. Inconceivable!
        latchkey 8y ago
        Maybe you could fathom that the BOSH model is designed the way it is, for scalability and reliability purposes. The CF team is a bunch of Pivots... they cooperate quite well and probably better than most other teams, especially since they rotate across projects fairly often. Disclosure, I worked on CF in the early days, on a number of teams.
    - EngineerBetter 8y ago
      PCF is for those working at scale, and doesn't suit trivial deployments well.
      - twic 8y ago
        Someone really should have told the salesman that, then - they were trying to sell the client on the idea that they could have a separate PCF foundation for each project.
  - insomniacity 8y ago
    I use PCF at work. I’ve noticed most of these but they didn’t really bother me... We have low timeouts on most logins anyway, so having it on PCF isn’t a big deal. We have a CI/CD workflow, so I don’t really have to login to the CLI that much. Locked down environments are always a pain - but it would be no different if your org locked down AWS or Heroku, which they could.
  - EngineerBetter 8y ago
    > Devops become your enemies You're doing devops wrong, which may explain why you're having a bad time with the platform. > I have to login to PCF to check environments or add services to bind to an app. You should automate this with CI servers and pipelines. The complaints about RAM usage and support service I agree with. It sounds like your organisation is disfunctional, and its treatment of PCF is the symptom not the cause.
- nunez 8y ago
  PCF works best with the VMware stack (vSphere + vROPs) hence the sell to Operations. It is an absolute beast to install. You can do a PoC with PCF Dev to see what using it as a developer will be like, but the actual implementation of it will absolutely require Pivotal consultants to get going. PCF itself is also very expensive. I'm not sure if they charge per node or per app; they've changed it over the last few years. I would get the skinny on that before committing to anything and do a TCO/ROI analysis. If you are going to go this route, I would also consider doing a PoC with OpenShift or Rancher. These are managed Kubernetes offerings, both on prem. Both have free open-source editions that mostly mirror what you'll get in the Enterprise offering at smaller scale. Rancher is also less "enterprisey" in that they aren't as aggressive about enterprise support or upsells, at least from what I've heard. I think you'll be locked into big contracts with either option, but Pivotal's lock-in is tighter, at least from what I can tell. This could change after IBM fully moves in, however.
mwcampbell 8y ago
> We are now starting to do some of our infrastructure on Kubernetes, much less headache. What does your build process look like? Can you use CF buildpacks in your Kubernetes cluster? A built-in system for building and deploying your application when you push to a Git repo seems like a real advantage of CF to me.
- fermuch 8y ago
  I've never used CF, but you can git push build to k8s with gitkube Gitkube thread on HN: https://news.ycombinator.com/item?id=16574893 https://news.ycombinator.com/item?id=16574893
  - mothsonasloth 8y ago
    We are not git ops yet but that's what I think we should try to achieve
- pat2man 8y ago
  OpenShift is great for this. For something like a ruby application I can just run `oc new-app .` and I get by current application built, deployed and available externally. All with full kubernetes support. (OpenShift uses https://github.com/openshift/source-to-image https://github.com/openshift/source-to-image which works like CF buildpacks)
jambay 8y ago
i work on pcf at pivotal. i'd love to hear how we can improve. spring boot should work great with "cf push" and when it doesn't we would love to know why and make it better. while pks and k8s are able to run stateless workloads, if pas isn't offering a lot of additional productivity with routing, logging, and multi-tenancy features then we've missed the mark. pivotal-cf-feedback at pivotal dot io will reach the product managers that work on different parts. i saw your note further below about having to login, the "email" prompt instead of username, you had to ask to get the redis tile, and that you find the cf cli confusing compared to other platform clis you are familiar with. we definitely want to know let us know if there are other major issues.
- Something1234 8y ago
  You sound like a suit. Too many acronyms.
  - latchkey 8y ago
    Having worked with James directly many years ago as a developer at/on Pivotal/Cloud Foundry, he is a bit of a suit... but he's also a really great human, very smart and he genuinely cares about the product. Cut him some slack please.
  - dang 8y ago
    Personal attacks will get you banned here. Please review https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and post civilly and substantively, or not at all.
- erikb 8y ago
  I work in a big corp myself. If you worked on cloud foundry you probably worked with some of my colleagues. ;-) Please don't mention the name though, so I can speak openly. I see this in my corp every day. Very smart people, super interested in actual feedback, and super interested in really providing great software. But in the end they still fail. I'm not 100% sure about the reasons, but I have a guess. Usually people who move up the ranks and get into positions where they can make decisions, tend to build relationships with other such people. So more and more they change from "why the hack is dns not working on server-13 anymore?" to "this is the big problem the market has, let's attack that". And slowly that thought croaches in that you should only think about the big picture problems and not worry about the nitty-gritty details of every day getting-things-to-run. And that's probably the problem. While it's clear that you can't solve all the details and work on big picture problems at the same time, you should not devaluate the details. Great solutions solve big picture problems BY SOLVING THE DETAILS. That's why K8S for instance is a mediocre solution. Sure they have great big picture ideas, but usually people spend their time with having DNS not working, having servers hanging, having etcd nodes not talking to each other without explaining why, having deployments say "SUCCESS" while actually not running. K8S provides a standardized API for every detail and every use-case, cool. But in the end you need to be an expert in everything to make it run around 50% of the time (outside of AWS and GCP). Nobody really wants to develop on k8s when the underlying platform is only available 50% of the time. And that's the current experience. So please, if you are one of these people who is really interested in creating great software, consider finding solutions to the detailed low-level problems an important part of the goals and tasks you define. Usually the core low-level problems are not a big amount. For instance with k8s it usually is networking, dynamic storage or security related. Have one person in your team really become an expert in one of these areas (by checking out problems and solutions that exist, not just by making up ppt slides with plans that have nothing to do with the real world) and allow them to influence your task planning.
scothis 8y ago
> We are now starting to do some of our infrastructure on Kubernetes, much less headache. Pivotal Function Service (PFS) builds directly on Kubernetes via Knative (https://github.com/knative/ https://github.com/knative/) and Project riff (https://projectriff.io https://projectriff.io).
- colemickens 8y ago
  What does PFS do that Knative+Riff doesn't get me? Is the title even accurate? It sounds/looks more like "Pivotal creates branded offering of Knative+Riff and uses one of their main selling points as their headline feature", but maybe I'm just biased from past impressions of Pivotal marketing. Especially since it seems to be in contact-only early access.
  - richards 8y ago
    Good question. I work at Pivotal. PFS is a commercial, supported packaging of Riff (which adds dev and ops aspects to Knative). As opposed to running and supporting the OSS components on your own.
erikb 8y ago
> We are now starting to do some of our infrastructure on Kubernetes, much less headache. I bet you do it on AWS or GCP, right? Try on-prem, then check your headackes again. ;-)
jacques_chester 8y ago
> I am having a terrible time working with PAS. Then we are not doing our job. I am happy to talk to you and/or to put you in touch with any team you want to give feedback to.
nunez 8y ago
I've finally gotten the chance to spend time with PCF in an enterprise setting. I've also spent time with Kubernetes (no KNative though, at least not yet) and Docker Datacenter. Unfortunately, I have to agree though I wouldn't call it terrible. My opinion of the product is that it is designed for organizations that are heavily invested in the VMware ecosystem that want to maintain that developer-operator separation through the guise of a PaaS with as little code or ramp-up (or investment from their own people) as possible. As a dev, my gripes with PCF are as follows: 1) PCF is massive. You need at least 20 instances to run it...and that's without any workloads on top of it. (If you are on AWS, the from-scratch installation instructions require that you bump your EC2 instance count minimums to 50 across several different instance sizes. Yikes.) It is very clearly designed for companies that already have a significant VMware footprint in house. Running it on AWS (I don't think Pivotal supports running it on Azure or GCP, but I could be wrong) is a very expensive exercise in futility. You cannot _just_ install PCF on your local machine unless you use PCF Dev or cf-local, both of which remove a lot to provide some semblance of parity to a larger install. (You also can't _just_ install PCF Dev either like you can with minikube or Swarm; you need to have a Pivotal account (free) and then download it from the Pivotal network. This can be automated, but it requires a lot of steps.) This leads me to... 2) There's no real way to test that code on your laptop will work in a production PCF deployment since the infrastructure they run on top of can differ heavily (that's not including the labor involved in getting a local PCF instance running on your machine in the first place). Docker and, by proxy, Kubernetes make it easy to do this since you're deploying images, not code. While PCF uses LXC containers behind the scenes via Diego, they rely on buildpacks as the runtime mechanism. This slows deployment time down since you're effectively installing everything from scratch on every deploy, and it just feels heavier weight than working with Docker images. 3) The PCF community is mostly comprised of large enterprises hidden behind paid Pivotal contracts. Getting help on PCF issues from the Internet is quite hard, and it's pretty clear that Pivotal is driving a lot of the development behind the product despite CF being an open-source project funded by the CF foundation. (Apparently raw CF is quite different from PCF, though I haven't tried it myself and the lack of community excitement around it doesn't make me want to). 4) PCF licensing is expesnive. Combined with (1), you basically need to be a multi-$B company to afford it. See also: this (https://news.ycombinator.com/item?id=16663077 https://news.ycombinator.com/item?id=16663077). As a not-yet-so-still-armchair tech manager, my main gripe with it is that managed Kubernetes products might be closer to IaaS than PaaS, but they offer the same level of application abstraction and deployment that PCF does with a SIGNIFICANTLY smaller footprint while having a significantly larger community driving its success. This gap will become even tighter as KNative, Helm and others mature. I personally like products that are vetted by companies large and small, not just enterprises. PCF is the latter, Kubernetes and others are the former. The things I like most about Kubernetes that aren't present in PCF are: 1. Using container images to deploy by default instead of code + buildpacks so that I have greater confidence in my code working in Production (yes, PCF supports it, but PCF org admins can and probably will turn that off). 2. More flexibility around networking (PCF basically requires VMware/Cisco networking, whereas Kubernetes does everything through CNI plugins) 3. Being able to configure all of Kubernetes through manifests instead of solely the application 4. Kubernetes is purely open-source and has one of the strongest open-source communities behind it. Unlike CF, who really only seems to suit PCF and it's enterprise variants (IBM Bluemix comes to mind), the CNCF is a real foundation that supports all sorts of cloud-native applications from all sorts of paths. OpenShift, Red Hat's enterprise take on it, contributes heavily to the project while still managing to sell to the same market that PCF is after.

codingminds 8y ago

That's a very interesting point for me. Our company is in discussion with Pivotal to use PAS or PKS on prem. Do you have a specific example or something similar what was the problem ? I've only seen the slides which should sell the products to OPS guys and the idea looked promising.

mothsonasloth 8y ago
I can only comment on PAS but the main issues I've experienced are: I have to login to PCF to check environments or add services to bind to an app. The CF cli is not very easy to understand compared to aws eb cli or herokus toolbelt The login screen says "email" but you actually have to enter your user ID Devops become your enemies, especially if they lock down PCF environments. I don't want to beg for a redis tile. PCF eats a lot of ram from our virtual servers. The PCF support is layered with different urgency levels, most of the time they ignore you or are slow to respond Basically I think it's a tool that gets in the way of developers and gives headaches to the platform team. However my company signed a contract with them, so we're tied in for a year.... I might quit before then
- codingminds 8y ago
  thanks for the answer
- twic 8y ago
  I remember being in a meeting with a client, with a Pivotal salesman alongside me, when the question of PCF's overhead came up. We managed to work out that a PCF installation required about 10 separate VMs before you even got to the ones that would run applications.
  - latchkey 8y ago
    The point being that those VM can be distributed across multiple hosts for scalability and reliability. I'm sure that if you wanted to really package it all up into a single VM, you could... but that wouldn't make much sense when you're building an onpremis deployment cloud.
    - twic 8y ago
      No, the ten VMs are not there for scalability or reliability, they're there because there are ten teams building the essential components of Cloud Foundry, and the BOSH model is that each component gets its own VM. You could squeeze them onto one VM, but that would require that CF teams cooperate on manifests. Inconceivable!
      - latchkey 8y ago
        Maybe you could fathom that the BOSH model is designed the way it is, for scalability and reliability purposes. The CF team is a bunch of Pivots... they cooperate quite well and probably better than most other teams, especially since they rotate across projects fairly often. Disclosure, I worked on CF in the early days, on a number of teams.
  - EngineerBetter 8y ago
    PCF is for those working at scale, and doesn't suit trivial deployments well.
    - twic 8y ago
      Someone really should have told the salesman that, then - they were trying to sell the client on the idea that they could have a separate PCF foundation for each project.
- insomniacity 8y ago
  I use PCF at work. I’ve noticed most of these but they didn’t really bother me... We have low timeouts on most logins anyway, so having it on PCF isn’t a big deal. We have a CI/CD workflow, so I don’t really have to login to the CLI that much. Locked down environments are always a pain - but it would be no different if your org locked down AWS or Heroku, which they could.
- EngineerBetter 8y ago
  > Devops become your enemies You're doing devops wrong, which may explain why you're having a bad time with the platform. > I have to login to PCF to check environments or add services to bind to an app. You should automate this with CI servers and pipelines. The complaints about RAM usage and support service I agree with. It sounds like your organisation is disfunctional, and its treatment of PCF is the symptom not the cause.
nunez 8y ago
PCF works best with the VMware stack (vSphere + vROPs) hence the sell to Operations. It is an absolute beast to install. You can do a PoC with PCF Dev to see what using it as a developer will be like, but the actual implementation of it will absolutely require Pivotal consultants to get going. PCF itself is also very expensive. I'm not sure if they charge per node or per app; they've changed it over the last few years. I would get the skinny on that before committing to anything and do a TCO/ROI analysis. If you are going to go this route, I would also consider doing a PoC with OpenShift or Rancher. These are managed Kubernetes offerings, both on prem. Both have free open-source editions that mostly mirror what you'll get in the Enterprise offering at smaller scale. Rancher is also less "enterprisey" in that they aren't as aggressive about enterprise support or upsells, at least from what I've heard. I think you'll be locked into big contracts with either option, but Pivotal's lock-in is tighter, at least from what I can tell. This could change after IBM fully moves in, however.

mwcampbell 8y ago

> We are now starting to do some of our infrastructure on Kubernetes, much less headache. What does your build process look like? Can you use CF buildpacks in your Kubernetes cluster? A built-in system for building and deploying your application when you push to a Git repo seems like a real advantage of CF to me.

fermuch 8y ago
I've never used CF, but you can git push build to k8s with gitkube Gitkube thread on HN: https://news.ycombinator.com/item?id=16574893 https://news.ycombinator.com/item?id=16574893
- mothsonasloth 8y ago
  We are not git ops yet but that's what I think we should try to achieve
pat2man 8y ago
OpenShift is great for this. For something like a ruby application I can just run `oc new-app .` and I get by current application built, deployed and available externally. All with full kubernetes support. (OpenShift uses https://github.com/openshift/source-to-image https://github.com/openshift/source-to-image which works like CF buildpacks)

jambay 8y ago

i work on pcf at pivotal. i'd love to hear how we can improve. spring boot should work great with "cf push" and when it doesn't we would love to know why and make it better. while pks and k8s are able to run stateless workloads, if pas isn't offering a lot of additional productivity with routing, logging, and multi-tenancy features then we've missed the mark. pivotal-cf-feedback at pivotal dot io will reach the product managers that work on different parts. i saw your note further below about having to login, the "email" prompt instead of username, you had to ask to get the redis tile, and that you find the cf cli confusing compared to other platform clis you are familiar with. we definitely want to know let us know if there are other major issues.

Something1234 8y ago
You sound like a suit. Too many acronyms.
- latchkey 8y ago
  Having worked with James directly many years ago as a developer at/on Pivotal/Cloud Foundry, he is a bit of a suit... but he's also a really great human, very smart and he genuinely cares about the product. Cut him some slack please.
- dang 8y ago
  Personal attacks will get you banned here. Please review https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and post civilly and substantively, or not at all.
erikb 8y ago
I work in a big corp myself. If you worked on cloud foundry you probably worked with some of my colleagues. ;-) Please don't mention the name though, so I can speak openly. I see this in my corp every day. Very smart people, super interested in actual feedback, and super interested in really providing great software. But in the end they still fail. I'm not 100% sure about the reasons, but I have a guess. Usually people who move up the ranks and get into positions where they can make decisions, tend to build relationships with other such people. So more and more they change from "why the hack is dns not working on server-13 anymore?" to "this is the big problem the market has, let's attack that". And slowly that thought croaches in that you should only think about the big picture problems and not worry about the nitty-gritty details of every day getting-things-to-run. And that's probably the problem. While it's clear that you can't solve all the details and work on big picture problems at the same time, you should not devaluate the details. Great solutions solve big picture problems BY SOLVING THE DETAILS. That's why K8S for instance is a mediocre solution. Sure they have great big picture ideas, but usually people spend their time with having DNS not working, having servers hanging, having etcd nodes not talking to each other without explaining why, having deployments say "SUCCESS" while actually not running. K8S provides a standardized API for every detail and every use-case, cool. But in the end you need to be an expert in everything to make it run around 50% of the time (outside of AWS and GCP). Nobody really wants to develop on k8s when the underlying platform is only available 50% of the time. And that's the current experience. So please, if you are one of these people who is really interested in creating great software, consider finding solutions to the detailed low-level problems an important part of the goals and tasks you define. Usually the core low-level problems are not a big amount. For instance with k8s it usually is networking, dynamic storage or security related. Have one person in your team really become an expert in one of these areas (by checking out problems and solutions that exist, not just by making up ppt slides with plans that have nothing to do with the real world) and allow them to influence your task planning.

scothis 8y ago

> We are now starting to do some of our infrastructure on Kubernetes, much less headache. Pivotal Function Service (PFS) builds directly on Kubernetes via Knative (https://github.com/knative/ https://github.com/knative/) and Project riff (https://projectriff.io https://projectriff.io).

colemickens 8y ago
What does PFS do that Knative+Riff doesn't get me? Is the title even accurate? It sounds/looks more like "Pivotal creates branded offering of Knative+Riff and uses one of their main selling points as their headline feature", but maybe I'm just biased from past impressions of Pivotal marketing. Especially since it seems to be in contact-only early access.
- richards 8y ago
  Good question. I work at Pivotal. PFS is a commercial, supported packaging of Riff (which adds dev and ops aspects to Knative). As opposed to running and supporting the OSS components on your own.

erikb 8y ago

> We are now starting to do some of our infrastructure on Kubernetes, much less headache. I bet you do it on AWS or GCP, right? Try on-prem, then check your headackes again. ;-)

jacques_chester 8y ago

> I am having a terrible time working with PAS. Then we are not doing our job. I am happy to talk to you and/or to put you in touch with any team you want to give feedback to.

nunez 8y ago

I've finally gotten the chance to spend time with PCF in an enterprise setting. I've also spent time with Kubernetes (no KNative though, at least not yet) and Docker Datacenter. Unfortunately, I have to agree though I wouldn't call it terrible. My opinion of the product is that it is designed for organizations that are heavily invested in the VMware ecosystem that want to maintain that developer-operator separation through the guise of a PaaS with as little code or ramp-up (or investment from their own people) as possible. As a dev, my gripes with PCF are as follows: 1) PCF is massive. You need at least 20 instances to run it...and that's without any workloads on top of it. (If you are on AWS, the from-scratch installation instructions require that you bump your EC2 instance count minimums to 50 across several different instance sizes. Yikes.) It is very clearly designed for companies that already have a significant VMware footprint in house. Running it on AWS (I don't think Pivotal supports running it on Azure or GCP, but I could be wrong) is a very expensive exercise in futility. You cannot _just_ install PCF on your local machine unless you use PCF Dev or cf-local, both of which remove a lot to provide some semblance of parity to a larger install. (You also can't _just_ install PCF Dev either like you can with minikube or Swarm; you need to have a Pivotal account (free) and then download it from the Pivotal network. This can be automated, but it requires a lot of steps.) This leads me to... 2) There's no real way to test that code on your laptop will work in a production PCF deployment since the infrastructure they run on top of can differ heavily (that's not including the labor involved in getting a local PCF instance running on your machine in the first place). Docker and, by proxy, Kubernetes make it easy to do this since you're deploying images, not code. While PCF uses LXC containers behind the scenes via Diego, they rely on buildpacks as the runtime mechanism. This slows deployment time down since you're effectively installing everything from scratch on every deploy, and it just feels heavier weight than working with Docker images. 3) The PCF community is mostly comprised of large enterprises hidden behind paid Pivotal contracts. Getting help on PCF issues from the Internet is quite hard, and it's pretty clear that Pivotal is driving a lot of the development behind the product despite CF being an open-source project funded by the CF foundation. (Apparently raw CF is quite different from PCF, though I haven't tried it myself and the lack of community excitement around it doesn't make me want to). 4) PCF licensing is expesnive. Combined with (1), you basically need to be a multi-$B company to afford it. See also: this (https://news.ycombinator.com/item?id=16663077 https://news.ycombinator.com/item?id=16663077). As a not-yet-so-still-armchair tech manager, my main gripe with it is that managed Kubernetes products might be closer to IaaS than PaaS, but they offer the same level of application abstraction and deployment that PCF does with a SIGNIFICANTLY smaller footprint while having a significantly larger community driving its success. This gap will become even tighter as KNative, Helm and others mature. I personally like products that are vetted by companies large and small, not just enterprises. PCF is the latter, Kubernetes and others are the former. The things I like most about Kubernetes that aren't present in PCF are: 1. Using container images to deploy by default instead of code + buildpacks so that I have greater confidence in my code working in Production (yes, PCF supports it, but PCF org admins can and probably will turn that off). 2. More flexibility around networking (PCF basically requires VMware/Cisco networking, whereas Kubernetes does everything through CNI plugins) 3. Being able to configure all of Kubernetes through manifests instead of solely the application 4. Kubernetes is purely open-source and has one of the strongest open-source communities behind it. Unlike CF, who really only seems to suit PCF and it's enterprise variants (IBM Bluemix comes to mind), the CNCF is a real foundation that supports all sorts of cloud-native applications from all sorts of paths. OpenShift, Red Hat's enterprise take on it, contributes heavily to the project while still managing to sell to the same market that PCF is after.