7 ms·

I had a very interesting discussion about this subject a couple of weeks ago with a CTO of a big company. I've commented before that Kube absolutely takes over

I had a very interesting discussion about this subject a couple of weeks ago with a CTO of a big company. I've commented before that Kube absolutely takes over the bigger more complex cloud installations out there, you can see how many companies are betting their infrastructure on it. The only thing that I don't see is standardization of the cloud, just like what Amazon did. You see too many companies doing too many of the same things and reinventing the wheel. Personally, I would love to see smaller installation as a standard of how to take things into the cloud as a cluster. Imagine what Heroku did to deployments. You can't beat this ease of use. Deis and Convox are both trying but not really "hitting it". As for Walmart, absolutely stoked seeing it from them. This move and what the white-house did with the digital shows a lot of promise and hope. I wonder how much of this is on top of "older" management and how much is just complete restructuring.

jacques_chester 10y ago
> The only thing that I don't see is standardization of the cloud, just like what Amazon did ... Deis and Convox are trying but not really "hitting it" My colleagues at Red Hat might disagree, they're working on OpenShift. My more immediate colleagues at Pivotal, IBM, SAP, Microsoft, Google, Cisco, Dell-EMC, VMWare et al might disagree too. We're working on Cloud Foundry and BOSH. Disclosure: I work for Pivotal.
- avitzurel 10y ago
  OpenShift origin looks very promising for sure. That's the thing though, you still see too many people not using it. Here are things that should be standard (IMHO) and you shouldn't reinvent 1. Underlying infrastructure auto scaling (Google, AWS) 2. Service Discovery 3. DNS (internal and external for multiple sources) 4. Networking The real issue that I feel no-one really answered is who's this for. If a SRE is the target audience than there's a lot more we're missing as a community.
  - jacques_chester 10y ago
    I think Red Hat's problem with OpenShift is getting out from under the brand-masking power of tech they chose. Too many engineers want to roll their own Docker+Kubernetes platform and underestimate the difficulty of doing so. The thing is, everyone has a different don't-reinvent list. And they want to be not reinvented in different ways. Then they discover all the things that they didn't realise they'll need to reinvent. > The real issue that I feel no-one really answered is who's this for. I see PaaSes as serving three constituencies. Operators, who wish to christ that Developers would stop making their lives hell by breaking stuff. Developers, who wish to christ that Operators would stop making their lives hell by blocking stuff. Business, who wonder why everything takes so long, costs so much and breaks so often.
    - deleted 10y ago
      [deleted]
    - jlgaddis 10y ago
      Maybe (hopefully!) things have improved since I last looked at it, but OpenStack is (was?) a complicated, unorganized, over-engineered conglomeration of independent parts. I sincerely hope that's not the case anymore but that lasting first impression has stopped me from looking into it since then.
      - cpitman 10y ago
        OpenStack (IaaS) != OpenShift (PaaS) Getting up and running with OpenShift is pretty easy. You can either use the all in one Vagrant VM[1] or you can download the CLI[2] and run `oc cluster up`[3] to install the docker container version. Either one doesn't give you an "HA" install (probably want a cluster for that). I regularly deliver OpenShift installs, and in most enterprise environments an HA install takes 3-4 weeks, most of which is communicating to all the silos (networking, storage, security, virtualization, etc) what the requirements are. Disclaimer: I work for Red Hat. 1: https://www.openshift.org/vm/ https://www.openshift.org/vm/ 2: https://github.com/openshift/origin/releases https://github.com/openshift/origin/releases 3: https://github.com/openshift/origin/blob/master/docs/cluster_up_down.md https://github.com/openshift/origin/blob/master/docs/cluster...
hueving 10y ago
Kubernetes != virtualization Some virtualization use cases can be solved with kubernetes, but many things an enterprise runs do not work at all in the kube paradigm. The majority of applications are pets (i.e. stateful) and many applications run on Windows or have some other kernel requirements that mismatch the baremetal kubelet. If you think Kubernetes 'absolutely takes over the more complex cloud installations', you're living in an echo chamber of 12 factor apps that doesn't line up with the majority of what I've seen in big enterprise cloud workloads. Case in point: Walmart has one of the largest openstack deployments in the world.
- maloney 10y ago
  Have you checked out pet sets[1]? The last couple of kube releases have introduced some really nice work to help with these cases [1]http://kubernetes.io/docs/user-guide/petset/ http://kubernetes.io/docs/user-guide/petset/
  - tmzt 10y ago
    Does this fit in with hyperkube in some way? Can you see 'pods' that are really complete VM's which may or may not include OCI containers on them. For instance, would a complete Windows Server instance fit the Pet Set concept in the scheduler?
    - 101km 10y ago
      Nothing to do with hyperkube. PetSets are simply ordinally ordered container collections instead of randomly ordered. The usual concept in Kubernetes is that a container has a name like 'thing-randomsuffix4242'. It is pushing you towards making your pods stateless and not precious so the failure handing logic is simple. If it is blown away for whatever reason can easily be replaced by the scheduler and 'thing-randomsuffix242b' is never far away. With a predictable ordering you can actually make assumptions you otherwise couldn't. For example, perhaps you want the convention that 'thing-predictable1' is the master and 'thing-rpredictable{2,3}' are the slaves.
    - sandGorgon 10y ago
olalonde 10y ago
> Deis and Convox are both trying but not really "hitting it". I'm heavily biased towards Deis as I've been using it for a while (and even wrote an UI for it[0]), but what do you feel it is missing? Feature wise, it's almost at parity with Heroku and stability wise, it's not perfect but it doesn't require a PhD in DevOps to maintain either. [0] https://github.com/olalonde/deisdash#deis-dash https://github.com/olalonde/deisdash#deis-dash
- avitzurel 10y ago
  Your dash looks awesome! I replied to the Deis engineer above with my thoughts.
bacongobbler 10y ago
> Deis and Convox are both trying but not really "hitting it". Can you elaborate on this? Any feedback on how we "missed the mark" is always helpful. (I am an engineer working @ Deis)
- avitzurel 10y ago
  First, important to note: You are awesome and doing amazing work. (Link to what I am doing later in this message). The problem with all projects, not just deis is that the end user needs to know too much. I launch about 30 projects a year on top of Heroku and it never ceases to amaze me how simple it is. Yes, the applications are simple. Classic app->db. That being said, the setup is stupid simple. When I started working on the-startup-stack[1], I was amazed on how much you need to do when you start from scratch. Things I forgot I even did since I have a system running in production with incremental changes for 5 years. You need to configure networking, decide on DNS, decide on so many thing. It's a lot of work. Here's what I want (and what I am trying to do with the-startup-stack[1]). 0. AWS KeyPair 1. create-stack 2. launch-app Lot of what's missing is best practices and production-ready environments. Going back to that same lunch discussion I had with the CTO of a big company doing a replatforming of their infrastructure: Generalization is the hardest thing here. Reasoning on who's the customer and what does their stack look like. If you give Deis/OpenShift/Kubernetes to the typical YC founder (I am not trying to insult anyone here) that's trying to get their app in their cloud, it's just too much. It's too much of things they don't care about right now. By not caring about it right now they are likely vendor-locking themselves for a very long time. [1] http://docs.the-startup-stack.com http://docs.the-startup-stack.com
  - avitzurel 10y ago
    and b.t.w, this is all AFTER you have a docker container. Lets not forget about the huge leap of master -> ci -> docker -> deploy headache.
    - bacongobbler 10y ago
      Agreed. That's the problem with running your own private Heroku; you've gotta know the entire stack to use it. We had this dream that users can use Deis as a self-serve dev shop but we've been seeing a lot of people interested in running their entire PaaS stack, so the docs definitely reflect the administrator more so than the platform user (like Heroku).
doublerebel 10y ago
Joyent and Distelli make this about as painless as possible with any current workflow. Docker containers can be launched directly in any number or size to Joyent, and Distelli will app-ify and/or dockerize your app as part of the CI/CD workflow and watch the processes. Any custom script is just bash, and I can deploy to any OS. I have a cluster of 5 or 6 instances running 20-30 services in this setup and it's been a dream. I tried to do something similar before, but Cloud66 dropped Joyent. It makes perfect sense because apps can be anywhere between traditionally deployed to fully containerized and still are managed with the same interface and config. Since it's all open-source my deploy process is portable. But I've evaluated other vendors and processes and all seem more manual and less transparent.
gogopuppygogo 10y ago
Standardization of the cloud is what http://www.ucxchange.com http://www.ucxchange.com is working on out of Chicago. It's an interesting model that I was shown recently. The basic premise is that with standardization of hosting environment platforms an exchange is able to offer their customers a multitude of vendors who compete on price and will have the same features within their environments. The most interesting opportunity with this is for resellers of cloud computing who can sell compute resources to customers at full price but only pay the exchange based on what they use. It won't last long but if it works resellers will make a fortune migrating customers from big names like Amazon to other big names like IBM through UCX.
- _joel 10y ago
  I only see US states listed. Is it assumed on this site that cloud infrastructure only exists in the US, or is this due to regulatory constraints?
sorich87 10y ago
Not sure when you tried Convox – maybe they have improved since then – but my experience with it last week has been amazing (migrated our application from Heroku). It was almost as easy and quick to setup as Heroku. All the docker and AWS config was automatically done by Convox. The only issue I had was with Docker: an old Docker Toolbox installation left some environment variables preventing Docker for Mac from starting. Granted, I have some previous experience managing deployment on AWS via AWS CLI and dashboard so maybe that's what made me quickly understand Convox concepts. My only regret is I didn't setup with Convox from the start. :)

jacques_chester 10y ago

> The only thing that I don't see is standardization of the cloud, just like what Amazon did ... Deis and Convox are trying but not really "hitting it" My colleagues at Red Hat might disagree, they're working on OpenShift. My more immediate colleagues at Pivotal, IBM, SAP, Microsoft, Google, Cisco, Dell-EMC, VMWare et al might disagree too. We're working on Cloud Foundry and BOSH. Disclosure: I work for Pivotal.

avitzurel 10y ago
OpenShift origin looks very promising for sure. That's the thing though, you still see too many people not using it. Here are things that should be standard (IMHO) and you shouldn't reinvent 1. Underlying infrastructure auto scaling (Google, AWS) 2. Service Discovery 3. DNS (internal and external for multiple sources) 4. Networking The real issue that I feel no-one really answered is who's this for. If a SRE is the target audience than there's a lot more we're missing as a community.
- jacques_chester 10y ago
  I think Red Hat's problem with OpenShift is getting out from under the brand-masking power of tech they chose. Too many engineers want to roll their own Docker+Kubernetes platform and underestimate the difficulty of doing so. The thing is, everyone has a different don't-reinvent list. And they want to be not reinvented in different ways. Then they discover all the things that they didn't realise they'll need to reinvent. > The real issue that I feel no-one really answered is who's this for. I see PaaSes as serving three constituencies. Operators, who wish to christ that Developers would stop making their lives hell by breaking stuff. Developers, who wish to christ that Operators would stop making their lives hell by blocking stuff. Business, who wonder why everything takes so long, costs so much and breaks so often.
  - deleted 10y ago
    [deleted]
  - jlgaddis 10y ago
    Maybe (hopefully!) things have improved since I last looked at it, but OpenStack is (was?) a complicated, unorganized, over-engineered conglomeration of independent parts. I sincerely hope that's not the case anymore but that lasting first impression has stopped me from looking into it since then.
    - cpitman 10y ago
      OpenStack (IaaS) != OpenShift (PaaS) Getting up and running with OpenShift is pretty easy. You can either use the all in one Vagrant VM[1] or you can download the CLI[2] and run `oc cluster up`[3] to install the docker container version. Either one doesn't give you an "HA" install (probably want a cluster for that). I regularly deliver OpenShift installs, and in most enterprise environments an HA install takes 3-4 weeks, most of which is communicating to all the silos (networking, storage, security, virtualization, etc) what the requirements are. Disclaimer: I work for Red Hat. 1: https://www.openshift.org/vm/ https://www.openshift.org/vm/ 2: https://github.com/openshift/origin/releases https://github.com/openshift/origin/releases 3: https://github.com/openshift/origin/blob/master/docs/cluster_up_down.md https://github.com/openshift/origin/blob/master/docs/cluster...

hueving 10y ago

Kubernetes != virtualization Some virtualization use cases can be solved with kubernetes, but many things an enterprise runs do not work at all in the kube paradigm. The majority of applications are pets (i.e. stateful) and many applications run on Windows or have some other kernel requirements that mismatch the baremetal kubelet. If you think Kubernetes 'absolutely takes over the more complex cloud installations', you're living in an echo chamber of 12 factor apps that doesn't line up with the majority of what I've seen in big enterprise cloud workloads. Case in point: Walmart has one of the largest openstack deployments in the world.

maloney 10y ago
Have you checked out pet sets[1]? The last couple of kube releases have introduced some really nice work to help with these cases [1]http://kubernetes.io/docs/user-guide/petset/ http://kubernetes.io/docs/user-guide/petset/
- tmzt 10y ago
  Does this fit in with hyperkube in some way? Can you see 'pods' that are really complete VM's which may or may not include OCI containers on them. For instance, would a complete Windows Server instance fit the Pet Set concept in the scheduler?
  - 101km 10y ago
    Nothing to do with hyperkube. PetSets are simply ordinally ordered container collections instead of randomly ordered. The usual concept in Kubernetes is that a container has a name like 'thing-randomsuffix4242'. It is pushing you towards making your pods stateless and not precious so the failure handing logic is simple. If it is blown away for whatever reason can easily be replaced by the scheduler and 'thing-randomsuffix242b' is never far away. With a predictable ordering you can actually make assumptions you otherwise couldn't. For example, perhaps you want the convention that 'thing-predictable1' is the master and 'thing-rpredictable{2,3}' are the slaves.
  - sandGorgon 10y ago

olalonde 10y ago

> Deis and Convox are both trying but not really "hitting it". I'm heavily biased towards Deis as I've been using it for a while (and even wrote an UI for it[0]), but what do you feel it is missing? Feature wise, it's almost at parity with Heroku and stability wise, it's not perfect but it doesn't require a PhD in DevOps to maintain either. [0] https://github.com/olalonde/deisdash#deis-dash https://github.com/olalonde/deisdash#deis-dash

avitzurel 10y ago
Your dash looks awesome! I replied to the Deis engineer above with my thoughts.

bacongobbler 10y ago

> Deis and Convox are both trying but not really "hitting it". Can you elaborate on this? Any feedback on how we "missed the mark" is always helpful. (I am an engineer working @ Deis)

avitzurel 10y ago
First, important to note: You are awesome and doing amazing work. (Link to what I am doing later in this message). The problem with all projects, not just deis is that the end user needs to know too much. I launch about 30 projects a year on top of Heroku and it never ceases to amaze me how simple it is. Yes, the applications are simple. Classic app->db. That being said, the setup is stupid simple. When I started working on the-startup-stack[1], I was amazed on how much you need to do when you start from scratch. Things I forgot I even did since I have a system running in production with incremental changes for 5 years. You need to configure networking, decide on DNS, decide on so many thing. It's a lot of work. Here's what I want (and what I am trying to do with the-startup-stack[1]). 0. AWS KeyPair 1. create-stack 2. launch-app Lot of what's missing is best practices and production-ready environments. Going back to that same lunch discussion I had with the CTO of a big company doing a replatforming of their infrastructure: Generalization is the hardest thing here. Reasoning on who's the customer and what does their stack look like. If you give Deis/OpenShift/Kubernetes to the typical YC founder (I am not trying to insult anyone here) that's trying to get their app in their cloud, it's just too much. It's too much of things they don't care about right now. By not caring about it right now they are likely vendor-locking themselves for a very long time. [1] http://docs.the-startup-stack.com http://docs.the-startup-stack.com
- avitzurel 10y ago
  and b.t.w, this is all AFTER you have a docker container. Lets not forget about the huge leap of master -> ci -> docker -> deploy headache.
  - bacongobbler 10y ago
    Agreed. That's the problem with running your own private Heroku; you've gotta know the entire stack to use it. We had this dream that users can use Deis as a self-serve dev shop but we've been seeing a lot of people interested in running their entire PaaS stack, so the docs definitely reflect the administrator more so than the platform user (like Heroku).

doublerebel 10y ago

Joyent and Distelli make this about as painless as possible with any current workflow. Docker containers can be launched directly in any number or size to Joyent, and Distelli will app-ify and/or dockerize your app as part of the CI/CD workflow and watch the processes. Any custom script is just bash, and I can deploy to any OS. I have a cluster of 5 or 6 instances running 20-30 services in this setup and it's been a dream. I tried to do something similar before, but Cloud66 dropped Joyent. It makes perfect sense because apps can be anywhere between traditionally deployed to fully containerized and still are managed with the same interface and config. Since it's all open-source my deploy process is portable. But I've evaluated other vendors and processes and all seem more manual and less transparent.

gogopuppygogo 10y ago

Standardization of the cloud is what http://www.ucxchange.com http://www.ucxchange.com is working on out of Chicago. It's an interesting model that I was shown recently. The basic premise is that with standardization of hosting environment platforms an exchange is able to offer their customers a multitude of vendors who compete on price and will have the same features within their environments. The most interesting opportunity with this is for resellers of cloud computing who can sell compute resources to customers at full price but only pay the exchange based on what they use. It won't last long but if it works resellers will make a fortune migrating customers from big names like Amazon to other big names like IBM through UCX.

_joel 10y ago
I only see US states listed. Is it assumed on this site that cloud infrastructure only exists in the US, or is this due to regulatory constraints?

sorich87 10y ago

Not sure when you tried Convox – maybe they have improved since then – but my experience with it last week has been amazing (migrated our application from Heroku). It was almost as easy and quick to setup as Heroku. All the docker and AWS config was automatically done by Convox. The only issue I had was with Docker: an old Docker Toolbox installation left some environment variables preventing Docker for Mac from starting. Granted, I have some previous experience managing deployment on AWS via AWS CLI and dashboard so maybe that's what made me quickly understand Convox concepts. My only regret is I didn't setup with Convox from the start. :)