4 ms·

I never see this mentioned, but to me the most important feature of Docker based CaaS (or PaaS) is that they are no longer purely for hosting custom software. T

by cpitman 11y ago

I never see this mentioned, but to me the most important feature of Docker based CaaS (or PaaS) is that they are no longer purely for hosting custom software. The focus of Heroku style PaaS systems has been the flow of code to running software, but what about applications I have no code for? I want to run GitLab, Mattermost, an API gateway, etc. With Docker based CaaS/PaaS, this is easy since apps can be easily packaged as containers that are treated the same as your own custom apps. This is only my opinion, but I think we are getting to the point where all server software can be run on something like Kubernetes and traditional virtualization becomes even more of a commodity. Disclaimer, I work for Red Hat! We have both a CaaS (RHEL Atomic Enterprise Platform ie Kubernetes) and a PaaS built on top of it (OpenShift v3). Containers taking over the entire data center is my own opinion, but it would be great!

eranation 11y ago
Sorry for the silly question but is CaaS "Containers as a Service"? (I assume it is not Content as a Service: https://en.wikipedia.org/wiki/Content_as_a_Service https://en.wikipedia.org/wiki/Content_as_a_Service) If so, then the wikipedia disambiguation page (https://en.wikipedia.org/wiki/CAAS https://en.wikipedia.org/wiki/CAAS) redirects to a yet unwritten one: https://en.wikipedia.org/wiki/Containers_as_a_Service https://en.wikipedia.org/wiki/Containers_as_a_Service ;)
- phildougherty 11y ago
  Yes, Containers as a Service :)
  - eranation 11y ago
    Well... Here is your chance to be able to add "The guy who wrote the encyclopedia entry on CaaS" to your resume ;)
phildougherty 11y ago
I am in complete agreement. I don't think it's a question of if, and more a question of WHEN everything in the data center is containerized. The benefits far outweigh the negatives (like educating your whole staff on a new workflow or set of tools), and it seems that ease of use when it comes to the ecosystem/tools/orchestration is really the only hurdle keeping it from getting there. We're working on making those "non-custom" software packages easy to stand up and use via http://containership.io/#marketplace http://containership.io/#marketplace
sytse 11y ago
Awesome that you want to run GitLab in a CaaS. We want to make sure you can deploy all your software from GitLab to Kubernetes without using any glue. Work on this is being done by the Redspread team in https://gitlab.com/gitlab-org/gitlab-ce/issues/1300648 https://gitlab.com/gitlab-org/gitlab-ce/issues/1300648
- cpitman 11y ago
  Yes, I've been following that! GitLab Deploy is also looking like it could be exciting. I was disappointed though that the merge request that added Kubernetes support to the GitLab CI MultiRunner was rejected. Right now Jenkins is usually the default choice for our customers that use OpenShift, but being able to dynamically provision testing containers would be an amazing feature.
  - ayufan 11y ago
    You shouldn't be. Executing builds on Kubernetes is not the same as deploying an application :) We will solve that need with plugins :) GitLab CI Lead
    - cpitman 11y ago
      That would be great, I'm not married to the implementation as much as the convenience of using the cluster manager we already have (kubernetes) to distribute builds. I currently am using a VM with a docker runner, and there are two (somewhat obvious) issues: * It requires root access to the machine to be able to spin up containers * Every VM with a docker runner needs enough storage to handle every docker image used for builds, instead of centralizing the images. I really need to put in a cron job or something to clean these up. Kubernetes should fix both of these, without introducing another technology in our environment like Docker Machine.
      - ayufan 11y ago
        I understand your reasoning. At this point it doesn't make sense to introduce support for another clustering manager. The GitLab Runner does support docker-machine (you can pretty easily configure it to create machines on-demand on any cloud provider). These machines will be pretty much self-managed. Soon the Runner will be also compatible with Docker Swarm, the implementation is already on separate branch. From my perspective Kubernetes is great as platform for deploying applications, but I'm not so sure if it comes for distributing builds. The problem with cleaning old images and caches is also addressed. You may be interested in checking out this application: https://gitlab.com/gitlab-org/gitlab-runner-docker-cleanup https://gitlab.com/gitlab-org/gitlab-runner-docker-cleanup What is important that GitLab Runner is being constantly improved and at some point with more convincing (for example coming from You) we can get back to native Kubernetes executor :) GitLab CI Lead
cat-dev-null 11y ago
I think you're looking for Chef or Puppet recipes, real configuration management tools which can flexibly deploy a complete stack, not some Bitnami toy which doesn't solve anything. The limitations with most all container platforms is what they don't do major things which need to be addressed somehow in production systems: - data backups / restores - security practices / policies (good luck running SELinux) - migrations to new versions - security updates - firewall rules - monitoring & performance agents - hard resource isolation (cpu, ram, network, IOPS) - [something else obviously important here] Containers are great for limited testing and local development, but for production they are like anti-type 1 hypervisors... more trouble that they're worth (docker instances still get stuck all the time as zombie processes, requiring a hard host reboot). Worse, most of these containers are built by unknown persons, are unsigned and hosted publicly, so it's effectively running untrustworthy code however they decided to package and deliver it. Finally, to add insult to injury, there's no reliable way to rebuild an image from scratch without some sources, whereas configuration management tools are intended to describe how to deploy apps and put systems into a desired configuration.
- cpitman 11y ago
  I actually am very experienced with Puppet, and I agree configuration management tools still have a (now narrower) space. But most of your concerns are actually handled by what we are doing at Red Hat. OpenShift v3: * applies SELinux policies to each running container * can handle multiple styles of container upgrades (rolling, green/blue, canary, etc) * includes an automated build process for automatically patching all of your containers when there is an update to their base image (including supported releases of RHEL containers that are kept up to date with errata) * automatically reconfigures iptables as containers go up/dowm/migrate * collects metrics on all running containers * health checks for running containers to ensure they are not zombies Since it is built on top of Kubernetes, the only configuration that you need to apply/maintain is the pod description, a short json document that describes what to run, what ports to expose, etc.

by cpitman 11y ago

eranation 11y ago

Sorry for the silly question but is CaaS "Containers as a Service"? (I assume it is not Content as a Service: https://en.wikipedia.org/wiki/Content_as_a_Service https://en.wikipedia.org/wiki/Content_as_a_Service) If so, then the wikipedia disambiguation page (https://en.wikipedia.org/wiki/CAAS https://en.wikipedia.org/wiki/CAAS) redirects to a yet unwritten one: https://en.wikipedia.org/wiki/Containers_as_a_Service https://en.wikipedia.org/wiki/Containers_as_a_Service ;)

phildougherty 11y ago
Yes, Containers as a Service :)
- eranation 11y ago
  Well... Here is your chance to be able to add "The guy who wrote the encyclopedia entry on CaaS" to your resume ;)

phildougherty 11y ago

I am in complete agreement. I don't think it's a question of if, and more a question of WHEN everything in the data center is containerized. The benefits far outweigh the negatives (like educating your whole staff on a new workflow or set of tools), and it seems that ease of use when it comes to the ecosystem/tools/orchestration is really the only hurdle keeping it from getting there. We're working on making those "non-custom" software packages easy to stand up and use via http://containership.io/#marketplace http://containership.io/#marketplace

sytse 11y ago

Awesome that you want to run GitLab in a CaaS. We want to make sure you can deploy all your software from GitLab to Kubernetes without using any glue. Work on this is being done by the Redspread team in https://gitlab.com/gitlab-org/gitlab-ce/issues/1300648 https://gitlab.com/gitlab-org/gitlab-ce/issues/1300648

cpitman 11y ago
Yes, I've been following that! GitLab Deploy is also looking like it could be exciting. I was disappointed though that the merge request that added Kubernetes support to the GitLab CI MultiRunner was rejected. Right now Jenkins is usually the default choice for our customers that use OpenShift, but being able to dynamically provision testing containers would be an amazing feature.
- ayufan 11y ago
  You shouldn't be. Executing builds on Kubernetes is not the same as deploying an application :) We will solve that need with plugins :) GitLab CI Lead
  - cpitman 11y ago
    That would be great, I'm not married to the implementation as much as the convenience of using the cluster manager we already have (kubernetes) to distribute builds. I currently am using a VM with a docker runner, and there are two (somewhat obvious) issues: * It requires root access to the machine to be able to spin up containers * Every VM with a docker runner needs enough storage to handle every docker image used for builds, instead of centralizing the images. I really need to put in a cron job or something to clean these up. Kubernetes should fix both of these, without introducing another technology in our environment like Docker Machine.
    - ayufan 11y ago
      I understand your reasoning. At this point it doesn't make sense to introduce support for another clustering manager. The GitLab Runner does support docker-machine (you can pretty easily configure it to create machines on-demand on any cloud provider). These machines will be pretty much self-managed. Soon the Runner will be also compatible with Docker Swarm, the implementation is already on separate branch. From my perspective Kubernetes is great as platform for deploying applications, but I'm not so sure if it comes for distributing builds. The problem with cleaning old images and caches is also addressed. You may be interested in checking out this application: https://gitlab.com/gitlab-org/gitlab-runner-docker-cleanup https://gitlab.com/gitlab-org/gitlab-runner-docker-cleanup What is important that GitLab Runner is being constantly improved and at some point with more convincing (for example coming from You) we can get back to native Kubernetes executor :) GitLab CI Lead

cat-dev-null 11y ago

I think you're looking for Chef or Puppet recipes, real configuration management tools which can flexibly deploy a complete stack, not some Bitnami toy which doesn't solve anything. The limitations with most all container platforms is what they don't do major things which need to be addressed somehow in production systems: - data backups / restores - security practices / policies (good luck running SELinux) - migrations to new versions - security updates - firewall rules - monitoring & performance agents - hard resource isolation (cpu, ram, network, IOPS) - [something else obviously important here] Containers are great for limited testing and local development, but for production they are like anti-type 1 hypervisors... more trouble that they're worth (docker instances still get stuck all the time as zombie processes, requiring a hard host reboot). Worse, most of these containers are built by unknown persons, are unsigned and hosted publicly, so it's effectively running untrustworthy code however they decided to package and deliver it. Finally, to add insult to injury, there's no reliable way to rebuild an image from scratch without some sources, whereas configuration management tools are intended to describe how to deploy apps and put systems into a desired configuration.

cpitman 11y ago
I actually am very experienced with Puppet, and I agree configuration management tools still have a (now narrower) space. But most of your concerns are actually handled by what we are doing at Red Hat. OpenShift v3: * applies SELinux policies to each running container * can handle multiple styles of container upgrades (rolling, green/blue, canary, etc) * includes an automated build process for automatically patching all of your containers when there is an update to their base image (including supported releases of RHEL containers that are kept up to date with errata) * automatically reconfigures iptables as containers go up/dowm/migrate * collects metrics on all running containers * health checks for running containers to ensure they are not zombies Since it is built on top of Kubernetes, the only configuration that you need to apply/maintain is the pod description, a short json document that describes what to run, what ports to expose, etc.