7 ms·

Google Skaffold – Easy and Repeatable Kubernetes Development

199 points by nikolay 9y ago

antoncohen 9y ago
Reminds me of Draft from Microsoft (https://github.com/Azure/draft https://github.com/Azure/draft). It also targets what they call the "inner loop" of the development workflow. It looks like Skaffold watches for code changes, and Draft doesn't. But Draft automatically creates the Docker and Kubernetes configs, while Skaffold requires you to create (or copy and edit) them.
- ppog 9y ago
  Draft can watch for code changes via a `watch` setting in draft.toml. (Disclosure: I work on the containers team at Microsoft.)
  - majewsky 9y ago
    Not directed at you specifically, but what is it with people these days feeling the need for a "disclosure" when just stating plain hard facts? Disclosures are only necessary when you state an opinion about something that you are involved with.
    - jacques_chester 9y ago
      It's easier to be over-honest, honestly. Disclosure: I suspect I am partly responsible for the "Disclosure:" thing become a fixture.
    - alexeldeib 9y ago
      I think it parallels the online culture around shilling/astroturfing -- no one wants to be lumped in that category, so people are extra careful to air their laundry before someone tries to spin an armchair-investigative Google search against them after finding their employment history.
cygned 9y ago
We are moving to Kubernetes at the moment. What is still miss is this “just run docker-compose up and everything works” setup. Maybe this or Draft from Microsoft is something to look into.
- modernpacifist 9y ago
  You can sort of achieve the same outcome just my having all the various services/container specs inside a single .yml file that gets fed to 'kubectl apply'. docker-compose up does create networks and containers unique to the app your developing, but you could have the .yml setup a custom namespace and pile everything into that and get close. No option to concat all the log output though AFAIK.
  - xnxn 9y ago
    I use stern for multi-pod log tailing: https://github.com/wercker/stern https://github.com/wercker/stern
    - ah- 9y ago
      I've switched to https://github.com/boz/kail https://github.com/boz/kail now, as stern seems to somehow miss pods coming up.
      - alpb 9y ago
        There's no shortage of tools that do the same thing in Kubernetes universe :) https://github.com/johanhaleby/kubetail https://github.com/johanhaleby/kubetail
- tedmiston 9y ago
  Kubernetes support in the Edge release of Docker for Mac provides this or via minikube on other platforms. We wrote a short doc on the former for Kubernetes itself if you happen to be on a Mac. https://enterprise.astronomer.io/guides/docker-for-mac/index.html https://enterprise.astronomer.io/guides/docker-for-mac/index... As far as app code, I highly recommend building on Helm if you're not already. I imagine that's what skaffold and draft are doing behind the scenes.
tyingq 9y ago
The shortfall, currently is some sort of long-term-supported version. Google's GKS and Amazon's (currently beta) EKS, seem to be the answer for the spaces I'm in. Love K8S, but not wanting to manage it like I currently manage operating systems. The pace of K8S versions, and subsequently things like ingress controllers for various tech stacks, is currently a whirlwind.
- tedmiston 9y ago
  AWS Fargate will be interesting too. But, like you said for EKS, it's still in private beta as well. I'm also keeping a pulse on riff [1] as a k8s FaaS. [1]: https://github.com/projectriff/riff https://github.com/projectriff/riff
  - tyingq 9y ago
    That's great, thanks for sharing. I'm currently the "why don't we wait a little" guy in a F500 where apps are trying to manage their own K8S installs. Still feel pretty strongly that "just wait a bit" is the right answer. I'm not loved at the moment for that opinion ;)
    - kevin_nisbet 9y ago
      At gravitational we actually build a platform for this: https://gravitational.com/telekube/ https://gravitational.com/telekube/ Among other things, let's you package an app and kubernetes together into a unified installer, upgrade it, etc. No internet access required or kubernetes experience needed for end users.
      - tedmiston 9y ago
        Is your product like Pivotal's PCF / PAS? https://pivotal.io/platform/pivotal-application-service https://pivotal.io/platform/pivotal-application-service
        jacques_chester 9y ago
alpb 9y ago
Mods probably should consider updating "Google Skaffold" to "Skaffold by Google" as that's not the project name.
- nikolay 9y ago
  Agreed. Only @dang can change it now as editing is now disabled.
rad_gruchalski 9y ago
Cool, looks interesting. I just spent a couple of weeks getting a k8s setup up and running on AWS, from zero. As much as I appreciate these sort of platforms, I did enjoy learning the inner details of how k8s components communicate together and how they work.
philip1209 9y ago
This looks promising. My main concern is whether it requires changing the way code is built and deployed too much. For instance, I don't want `npm run build` to be executed every time I change a Javascript file.
agency 9y ago
This looks interesting, but having to perform a Docker image build (even if most layers are cached) as part of your iteration loop seems potentially slow. We use minikube to run local stacks but have been using telepresence[1] to swap out services for ones running from a local build (outside of a container) with incremental compilation and hot reloading and all of that good stuff. [1] https://www.telepresence.io/ https://www.telepresence.io/
- ahnick 9y ago
  I wasn't aware of telepresence. Why have you opted to go this route over using docker compose and bind mounting host volumes? Does this work across all major platforms? (Windows/Mac/Linux)
  - agency 9y ago
    We initially went that route but changed for a couple of reasons: 1. The performance of bind mounted host volumes in Docker for Mac was pretty disappointing. We found our initial builds were substantially (~3x?) slower inside a (volume mounting) container than out. 2. Having to build all of our services when you started the stack was also slow, versus the minikube + telepresence setup where you by default run pre-build images (the same ones we deploy) and swap out services as needed. You can mitigate this with well written Dockerfiles that do a good job of caching layers but it can be tedious.
- jacques_chester 9y ago
  Telepresence is really neat, I saw a demo last week from another team member. Speeding up builds is something I'm interested in. Google's FTL is one good solution; I think we'll see some more from their container tech team before long. Personally I've pitched a notion based on automatically building layers for "synthetic" images by identifying commonalities between filesystems.
  - dlor 9y ago
    Hey Jacques, Do you have any more info on the idea about identifying commonalities between filesystems? I work on Skaffold and really want to speed up builds.
darren0 9y ago
I find it odd when I see projects in the GitHub org github.com/GoogleCloudPlatform like this one. Being that the org has the same name as Googles product (GCP) I would not expect experimental or nacent projects. I never know what to expect. Is this an official Google product that will be supported or some experiment by some devs in Google.
- fredsted 9y ago
  No matter what, it's probably best to expect that Google may deprecate any feature or project at any time.
davidbanham 9y ago
This looks really interesting. I have a project that does something similar for the remote deployment but uses only make and kubectl. https://github.com/davidbanham/kube_maker https://github.com/davidbanham/kube_maker I've been thinking about integrating a local minikube lately but haven't started yet. I may end using Skaffold as a backend or just deprecating kube_maker entirely if it becomes obvious that Skaffold is the way to go.
TraceyMorgannn 9y ago
Despite the fact that I discouraged my friends severally about hiring hackers to catch their cheating partners, I resorted to hiring a private investigator during my divorce. My husband hid his affair too well and I had no evidence to prove his infidelity to the court so I went on Craigslist to hire a professional whom I met and told my problem.. His services were topnotch which made working with him very easy, he got me everything I needed within 24 hours and it was amazing, I paid upfront despite my fears but I got results because I currently have my husband’s messages diverted to my phone, Call logs, Facebook messenger, WhatasApp, Instant chat, Viber, Skype Password, retrieved all his deleted messages, . David completed the hack without any physical access or installation on my husband’s phone so he never found out until I presented the information in court..Many thanks to David Consultants I got what I deserve Contact him for help. Gmail: Hotcyberclown @ gmail.com Text: +13478993017 Whatsapp Number: 12675265346 I know you will surely refer him.

199 points by nikolay 9y ago

Reminds me of Draft from Microsoft (https://github.com/Azure/draft https://github.com/Azure/draft). It also targets what they call the "inner loop" of the development workflow. It looks like Skaffold watches for code changes, and Draft doesn't. But Draft automatically creates the Docker and Kubernetes configs, while Skaffold requires you to create (or copy and edit) them.

ppog 9y ago
Draft can watch for code changes via a `watch` setting in draft.toml. (Disclosure: I work on the containers team at Microsoft.)
- majewsky 9y ago
  Not directed at you specifically, but what is it with people these days feeling the need for a "disclosure" when just stating plain hard facts? Disclosures are only necessary when you state an opinion about something that you are involved with.
  - jacques_chester 9y ago
    It's easier to be over-honest, honestly. Disclosure: I suspect I am partly responsible for the "Disclosure:" thing become a fixture.
  - alexeldeib 9y ago
    I think it parallels the online culture around shilling/astroturfing -- no one wants to be lumped in that category, so people are extra careful to air their laundry before someone tries to spin an armchair-investigative Google search against them after finding their employment history.

cygned 9y ago

We are moving to Kubernetes at the moment. What is still miss is this “just run docker-compose up and everything works” setup. Maybe this or Draft from Microsoft is something to look into.

modernpacifist 9y ago
You can sort of achieve the same outcome just my having all the various services/container specs inside a single .yml file that gets fed to 'kubectl apply'. docker-compose up does create networks and containers unique to the app your developing, but you could have the .yml setup a custom namespace and pile everything into that and get close. No option to concat all the log output though AFAIK.
- xnxn 9y ago
  I use stern for multi-pod log tailing: https://github.com/wercker/stern https://github.com/wercker/stern
  - ah- 9y ago
    I've switched to https://github.com/boz/kail https://github.com/boz/kail now, as stern seems to somehow miss pods coming up.
    - alpb 9y ago
      There's no shortage of tools that do the same thing in Kubernetes universe :) https://github.com/johanhaleby/kubetail https://github.com/johanhaleby/kubetail
tedmiston 9y ago
Kubernetes support in the Edge release of Docker for Mac provides this or via minikube on other platforms. We wrote a short doc on the former for Kubernetes itself if you happen to be on a Mac. https://enterprise.astronomer.io/guides/docker-for-mac/index.html https://enterprise.astronomer.io/guides/docker-for-mac/index... As far as app code, I highly recommend building on Helm if you're not already. I imagine that's what skaffold and draft are doing behind the scenes.

tyingq 9y ago

The shortfall, currently is some sort of long-term-supported version. Google's GKS and Amazon's (currently beta) EKS, seem to be the answer for the spaces I'm in. Love K8S, but not wanting to manage it like I currently manage operating systems. The pace of K8S versions, and subsequently things like ingress controllers for various tech stacks, is currently a whirlwind.

tedmiston 9y ago
AWS Fargate will be interesting too. But, like you said for EKS, it's still in private beta as well. I'm also keeping a pulse on riff [1] as a k8s FaaS. [1]: https://github.com/projectriff/riff https://github.com/projectriff/riff
- tyingq 9y ago
  That's great, thanks for sharing. I'm currently the "why don't we wait a little" guy in a F500 where apps are trying to manage their own K8S installs. Still feel pretty strongly that "just wait a bit" is the right answer. I'm not loved at the moment for that opinion ;)
  - kevin_nisbet 9y ago
    At gravitational we actually build a platform for this: https://gravitational.com/telekube/ https://gravitational.com/telekube/ Among other things, let's you package an app and kubernetes together into a unified installer, upgrade it, etc. No internet access required or kubernetes experience needed for end users.
    - tedmiston 9y ago
      Is your product like Pivotal's PCF / PAS? https://pivotal.io/platform/pivotal-application-service https://pivotal.io/platform/pivotal-application-service
      - jacques_chester 9y ago

alpb 9y ago

Mods probably should consider updating "Google Skaffold" to "Skaffold by Google" as that's not the project name.

nikolay 9y ago
Agreed. Only @dang can change it now as editing is now disabled.

rad_gruchalski 9y ago

Cool, looks interesting. I just spent a couple of weeks getting a k8s setup up and running on AWS, from zero. As much as I appreciate these sort of platforms, I did enjoy learning the inner details of how k8s components communicate together and how they work.

philip1209 9y ago

This looks promising. My main concern is whether it requires changing the way code is built and deployed too much. For instance, I don't want `npm run build` to be executed every time I change a Javascript file.

agency 9y ago

This looks interesting, but having to perform a Docker image build (even if most layers are cached) as part of your iteration loop seems potentially slow. We use minikube to run local stacks but have been using telepresence[1] to swap out services for ones running from a local build (outside of a container) with incremental compilation and hot reloading and all of that good stuff. [1] https://www.telepresence.io/ https://www.telepresence.io/

ahnick 9y ago
I wasn't aware of telepresence. Why have you opted to go this route over using docker compose and bind mounting host volumes? Does this work across all major platforms? (Windows/Mac/Linux)
- agency 9y ago
  We initially went that route but changed for a couple of reasons: 1. The performance of bind mounted host volumes in Docker for Mac was pretty disappointing. We found our initial builds were substantially (~3x?) slower inside a (volume mounting) container than out. 2. Having to build all of our services when you started the stack was also slow, versus the minikube + telepresence setup where you by default run pre-build images (the same ones we deploy) and swap out services as needed. You can mitigate this with well written Dockerfiles that do a good job of caching layers but it can be tedious.
jacques_chester 9y ago
Telepresence is really neat, I saw a demo last week from another team member. Speeding up builds is something I'm interested in. Google's FTL is one good solution; I think we'll see some more from their container tech team before long. Personally I've pitched a notion based on automatically building layers for "synthetic" images by identifying commonalities between filesystems.
- dlor 9y ago
  Hey Jacques, Do you have any more info on the idea about identifying commonalities between filesystems? I work on Skaffold and really want to speed up builds.

darren0 9y ago

I find it odd when I see projects in the GitHub org github.com/GoogleCloudPlatform like this one. Being that the org has the same name as Googles product (GCP) I would not expect experimental or nacent projects. I never know what to expect. Is this an official Google product that will be supported or some experiment by some devs in Google.

fredsted 9y ago
No matter what, it's probably best to expect that Google may deprecate any feature or project at any time.

davidbanham 9y ago

This looks really interesting. I have a project that does something similar for the remote deployment but uses only make and kubectl. https://github.com/davidbanham/kube_maker https://github.com/davidbanham/kube_maker I've been thinking about integrating a local minikube lately but haven't started yet. I may end using Skaffold as a backend or just deprecating kube_maker entirely if it becomes obvious that Skaffold is the way to go.

TraceyMorgannn 9y ago

Despite the fact that I discouraged my friends severally about hiring hackers to catch their cheating partners, I resorted to hiring a private investigator during my divorce. My husband hid his affair too well and I had no evidence to prove his infidelity to the court so I went on Craigslist to hire a professional whom I met and told my problem.. His services were topnotch which made working with him very easy, he got me everything I needed within 24 hours and it was amazing, I paid upfront despite my fears but I got results because I currently have my husband’s messages diverted to my phone, Call logs, Facebook messenger, WhatasApp, Instant chat, Viber, Skype Password, retrieved all his deleted messages, . David completed the hack without any physical access or installation on my husband’s phone so he never found out until I presented the information in court..Many thanks to David Consultants I got what I deserve Contact him for help. Gmail: Hotcyberclown @ gmail.com Text: +13478993017 Whatsapp Number: 12675265346 I know you will surely refer him.