7 ms·

I cannot give enough thanks to everyone that works so hard on Kubernetes. The speed of these releases, community support, industry adoption, feature set, and ea

I cannot give enough thanks to everyone that works so hard on Kubernetes. The speed of these releases, community support, industry adoption, feature set, and ease of use are beyond anything I could have asked for. I started a startup about 18 months ago and our product requires a massive amount data pipelining. The infrastructure we have running on Kubernetes is beyond what a small group should be able to deploy and maintain by ourselves, but Kubernetes makes it not only possible, but enjoyable (really). To everyone involved (and there are a lot): Thank you SO much. I can't wait to see what 2018 looks like for Kubernetes.

ec109685 9y ago
What type of infrastructures do you run kubernetes on?
- slap_shot 9y ago
  We run Kubernetes on all the three major clouds: AWS, GCP and Azure.
cirowrc 9y ago
That's so great to hear! I love to see how it empowers small teams to deliver big stuff. What was the trickiest part for you when it comes to real production usage?
- linkmotif 9y ago
  Using it on GKE. Literally no tricky parts except figuring out how to keep my images on GCR in sync with my manifests. For that this tool is quite helpful: https://github.com/dminkovsky/kube-cloud-build https://github.com/dminkovsky/kube-cloud-build. Otherwise it's been such a pleasure. PVs and StatefulSets are a total game changer. Looking forward to 2018. EDIT: I should say RBAC has been a little frustrating. But obviously it's for the better.
linkmotif 9y ago
Couldn’t have said it better myself. I am so grateful.
ericand 9y ago
Great post. Could you describe further what you mean by data pipelining? Is this something that you could also use Spark, Kafka for? I'd be interested to know if Kubernetes would work for me as well.
- deleted 9y ago
  [deleted]
- lobster_johnson 9y ago
  One promising piece of tech is Pachyderm [1], which runs natively on Kubernetes. It allows you to set up pipelines of batch jobs (no streaming yet, as far as I know) that operate on immutable, versioned data. There are some gotchas, though. For one [2], it requires read-write access to your storage buckets (!), and so on GKE you have to modify your nodepool's access scopes to give the whole node read-write access to all your buckets. Not is this terrible security practice, but access scopes are deprecated on Google Cloud. You should always grant access through service accounts, not scopes. We spent some time fighting this, as there's literally no way to make Pachyderm use a service account. There are some other niggling issues, and the Pachyderm team has been less than enthuastic in addressing them (e.g. see [3], which has zero activity despite being absolutely crucial to run in production), but it does seem quite promising. The architecture, at least, is sound. [1] http://pachyderm.io http://pachyderm.io [2] https://github.com/pachyderm/pachyderm/issues/2538 https://github.com/pachyderm/pachyderm/issues/2538 [3] https://github.com/pachyderm/pachyderm/issues/2537 https://github.com/pachyderm/pachyderm/issues/2537
  - jdoliner 9y ago
    Hey lobster_johnson, Pachyderm CEO here. Thanks for the shout out and we're glad you feel Pachyderm is promising for these use cases. I just wanted to clarify / respond to a few things. > no streaming yet, as far as I know Pachyderm actually does do streaming, there's nothing in the API about it because it all happens automatically. This is one of the huge advantages of immutable data, the system can automatically detect if it's already done a computation by looking at a hash of the input and the code. The system will never do the same computation twice, all pipelines are streaming by default. Sorry we've been less than enthusiastic in addressing these niggling issues. The truth is it's pretty hard to be enthusiastic in addressing issues for specific deployment scenarios. AFAIK the Google Cloud Storage library we're using doesn't support service accounts, the whole library has been deprecated, the second GCS library we've had deprecated underneath us. So upgrading that is a bit of a pain but the bigger pain is that now users with existing Google clusters will have their clusters break under them unless we can figure out a way to get scopes and service accounts to both work. And we have every reason to believe that all of this will be deprecated within the next 6 months and we'll be on to our 4th GCS library. And that's only one deployment scenario, we're doing the same thing for AWS, Azure and a host of local deployment options. Anyways, these are our problems to solve, and we will solve them. We're thrilled that you want to use Pachyderm and we want to make that experience as smooth as possible. We just need to ask for a bit of patience regarding these deployment scenarios.
jaz46 9y ago
Have you checked out Pachyderm (disclaimer: my company) for data pipelining on Kubernetes? Our product has been available since the first Kubernetes release and is specifically designed for data analysis and pipelining tasks on Kubernetes. github.com/pachyderm/pachyderm pachyderm.io
lemoncucumber 9y ago
As much as I like kubernetes, I find the whole ecosystem to be frustratingly immature and unreliable. I'm sure it's trending in the right direction with respect to reliability, but it just doesn't feel like it's there yet. Minikube seems especially buggy and is missing some obvious features.
- akvadrako 9y ago
  Well minikube is not for production, so maybe it's buginess is evidence they are focusing their time on the more important components.

ec109685 9y ago

What type of infrastructures do you run kubernetes on?

slap_shot 9y ago
We run Kubernetes on all the three major clouds: AWS, GCP and Azure.

cirowrc 9y ago

That's so great to hear! I love to see how it empowers small teams to deliver big stuff. What was the trickiest part for you when it comes to real production usage?

linkmotif 9y ago
Using it on GKE. Literally no tricky parts except figuring out how to keep my images on GCR in sync with my manifests. For that this tool is quite helpful: https://github.com/dminkovsky/kube-cloud-build https://github.com/dminkovsky/kube-cloud-build. Otherwise it's been such a pleasure. PVs and StatefulSets are a total game changer. Looking forward to 2018. EDIT: I should say RBAC has been a little frustrating. But obviously it's for the better.

linkmotif 9y ago

Couldn’t have said it better myself. I am so grateful.

ericand 9y ago

Great post. Could you describe further what you mean by data pipelining? Is this something that you could also use Spark, Kafka for? I'd be interested to know if Kubernetes would work for me as well.

deleted 9y ago
[deleted]
lobster_johnson 9y ago
One promising piece of tech is Pachyderm [1], which runs natively on Kubernetes. It allows you to set up pipelines of batch jobs (no streaming yet, as far as I know) that operate on immutable, versioned data. There are some gotchas, though. For one [2], it requires read-write access to your storage buckets (!), and so on GKE you have to modify your nodepool's access scopes to give the whole node read-write access to all your buckets. Not is this terrible security practice, but access scopes are deprecated on Google Cloud. You should always grant access through service accounts, not scopes. We spent some time fighting this, as there's literally no way to make Pachyderm use a service account. There are some other niggling issues, and the Pachyderm team has been less than enthuastic in addressing them (e.g. see [3], which has zero activity despite being absolutely crucial to run in production), but it does seem quite promising. The architecture, at least, is sound. [1] http://pachyderm.io http://pachyderm.io [2] https://github.com/pachyderm/pachyderm/issues/2538 https://github.com/pachyderm/pachyderm/issues/2538 [3] https://github.com/pachyderm/pachyderm/issues/2537 https://github.com/pachyderm/pachyderm/issues/2537
- jdoliner 9y ago
  Hey lobster_johnson, Pachyderm CEO here. Thanks for the shout out and we're glad you feel Pachyderm is promising for these use cases. I just wanted to clarify / respond to a few things. > no streaming yet, as far as I know Pachyderm actually does do streaming, there's nothing in the API about it because it all happens automatically. This is one of the huge advantages of immutable data, the system can automatically detect if it's already done a computation by looking at a hash of the input and the code. The system will never do the same computation twice, all pipelines are streaming by default. Sorry we've been less than enthusiastic in addressing these niggling issues. The truth is it's pretty hard to be enthusiastic in addressing issues for specific deployment scenarios. AFAIK the Google Cloud Storage library we're using doesn't support service accounts, the whole library has been deprecated, the second GCS library we've had deprecated underneath us. So upgrading that is a bit of a pain but the bigger pain is that now users with existing Google clusters will have their clusters break under them unless we can figure out a way to get scopes and service accounts to both work. And we have every reason to believe that all of this will be deprecated within the next 6 months and we'll be on to our 4th GCS library. And that's only one deployment scenario, we're doing the same thing for AWS, Azure and a host of local deployment options. Anyways, these are our problems to solve, and we will solve them. We're thrilled that you want to use Pachyderm and we want to make that experience as smooth as possible. We just need to ask for a bit of patience regarding these deployment scenarios.

jaz46 9y ago

Have you checked out Pachyderm (disclaimer: my company) for data pipelining on Kubernetes? Our product has been available since the first Kubernetes release and is specifically designed for data analysis and pipelining tasks on Kubernetes. github.com/pachyderm/pachyderm pachyderm.io

lemoncucumber 9y ago

As much as I like kubernetes, I find the whole ecosystem to be frustratingly immature and unreliable. I'm sure it's trending in the right direction with respect to reliability, but it just doesn't feel like it's there yet. Minikube seems especially buggy and is missing some obvious features.

akvadrako 9y ago
Well minikube is not for production, so maybe it's buginess is evidence they are focusing their time on the more important components.