4 ms·

It seems you have not discovered the wide possibilities of kubernetes yet :) How are you setting up your cluster? A) Kubernetes abstracts the networks solutio

by eicnix 9y ago

It seems you have not discovered the wide possibilities of kubernetes yet :) How are you setting up your cluster? A) Kubernetes abstracts the networks solution so people can choose between multiple implementations. Some people already use openstack and want to use their openvswitch network for their containers others want to use a pure container network like flannel. Kubernetes distributions usually come with a integrated network solution. B) You can still retrieve the logs from a container is CrashLoopBackOffState. You can even retrieve the logs from the previously failed container by using the `kubectl logs <container> --previous`. Applications can write information about their failure to /dev/termination-log which can be used to debug the failure. C) You can define the port yourself otherwise kubernetes defaults to a random port to avoid port conflicts. The recommended way to expose HTTP services would be by using ingress. D) You can run an instance on every node by using a daemonset. E) Are you talking about outbound traffic from your containers to an external system? You would need to configure this in the container engine and the container itself. I had little problem doing this in an enterprise environment that required an http proxy for all external communication. F) You can attach you existing SAN solution over iSCSI, fibre channel or even NFS. Another solution would be to run a distribution storage like ceph or glusterfs in kubernetes for kubernetes. You then provision persistent volumes that are attached to the node your pod is running on. If your pod is rescheduled the volume will be moved too. G) If you have resource requests/limits set kubernetes will not schedule your pods if you have no resources available.

erikb 9y ago
I'll read all of the details you provided. Thanks a lot. It may be lack of knowledge in our team, not kubernetes. Sorry if I was too frustrated and blaming the tool instead of my lack of knowledge. Re proxy: Not only that, but also. Let's say you have single instance deployment, and run kube-dashboard. You want to access the dashboard on localhost:8080 so you start the kube-proxy shell command. What actually happens is that the go code underneath kube-proxy will redirect the request through your $http_proxy, even if localhost, the dashboard's internal address, and your hosts external address are in the $no_proxy environment. And if your network proxy doesn't allow the dashboard's port you get a HTTP 403 instead of the dashboard. Re storage: We have a cluster with 10+ nodes, each with 5+ disks a 10 TB. How would you make sure that your software talks to the correct disk, after it gets restarted and may end up on another node? Re limits: The highest goal is not avoiding ressource exhaustion, the highest goal is service continuity, though. It can all run on 100% and die. no problem. I just need to know that I should exchange the first few disks/processors/machines before the customer facing service slows down.
- eicnix 9y ago
  @Proxy I am not sure what the issue is that you are facing. kubectl proxy should create a tcp-proxy from your loopback device to the pod. So there should be no http_proxy involved. If you think a kubernetes component does not respect the no_proxy setting you should create an issue. @storage If you want to move your disks with your containers you need an additional storage system(SAN, cloud provider or distributed storage in kubernetes). Then you create persistence volumes in kubernetes that references a disk from the storage provider. This allows you to assign this disk to a pod with a persistence volume claim. The disk that is linked to the container through persistent volume and persistent volume claim will be moved on the same node the container is scheduled. If you want to run stateful workloads on kubernetes I would advice you to use a storage system. You can use local disks but you loose some of the flexibility that you gain from kubernetes by tying your containers to certain nodes with the data. There is also work being done on improving the handling of local storage to treat it more like a resource and introduce separation by using local volume per kubernetes local disk volumes.
- pas 9y ago
  The local persistent volume claim is a completely sane and valid feature, but it was not implemented, because no one stepped up to do it initially. But it'll be alpha in 1.7: https://github.com/kubernetes/features/issues/121 https://github.com/kubernetes/features/issues/121

by eicnix 9y ago

erikb 9y ago

I'll read all of the details you provided. Thanks a lot. It may be lack of knowledge in our team, not kubernetes. Sorry if I was too frustrated and blaming the tool instead of my lack of knowledge. Re proxy: Not only that, but also. Let's say you have single instance deployment, and run kube-dashboard. You want to access the dashboard on localhost:8080 so you start the kube-proxy shell command. What actually happens is that the go code underneath kube-proxy will redirect the request through your $http_proxy, even if localhost, the dashboard's internal address, and your hosts external address are in the $no_proxy environment. And if your network proxy doesn't allow the dashboard's port you get a HTTP 403 instead of the dashboard. Re storage: We have a cluster with 10+ nodes, each with 5+ disks a 10 TB. How would you make sure that your software talks to the correct disk, after it gets restarted and may end up on another node? Re limits: The highest goal is not avoiding ressource exhaustion, the highest goal is service continuity, though. It can all run on 100% and die. no problem. I just need to know that I should exchange the first few disks/processors/machines before the customer facing service slows down.

eicnix 9y ago
@Proxy I am not sure what the issue is that you are facing. kubectl proxy should create a tcp-proxy from your loopback device to the pod. So there should be no http_proxy involved. If you think a kubernetes component does not respect the no_proxy setting you should create an issue. @storage If you want to move your disks with your containers you need an additional storage system(SAN, cloud provider or distributed storage in kubernetes). Then you create persistence volumes in kubernetes that references a disk from the storage provider. This allows you to assign this disk to a pod with a persistence volume claim. The disk that is linked to the container through persistent volume and persistent volume claim will be moved on the same node the container is scheduled. If you want to run stateful workloads on kubernetes I would advice you to use a storage system. You can use local disks but you loose some of the flexibility that you gain from kubernetes by tying your containers to certain nodes with the data. There is also work being done on improving the handling of local storage to treat it more like a resource and introduce separation by using local volume per kubernetes local disk volumes.
pas 9y ago
The local persistent volume claim is a completely sane and valid feature, but it was not implemented, because no one stepped up to do it initially. But it'll be alpha in 1.7: https://github.com/kubernetes/features/issues/121 https://github.com/kubernetes/features/issues/121