5 ms·

When people ask for something to be lightweight, what they really mean is "keep the things I need and remove the things I don't..." A lot of these "why you don'

by nova22033 5y ago

When people ask for something to be lightweight, what they really mean is "keep the things I need and remove the things I don't..." A lot of these "why you don't need kubernetes" articles should really be "do you really need containers for your 10 user app?"

chrisandchris 5y ago
I think one aspect your reasoning does not properly respect is that running things in container is not all about scaling, but also about easy of deployment. A 10-user-app probably doesn't need k8s, but the advantage of packing it into a container and just starting that container (on a single machine) is very nice. It's so easy to have multiple apps with different languages (or even language versions) on a single host without worrying how to install 3 different PHP versions on that machine.
- stepbeek 5y ago
  Perhaps as a result of incorrectly classifying K8S or K8SaaS as a PaaS when it’s very far from it. Heroku offers the same - or better - devex without the operational burden.
- kasey_junk 5y ago
  It’s worth noting that advantage (and in my belief the large adoption of containers generally) is only valuable to interpreted languages that dont allow static linking. For statically compiled binaries I’ve not worried about different versions of libraries for my binaries for 20 years (and the technology is decades older than that). I do appreciate the docker file format but largely because it’s become used for true virtualization systems like firecracker. Plain Linux “containers” largely offer me very little.
  - pjmlp 5y ago
    Worse, except for stuff like what was being done at Xerox, static linking was the only way for compiled languages. Also on desktop systems, embedding resources into the binaries exists since the early 1980's, or just deliver every in the same directory like NeXTSTEP bundles, or what is called xcopy deployment on the PC world on homage how we used to do it on MS-DOS days.
  - zozbot234 5y ago
    Containers are still valuable for statically-linked code, as ways of comprehensively namespacing resources. (In fact, the converse is true; containers are built on the resource-namespacing features of modern Linux, which in turn were inspired by Plan9 a distributed operating system building on *nix).
    - kasey_junk 5y ago
      Right. But I’ve been able to use resource limits for my Linux processes for far longer than containers were popularized. With my processes anyway that’s the big win with namespaces. That is to say, if you control the hosts and process then the container features are available with much less ceremony while if you are hosting someone else’s process (or they you) containers don’t provide enough isolation.
      - nonameiguess 5y ago
        You're thinking of cgroups, which is what you gives you per-process group resource limits. Namespacing gives you quite a bit more than that. You can have all services running as the same user but still not able to see each other's data. You can have all services get a unique IP address even though they're on the same kernel. You can have them all listen on port 80. They can all read from the same socket file from their perspective but it won't actually be the same file. You can have them all use different DNS providers even though they're just delegating to /etc/resolv.conf. You can't do that with just static linking. Containers give you quite a bit more than incompatible dependencies being able to run on the same host, including features that can only be provided by a kernel, not a language runtime. They allow you to use arbitrarily many different languages. You can run third-party applications you can't rewrite in Java to get a WAR or force them to statically link.
  - hedora 5y ago
    I think the more fundamental advantage is on the dev side: How do you reproducibly build that static binary in a dozen different developer's environments, and in CI? In fact, containers kind of suck in prod environments. For example: - Kernel/binary mismatches break vDSO, and gettimeofday becomes 100x more expensive. - Current apache can't run in Synology Docker any more, because they require a new kernel secure RNG (even for http). - There's no reasonable built in story for persistent storage. VM's and bare metal solved these issues long ago.
- pjmlp 5y ago
  Back in the day we used to call it EAR files, or static linking with resource files embedded.
  - goodpoint 5y ago
    ...or a tarball, or a deb file or an rpm.
  - 0x445442 5y ago
    Thank you. If I had a dime for every time I've uttered, how is this easier or more efficient than "deploy war", over the past 5-7 years I'd be a very wealthy man.
    - acdha 5y ago
      To me, it really comes back to how badly the Java system was managed with regard to the quality of its tools. Portability and security would always have been a draw for every other language but even the Java projects I know used containers for basically two reasons: Tomcat was unrewarding to use and the Java web stack had a lot of complexity which users were forced to deal with. If Sun had spent, say, 5% of their marketing budget on making the user experience better I don't think there would have been anywhere near the pressure to avoid it.
      - pjmlp 5y ago
        Somehow I get the feeling that most Java haters never had the pleasure to deal with CORBA and COM/DCOM or MTS. So those newcomers never got the point why many of us actually enjoyed using Application Servers.
        acdha 5y ago
        I did, actually, have the misfortune of using COM (and touched CORBA enough not to want to use it more). I don’t hate Java but, having started using it in 1995, it felt like the community was prone to sweeping ambitions which exceeded the available time or appetite to improve further. One challenge is that while doing everything yourself allows the most control, it also means that improvements only come from within the smaller community of Java users.
koffiezet 5y ago
K8s makes small-scale hard, but large-scale easy. So no, for a 10 user app you probably won't need it, and would make your life a lot harder. But one of the things many people don't grasp is that scale comes in a few forms. The obvious everyone thinks about is the millions of users thing, but from what I've seen, that's not the most important-one that k8s enables - it's the organisational thing. Enabling self-servicing for a dozen+ dev-teams, having to manage 20+ totally different applications or just a microservice platform? That's where it becomes more practical to think about Kubernetes, because it forces the applications (and everybody involved) to streamline things, and you have to treat it as cattle, not pets. Metrics, logging, monitoring, failover, zero-downtime upgrades, scaling, managing secrets, network policies, auditing and security, platform upgrades, even the "replace & scrap" kind become WAY easier to manage by a very small team of people if they know what they're doing (that's another subject though).
mountainriver 5y ago
Yeah I run kube for all my small apps and it’s great. It still seems far better than the alternatives. I don’t buy this “kube is only for enterprise” story
closeparen 5y ago
Scheduling a large heterogeneous workload on a single cluster is both a very deep problem, and a very lucrative one for large companies to solve, in order to manage their capacity/utilization. Many Kubernetes deployments are for a homogeneous workload, or a heterogeneous workload is split up into many Kubernetes clusters. In these cases you are bringing in a whole lot of very complex infrastructure for bin-packing, isolating workloads from each other, etc. that you are not benefitting from.

chrisandchris 5y ago

I think one aspect your reasoning does not properly respect is that running things in container is not all about scaling, but also about easy of deployment. A 10-user-app probably doesn't need k8s, but the advantage of packing it into a container and just starting that container (on a single machine) is very nice. It's so easy to have multiple apps with different languages (or even language versions) on a single host without worrying how to install 3 different PHP versions on that machine.

stepbeek 5y ago
Perhaps as a result of incorrectly classifying K8S or K8SaaS as a PaaS when it’s very far from it. Heroku offers the same - or better - devex without the operational burden.
kasey_junk 5y ago
It’s worth noting that advantage (and in my belief the large adoption of containers generally) is only valuable to interpreted languages that dont allow static linking. For statically compiled binaries I’ve not worried about different versions of libraries for my binaries for 20 years (and the technology is decades older than that). I do appreciate the docker file format but largely because it’s become used for true virtualization systems like firecracker. Plain Linux “containers” largely offer me very little.
- pjmlp 5y ago
  Worse, except for stuff like what was being done at Xerox, static linking was the only way for compiled languages. Also on desktop systems, embedding resources into the binaries exists since the early 1980's, or just deliver every in the same directory like NeXTSTEP bundles, or what is called xcopy deployment on the PC world on homage how we used to do it on MS-DOS days.
- zozbot234 5y ago
  Containers are still valuable for statically-linked code, as ways of comprehensively namespacing resources. (In fact, the converse is true; containers are built on the resource-namespacing features of modern Linux, which in turn were inspired by Plan9 a distributed operating system building on *nix).
  - kasey_junk 5y ago
    Right. But I’ve been able to use resource limits for my Linux processes for far longer than containers were popularized. With my processes anyway that’s the big win with namespaces. That is to say, if you control the hosts and process then the container features are available with much less ceremony while if you are hosting someone else’s process (or they you) containers don’t provide enough isolation.
    - nonameiguess 5y ago
      You're thinking of cgroups, which is what you gives you per-process group resource limits. Namespacing gives you quite a bit more than that. You can have all services running as the same user but still not able to see each other's data. You can have all services get a unique IP address even though they're on the same kernel. You can have them all listen on port 80. They can all read from the same socket file from their perspective but it won't actually be the same file. You can have them all use different DNS providers even though they're just delegating to /etc/resolv.conf. You can't do that with just static linking. Containers give you quite a bit more than incompatible dependencies being able to run on the same host, including features that can only be provided by a kernel, not a language runtime. They allow you to use arbitrarily many different languages. You can run third-party applications you can't rewrite in Java to get a WAR or force them to statically link.
- hedora 5y ago
  I think the more fundamental advantage is on the dev side: How do you reproducibly build that static binary in a dozen different developer's environments, and in CI? In fact, containers kind of suck in prod environments. For example: - Kernel/binary mismatches break vDSO, and gettimeofday becomes 100x more expensive. - Current apache can't run in Synology Docker any more, because they require a new kernel secure RNG (even for http). - There's no reasonable built in story for persistent storage. VM's and bare metal solved these issues long ago.
pjmlp 5y ago
Back in the day we used to call it EAR files, or static linking with resource files embedded.
- goodpoint 5y ago
  ...or a tarball, or a deb file or an rpm.
- 0x445442 5y ago
  Thank you. If I had a dime for every time I've uttered, how is this easier or more efficient than "deploy war", over the past 5-7 years I'd be a very wealthy man.
  - acdha 5y ago
    To me, it really comes back to how badly the Java system was managed with regard to the quality of its tools. Portability and security would always have been a draw for every other language but even the Java projects I know used containers for basically two reasons: Tomcat was unrewarding to use and the Java web stack had a lot of complexity which users were forced to deal with. If Sun had spent, say, 5% of their marketing budget on making the user experience better I don't think there would have been anywhere near the pressure to avoid it.
    - pjmlp 5y ago
      Somehow I get the feeling that most Java haters never had the pleasure to deal with CORBA and COM/DCOM or MTS. So those newcomers never got the point why many of us actually enjoyed using Application Servers.
      - acdha 5y ago
        I did, actually, have the misfortune of using COM (and touched CORBA enough not to want to use it more). I don’t hate Java but, having started using it in 1995, it felt like the community was prone to sweeping ambitions which exceeded the available time or appetite to improve further. One challenge is that while doing everything yourself allows the most control, it also means that improvements only come from within the smaller community of Java users.

koffiezet 5y ago

K8s makes small-scale hard, but large-scale easy. So no, for a 10 user app you probably won't need it, and would make your life a lot harder. But one of the things many people don't grasp is that scale comes in a few forms. The obvious everyone thinks about is the millions of users thing, but from what I've seen, that's not the most important-one that k8s enables - it's the organisational thing. Enabling self-servicing for a dozen+ dev-teams, having to manage 20+ totally different applications or just a microservice platform? That's where it becomes more practical to think about Kubernetes, because it forces the applications (and everybody involved) to streamline things, and you have to treat it as cattle, not pets. Metrics, logging, monitoring, failover, zero-downtime upgrades, scaling, managing secrets, network policies, auditing and security, platform upgrades, even the "replace & scrap" kind become WAY easier to manage by a very small team of people if they know what they're doing (that's another subject though).

mountainriver 5y ago

Yeah I run kube for all my small apps and it’s great. It still seems far better than the alternatives. I don’t buy this “kube is only for enterprise” story

closeparen 5y ago

Scheduling a large heterogeneous workload on a single cluster is both a very deep problem, and a very lucrative one for large companies to solve, in order to manage their capacity/utilization. Many Kubernetes deployments are for a homogeneous workload, or a heterogeneous workload is split up into many Kubernetes clusters. In these cases you are bringing in a whole lot of very complex infrastructure for bin-packing, isolating workloads from each other, etc. that you are not benefitting from.