4 ms·

I used to advocate for bulk subscriptions at every employer, but with the growth of cloud services and abstractions like Kubernetes I feel like I can't justify

I used to advocate for bulk subscriptions at every employer, but with the growth of cloud services and abstractions like Kubernetes I feel like I can't justify it any longer. We don't even run any Linux ourselves at my current employer. Pods and lambdas and databases-as-a-service. I can't help but think this trend will damage shared knowledge in the long run.

jchw 5y ago
When running on Kubernetes or Lambda, you are still directly interfacing with the Linux kernel, unless you’re under something like gVisor. I also don’t think the endgame of “cloud” is that literally nobody knows how to run servers; in fact, Kubernetes is one of those technologies that makes bare metal much more appealing. I think cloud has outsized appeal for now simply because things are moving so quickly that it’s difficult to keep up, and I feel like that is probably not forever. Even just for Kubernetes administration, it’s probably good to keep tabs on Linux, especially technologies like cgroups, even if your nodes are managed.
- mschuster91 5y ago
  > in fact, Kubernetes is one of those technologies that makes bare metal much more appealing Bare metal on Kubernetes is a PITA though to set up. For extremely small scale stuff in private labs, okay, that takes half an hour, but as soon as you are on the Internet, maybe have a couple rented bare metal machines, it gets nasty. For ingresses, you'll usually need to set up MetalLB which requires acquisition of a range of IP addresses and it only is capable of running one server (the "speaker") as the one and only entrypoint for all traffic. Not to mention you cannot have roles such as "internet-exposed node" and "not internet-exposed node", since MetalLB wants to run on all nodes. Also, a Kubernetes environment will need at least four separate network ranges - a network for the Pod IPs, a network for cluster IPs of services, one for the nodes themselves and if you want to expose stuff to the Internet you need a fifth one that holds the public-routed IPs. This is really funny if you are in a corporate environment that uses all three of the historic private ranges (10.0.0.0/8,172.16.0.0/12,192.168.0.0/16), good luck carving out ranges that do not result in routing conflicts. Mesosphere DC/OS was more usable in that regard as it had pre-defined the roles of nodes, included the load balancer as part of the platform and had only two IP ranges (node-internal and whatever the public IPs of the public nodes were)... whereas it's clear that the only designed/supported way to run a major Kubernetes deployment is to use a fully-managed Kubernetes provider. Everything else is a world of undocumented, spurious-bug-filled stuff and incomplete documentation.
  - dewey 5y ago
    > Bare metal on Kubernetes is a PITA though to set up. I think you are misunderstanding the previous comment. I read it as Kubernetes being so complicated (I enjoy it, but that's another discussion) to operate is making the regular bare metal operation more appealing. So people might choose the well understood bare metal over Kubernetes.
  - champtar 5y ago
    MetalLB is the best baremetal LB (for k8s) but far from perfect indeed. For the IP conflicts, there are other private IPv4 range that you can use, just pick something in the CGNAT range (100.64.0.0/10) or in 198.18.0.0/15 ("Used for benchmark testing of inter-network communications between two separate subnets")
    - mschuster91 5y ago
      The best thing about HN: You learn something new every day. Thanks!
- lima 5y ago
  Even with gVisor you're still depending on things like the host kernel's scheduler and memory management.
c0l0 5y ago
I am pretty certain that > I can't help but think this trend will damage shared knowledge in the long run. is an understatement. There will be a few people in the know on how to run "the cloud", and lots and lots of people consuming their services, at bewildering premiums.
5e92cb50239222b 5y ago
The prevalence of this attitude is why we have things like Heartbleed and log4shell.
akuchling 5y ago
LWN also has coverage of language runtimes and other projects. Recent stories include one about python-dev discussion of adding a literal syntax for frozensets (https://lwn.net/SubscriberLink/881599/ae07750ec86ed3cc/ https://lwn.net/SubscriberLink/881599/ae07750ec86ed3cc/), a new OpenSSH feature on restricting agent keys (https://lwn.net/Articles/880458/ https://lwn.net/Articles/880458/), Lessons from log4j (https://lwn.net/Articles/878570/ https://lwn.net/Articles/878570/), etc. I often skip the really in-depth kernel articles, since it makes no difference to anything I do if Linux gets a new slab allocator or whatever, but still find my subscription valuable for hearing about new project releases, as well as these deep project dives.

jchw 5y ago

When running on Kubernetes or Lambda, you are still directly interfacing with the Linux kernel, unless you’re under something like gVisor. I also don’t think the endgame of “cloud” is that literally nobody knows how to run servers; in fact, Kubernetes is one of those technologies that makes bare metal much more appealing. I think cloud has outsized appeal for now simply because things are moving so quickly that it’s difficult to keep up, and I feel like that is probably not forever. Even just for Kubernetes administration, it’s probably good to keep tabs on Linux, especially technologies like cgroups, even if your nodes are managed.

mschuster91 5y ago
> in fact, Kubernetes is one of those technologies that makes bare metal much more appealing Bare metal on Kubernetes is a PITA though to set up. For extremely small scale stuff in private labs, okay, that takes half an hour, but as soon as you are on the Internet, maybe have a couple rented bare metal machines, it gets nasty. For ingresses, you'll usually need to set up MetalLB which requires acquisition of a range of IP addresses and it only is capable of running one server (the "speaker") as the one and only entrypoint for all traffic. Not to mention you cannot have roles such as "internet-exposed node" and "not internet-exposed node", since MetalLB wants to run on all nodes. Also, a Kubernetes environment will need at least four separate network ranges - a network for the Pod IPs, a network for cluster IPs of services, one for the nodes themselves and if you want to expose stuff to the Internet you need a fifth one that holds the public-routed IPs. This is really funny if you are in a corporate environment that uses all three of the historic private ranges (10.0.0.0/8,172.16.0.0/12,192.168.0.0/16), good luck carving out ranges that do not result in routing conflicts. Mesosphere DC/OS was more usable in that regard as it had pre-defined the roles of nodes, included the load balancer as part of the platform and had only two IP ranges (node-internal and whatever the public IPs of the public nodes were)... whereas it's clear that the only designed/supported way to run a major Kubernetes deployment is to use a fully-managed Kubernetes provider. Everything else is a world of undocumented, spurious-bug-filled stuff and incomplete documentation.
- dewey 5y ago
  > Bare metal on Kubernetes is a PITA though to set up. I think you are misunderstanding the previous comment. I read it as Kubernetes being so complicated (I enjoy it, but that's another discussion) to operate is making the regular bare metal operation more appealing. So people might choose the well understood bare metal over Kubernetes.
- champtar 5y ago
  MetalLB is the best baremetal LB (for k8s) but far from perfect indeed. For the IP conflicts, there are other private IPv4 range that you can use, just pick something in the CGNAT range (100.64.0.0/10) or in 198.18.0.0/15 ("Used for benchmark testing of inter-network communications between two separate subnets")
  - mschuster91 5y ago
    The best thing about HN: You learn something new every day. Thanks!
lima 5y ago
Even with gVisor you're still depending on things like the host kernel's scheduler and memory management.

c0l0 5y ago

I am pretty certain that > I can't help but think this trend will damage shared knowledge in the long run. is an understatement. There will be a few people in the know on how to run "the cloud", and lots and lots of people consuming their services, at bewildering premiums.

5e92cb50239222b 5y ago

The prevalence of this attitude is why we have things like Heartbleed and log4shell.

akuchling 5y ago

LWN also has coverage of language runtimes and other projects. Recent stories include one about python-dev discussion of adding a literal syntax for frozensets (https://lwn.net/SubscriberLink/881599/ae07750ec86ed3cc/ https://lwn.net/SubscriberLink/881599/ae07750ec86ed3cc/), a new OpenSSH feature on restricting agent keys (https://lwn.net/Articles/880458/ https://lwn.net/Articles/880458/), Lessons from log4j (https://lwn.net/Articles/878570/ https://lwn.net/Articles/878570/), etc. I often skip the really in-depth kernel articles, since it makes no difference to anything I do if Linux gets a new slab allocator or whatever, but still find my subscription valuable for hearing about new project releases, as well as these deep project dives.