5 ms·

If you prefer to read, this blogpost explains the same project: https://merlijn.sebrechts.be/blog/2022-09-05-wasm-k8s-controllers/ https://merlijn.sebrechts.be/

If you prefer to read, this blogpost explains the same project: https://merlijn.sebrechts.be/blog/2022-09-05-wasm-k8s-controllers/ https://merlijn.sebrechts.be/blog/2022-09-05-wasm-k8s-contro...

pickledish 4y ago
Thank you for the link! > Developers want to use Kubernetes in the edge, but it uses too much memory for most devices ...wow, I had absolutely no idea that this was a thing people wanted or a problem folks were out there trying to solve. No judgement, who am I to say with so little information whether it’s a good idea or not, just — what a concept eh
- MuffinFlavored 4y ago
  What does the edge mean? I've always understood it as, you are most likely using cloud hosting of some sort and edge means "an instance of some code running in the geolocation closest to your customers"? I can't think of many projects I've worked on where I've come across that need for performance. How do you handle connecting to a database or cache? Do you have a bunch of databases shaded at the edge? Does the edge talk back to your database that isn't at the edge? Buzzwords: Kubernetes, WebAssembly, edge Not saying that with any negative connotation. Interested in learning typical every day problems that require this. I thought edge was for like... get static HTML/CSS/JS/images/video to customers as quickly as possible (sub 50ms)
  - coredog64 4y ago
    Chik-Fil-A and Target run k8s in each store. CFA’s experience doing so had them providing lessons-learned feedback to the DoD so that they could replicate it for on-aircraft software deployments.
    - twalla 4y ago
      On-aircraft k8s gives CrashLoopBackoff a whole new meaning.
      - agoose77 4y ago
        I can only picture how one of those three terms would work
  - wrs 4y ago
    I would define it as “the closest place to the end user that you can deploy software”. In the case of a web app, yes, that may be a CDN point of presence. In the case of Walmart, there are servers in the store running business apps. On a farm there might be something running on a fancy tractor.
    - MuffinFlavored 4y ago
      If the servers running in-store at Walmart have to talk to a server hosted by Walmart that isn't in store, what's the benefit?
      - mesebrec 4y ago
        * much lower latency for most operations * the ability for graceful degradation when the network or cloud is down. You don't want the checkouts to stop working just because the cloud is down. If literally every action requires a round trip to the cloud, then edge computing isn't for you. That's not the case for a lot of functionality, however.
  - azakai 4y ago
    > What does the edge mean? I was confused by that too. I've seen "edge" mean certain things in the web space (Cloudflare workers, Fastly compute, etc.), but in this article it apparently means "IoT" (Internet of Things, tiny devices with internet connections).
  - cpitman 4y ago
    It means varying levels of "not a central data center". For anyone wanting to do edge, you first need to understand what it means to them. Sometimes the edge is distributed sites that still have a rack of hardware, say in factories or distribution centers. Sometimes it is one or two computers at all of your stores, but still "general purpose" hardware. And sometimes it is purpose built devices, like Point of Sales Systems or a radio on every train. Etc. Each form factor has different considerations, and each application different design patterns. For example, at a retailer I worked at each store had a message broker, and orders would be sent through that broker up to the central data center. If/when the store was disconnected, it could continue to queue up orders until the connection was restored. As opposed to housing everything in the data center, and needing to fall back to a paper process if the internet goes down.
  - factorialboy 4y ago
    > What does the edge mean? Server with limited capabilities located near you.
  - mesebrec 4y ago
    For this project, I'm specifically referring to low-power devices sitting inside of the user's premises. For example, home automation gateways, factory machines, drones etc. I'm not really talking about CDNs or offerings such as Fastly Compute@edge. Right now, most of these devices have bespoke solutions for deploying and updating software. Switching those devices to Kubernetes allows developers to use the same tools and processes from the cloud.
  - Arbortheus 4y ago
    I think in this scenario it just means a localised server. I imagine the problem people are trying to solve is just wanting to manage infrastructure in modern GitOps/Infrastructure-as-code ways. Ultimately people want to deploy all their tech infrastructure in such a way that they can address the "treat X as cattle, not pets" problem. Kubernetes is one way to do this.
- mesebrec 4y ago
  For clarification, I'm talking about devices such as home automation gateways, planes, and production lines. These devices often have bespoke toolchains for deploying and managing software on them. This is an additional barrier of entry: cloud teams need to learn a new toolchain to move to the edge. Moreover, these toolchains often need changes to how release management works. So naturally, the question arises "why can't we just use the same tools that we use in the cloud?" Right now, Kubernetes isn't the greatest fit for the edge. Distributions such as K3s and KubeEdge try to solve some of the problems with Kubernetes itself, but that's only half of the picture. A large advantage of Kubernetes is the ecosystem of third party operators. So that's what we're trying to address.
- Felmioytfr 4y ago
  K8s doesn't need to do unlimited things. Why run docker if you can push k8s further down with perhaps less features? K8s could be as lightweight as nomad.
  - p_l 4y ago
    There's separated kubernetes api-only server (to run just the control plane) and a virtual kubelet implementation that actually runs systemd services.
    - Felmioytfr 4y ago
      Do you have a name or link?
      - p_l 4y ago
        KCP (api-only server): https://github.com/kcp-dev/kcp https://github.com/kcp-dev/kcp SystemK (virtual kubelet): https://github.com/virtual-kubelet/systemk https://github.com/virtual-kubelet/systemk
yjftsjthsd-h 4y ago
More interesting to me is the line in the first graph labeled "no isolation", which has lower memory still. I would expect operators to be trusted, so I don't mind the security implications running them without isolation, if that's what that really means. Is that a viable way to reduce the resource use even further?

If you prefer to read, this blogpost explains the same project: https://merlijn.sebrechts.be/blog/2022-09-05-wasm-k8s-controllers/ https://merlijn.sebrechts.be/blog/2022-09-05-wasm-k8s-contro...

pickledish 4y ago

Thank you for the link! > Developers want to use Kubernetes in the edge, but it uses too much memory for most devices ...wow, I had absolutely no idea that this was a thing people wanted or a problem folks were out there trying to solve. No judgement, who am I to say with so little information whether it’s a good idea or not, just — what a concept eh

MuffinFlavored 4y ago
What does the edge mean? I've always understood it as, you are most likely using cloud hosting of some sort and edge means "an instance of some code running in the geolocation closest to your customers"? I can't think of many projects I've worked on where I've come across that need for performance. How do you handle connecting to a database or cache? Do you have a bunch of databases shaded at the edge? Does the edge talk back to your database that isn't at the edge? Buzzwords: Kubernetes, WebAssembly, edge Not saying that with any negative connotation. Interested in learning typical every day problems that require this. I thought edge was for like... get static HTML/CSS/JS/images/video to customers as quickly as possible (sub 50ms)
- coredog64 4y ago
  Chik-Fil-A and Target run k8s in each store. CFA’s experience doing so had them providing lessons-learned feedback to the DoD so that they could replicate it for on-aircraft software deployments.
  - twalla 4y ago
    On-aircraft k8s gives CrashLoopBackoff a whole new meaning.
    - agoose77 4y ago
      I can only picture how one of those three terms would work
- wrs 4y ago
  I would define it as “the closest place to the end user that you can deploy software”. In the case of a web app, yes, that may be a CDN point of presence. In the case of Walmart, there are servers in the store running business apps. On a farm there might be something running on a fancy tractor.
  - MuffinFlavored 4y ago
    If the servers running in-store at Walmart have to talk to a server hosted by Walmart that isn't in store, what's the benefit?
    - mesebrec 4y ago
      * much lower latency for most operations * the ability for graceful degradation when the network or cloud is down. You don't want the checkouts to stop working just because the cloud is down. If literally every action requires a round trip to the cloud, then edge computing isn't for you. That's not the case for a lot of functionality, however.
- azakai 4y ago
  > What does the edge mean? I was confused by that too. I've seen "edge" mean certain things in the web space (Cloudflare workers, Fastly compute, etc.), but in this article it apparently means "IoT" (Internet of Things, tiny devices with internet connections).
- cpitman 4y ago
  It means varying levels of "not a central data center". For anyone wanting to do edge, you first need to understand what it means to them. Sometimes the edge is distributed sites that still have a rack of hardware, say in factories or distribution centers. Sometimes it is one or two computers at all of your stores, but still "general purpose" hardware. And sometimes it is purpose built devices, like Point of Sales Systems or a radio on every train. Etc. Each form factor has different considerations, and each application different design patterns. For example, at a retailer I worked at each store had a message broker, and orders would be sent through that broker up to the central data center. If/when the store was disconnected, it could continue to queue up orders until the connection was restored. As opposed to housing everything in the data center, and needing to fall back to a paper process if the internet goes down.
- factorialboy 4y ago
  > What does the edge mean? Server with limited capabilities located near you.
- mesebrec 4y ago
  For this project, I'm specifically referring to low-power devices sitting inside of the user's premises. For example, home automation gateways, factory machines, drones etc. I'm not really talking about CDNs or offerings such as Fastly Compute@edge. Right now, most of these devices have bespoke solutions for deploying and updating software. Switching those devices to Kubernetes allows developers to use the same tools and processes from the cloud.
- Arbortheus 4y ago
  I think in this scenario it just means a localised server. I imagine the problem people are trying to solve is just wanting to manage infrastructure in modern GitOps/Infrastructure-as-code ways. Ultimately people want to deploy all their tech infrastructure in such a way that they can address the "treat X as cattle, not pets" problem. Kubernetes is one way to do this.
mesebrec 4y ago
For clarification, I'm talking about devices such as home automation gateways, planes, and production lines. These devices often have bespoke toolchains for deploying and managing software on them. This is an additional barrier of entry: cloud teams need to learn a new toolchain to move to the edge. Moreover, these toolchains often need changes to how release management works. So naturally, the question arises "why can't we just use the same tools that we use in the cloud?" Right now, Kubernetes isn't the greatest fit for the edge. Distributions such as K3s and KubeEdge try to solve some of the problems with Kubernetes itself, but that's only half of the picture. A large advantage of Kubernetes is the ecosystem of third party operators. So that's what we're trying to address.
Felmioytfr 4y ago
K8s doesn't need to do unlimited things. Why run docker if you can push k8s further down with perhaps less features? K8s could be as lightweight as nomad.
- p_l 4y ago
  There's separated kubernetes api-only server (to run just the control plane) and a virtual kubelet implementation that actually runs systemd services.
  - Felmioytfr 4y ago
    Do you have a name or link?
    - p_l 4y ago
      KCP (api-only server): https://github.com/kcp-dev/kcp https://github.com/kcp-dev/kcp SystemK (virtual kubelet): https://github.com/virtual-kubelet/systemk https://github.com/virtual-kubelet/systemk

yjftsjthsd-h 4y ago

More interesting to me is the line in the first graph labeled "no isolation", which has lower memory still. I would expect operators to be trusted, so I don't mind the security implications running them without isolation, if that's what that really means. Is that a viable way to reduce the resource use even further?