3 ms·

Service Fabric needs to die. It is an abomination on software. Why, why, WHY does each SF service need to have some terrible wrapper code that forces it to r

by Jenk 4y ago

Service Fabric needs to die. It is an abomination on software. Why, why, *WHY* does each SF service need to have some terrible wrapper code that forces it to run in, and only in, SF? Oh ya.. vendor lock-in. No thanks. Yes, I'm aware that MS added support for docker containers in SF. That integration is dreadful. They did about as good a job of it as the WCF team did of shoe-horning "RESTFul Services" in. > Windows doesn’t have “distributions”, so there’s less need for containers in that space. I don't follow. What does distributions have to do with need for containers?

jiggawatts 4y ago
> What does distributions have to do with need for containers? A lot (but not everything). It's important to recognise where the "stable API/ABI boundaries" are. In Linux, the stable boundary is the kernel. Userspace is unstable. There are many distributions, with many small and big differences. Different C runtimes, different folder structures, different configs, modules, etc... In Windows, the stable boundary is the Win32, COM, and .NET APIs, which are in user space. The kernel boundary is not stable[1] however, which also matters. Even Linus Torvalds has trouble distributing his hobby dive-computer software to Linux![2] He has no issues with MacOS and Windows, because they have stable APIs and distribution mechanisms. With Linux, the kernel is stable, but essentially nothing else is. If you want to distribute software to everyone, then this is a complex problem with many moving parts that you have to manage yourself. Containers sidestep this by packaging up the distribution along with the software being deployed. This works because the kernel ABI is stable. [1] In the Windows world you don't really need to package up the "Windows distribution" because there is only one: Microsoft Windows. Conversely, Windows containers aren't truly portable because the Windows kernel ABI isn't stable. However Microsoft changed this with Windows 11 and Server 2022, the kernel ABI is now "stable", or at least sufficiently stable for Server Core containers to be portable for 5+ years as-is. [2] He's complaining about "desktop applications", but the exact same rant would apply to server software also. This one talk made me understand containers and why they're so important in the Linux world: https://www.youtube.com/watch?v=Pzl1B7nB9Kc https://www.youtube.com/watch?v=Pzl1B7nB9Kc
- moondev 4y ago
  Even if Windows containers are not as portable, the ability to manage them from the same Kubernetes controlplane that manages Linux containers seems pretty powerful. It also enables you to do it outside of Azure/Service Fabric on other clouds or metal. https://kubernetes.io/docs/concepts/windows/ https://kubernetes.io/docs/concepts/windows/ https://docs.microsoft.com/en-us/azure/aks/windows-faq?tabs=azure-cli https://docs.microsoft.com/en-us/azure/aks/windows-faq?tabs=...
  - jiggawatts 4y ago
    You're trading one convenience for a world of hurt if you use Kubernetes with Windows. Just take a casual glance at the "Windows containers" issues list on GitHub: https://github.com/microsoft/Windows-Containers/issues https://github.com/microsoft/Windows-Containers/issues Some of the very recently closed issues were jaw-dropping, such as totally broken networking in common scenarios. DNS resolution is very different in Windows compared to Linux, making much of the "neatness" of Kubernetes pod-to-pod communication not work. There is no maximum memory limit in Windows kernel "job objects" (equivalent to cgroups), so one memory leak in one pod can kill an entire node or cluster. This is very hard to solve, and I've seen it take out Service Fabric clusters also. Etc, etc...
    - moondev 4y ago
      Thanks for sharing, it all seems terrifying but still very interesting to explore. I was curious how the Windows nodes would even join - turns out they support cloud-init just like Linux machines. https://cloudbase.it/cloudbase-init/ https://cloudbase.it/cloudbase-init/ In theory this means they could be managed then by something like KubeVirt instead of treating them like a worker node. https://kubevirt.io/user-guide/virtual_machines/startup_scripts/#cloud-init https://kubevirt.io/user-guide/virtual_machines/startup_scri... Excited to see this space continue to evolve for sure.
      - jiggawatts 4y ago
        Cloudbase-init is interesting, but Microsoft doesn't support or (apparently) use it. There's no mention of it in their docs: https://www.google.com/search?q=site%3Adocs.microsoft.com+%22cloudbase-init%22 https://www.google.com/search?q=site%3Adocs.microsoft.com+%2... New Windows VMs are generally built using PowerShell DSC or a similar native tool.

Service Fabric needs to die. It is an abomination on software. Why, why, *WHY* does each SF service need to have some terrible wrapper code that forces it to r

by Jenk 4y ago

jiggawatts 4y ago

> What does distributions have to do with need for containers? A lot (but not everything). It's important to recognise where the "stable API/ABI boundaries" are. In Linux, the stable boundary is the kernel. Userspace is unstable. There are many distributions, with many small and big differences. Different C runtimes, different folder structures, different configs, modules, etc... In Windows, the stable boundary is the Win32, COM, and .NET APIs, which are in user space. The kernel boundary is not stable[1] however, which also matters. Even Linus Torvalds has trouble distributing his hobby dive-computer software to Linux![2] He has no issues with MacOS and Windows, because they have stable APIs and distribution mechanisms. With Linux, the kernel is stable, but essentially nothing else is. If you want to distribute software to everyone, then this is a complex problem with many moving parts that you have to manage yourself. Containers sidestep this by packaging up the distribution along with the software being deployed. This works because the kernel ABI is stable. [1] In the Windows world you don't really need to package up the "Windows distribution" because there is only one: Microsoft Windows. Conversely, Windows containers aren't truly portable because the Windows kernel ABI isn't stable. However Microsoft changed this with Windows 11 and Server 2022, the kernel ABI is now "stable", or at least sufficiently stable for Server Core containers to be portable for 5+ years as-is. [2] He's complaining about "desktop applications", but the exact same rant would apply to server software also. This one talk made me understand containers and why they're so important in the Linux world: https://www.youtube.com/watch?v=Pzl1B7nB9Kc https://www.youtube.com/watch?v=Pzl1B7nB9Kc

moondev 4y ago
Even if Windows containers are not as portable, the ability to manage them from the same Kubernetes controlplane that manages Linux containers seems pretty powerful. It also enables you to do it outside of Azure/Service Fabric on other clouds or metal. https://kubernetes.io/docs/concepts/windows/ https://kubernetes.io/docs/concepts/windows/ https://docs.microsoft.com/en-us/azure/aks/windows-faq?tabs=azure-cli https://docs.microsoft.com/en-us/azure/aks/windows-faq?tabs=...
- jiggawatts 4y ago
  You're trading one convenience for a world of hurt if you use Kubernetes with Windows. Just take a casual glance at the "Windows containers" issues list on GitHub: https://github.com/microsoft/Windows-Containers/issues https://github.com/microsoft/Windows-Containers/issues Some of the very recently closed issues were jaw-dropping, such as totally broken networking in common scenarios. DNS resolution is very different in Windows compared to Linux, making much of the "neatness" of Kubernetes pod-to-pod communication not work. There is no maximum memory limit in Windows kernel "job objects" (equivalent to cgroups), so one memory leak in one pod can kill an entire node or cluster. This is very hard to solve, and I've seen it take out Service Fabric clusters also. Etc, etc...
  - moondev 4y ago
    Thanks for sharing, it all seems terrifying but still very interesting to explore. I was curious how the Windows nodes would even join - turns out they support cloud-init just like Linux machines. https://cloudbase.it/cloudbase-init/ https://cloudbase.it/cloudbase-init/ In theory this means they could be managed then by something like KubeVirt instead of treating them like a worker node. https://kubevirt.io/user-guide/virtual_machines/startup_scripts/#cloud-init https://kubevirt.io/user-guide/virtual_machines/startup_scri... Excited to see this space continue to evolve for sure.
    - jiggawatts 4y ago
      Cloudbase-init is interesting, but Microsoft doesn't support or (apparently) use it. There's no mention of it in their docs: https://www.google.com/search?q=site%3Adocs.microsoft.com+%22cloudbase-init%22 https://www.google.com/search?q=site%3Adocs.microsoft.com+%2... New Windows VMs are generally built using PowerShell DSC or a similar native tool.