3 ms·

If they deliver their EMR as a container, it no longer matters what host OS it runs on because the container doesn't have to interact with it. Just like RHEL d

If they deliver their EMR as a container, it no longer matters what host OS it runs on because the container doesn't have to interact with it. Just like RHEL doesn't have to care if it runs on AMD or Intel or if it is virtual or physically hosted, or if the RAM is from Samsung or Micron. And if it needs to talk to a database, say, Oracle, it really doesn't care if it's oracle on Oracle Linux or oracle on SunOS, or if it's in the same rack, or even hall. It just wants to talk to oracle, and it probably has a latency and TNS requirement and that's about it from the hosts's perspective. With 'install my package in a userland' you bind to the userland. But with 'run my container' you only bind to a container runtime, which has a very small and very portable interface in comparison. So no, I don't think the host OS would be relevant at all in a containerised case, as long as it can run containers. As for the userland inside the container: that's up to the EMR provider to choose and deliver. Instead of delivering RPMs you'd get OCI images (or tgz images). In a way, it would work the same way an OVA delivery would work, but smaller and more portable. In some other markets this is already done for WMS software for example, albeit using outdated methods like having a Swarm dependency so you're still runtime-locked.

hedora 3y ago
Containers have to be compatible with the host operating system kernel. In my experience, getting containers to run on non-mainstream Linux distributions is a complete crapshoot.
- oneplane 3y ago
  That's why I mentioned kernel and runtime compatibility.
- pxc 3y ago
  What non-mainstream distros did you run into this with? Was it Docker containers or desktop containers like Flatpak? I thought with Docker the only thing you depend on from the host system was the kernel.
  - hedora 3y ago
    Devuan and Synology come to mind. Devuan's kernels lead to segfaults that I didn't successfully diagnose. Synology's are missing some obscure system call that non-obsolete OpenSSL needs. SmartOS (an Illumnos / solaris variant) is also a bit flaky, though was more stable than those two the last time I checked. edit: Also, switching between RHEL and Ubuntu sometimes leads to VDSO mismatches, which can cause severe performance regressions. I've seen that with and without containers.
- TheNewsIsHere 3y ago
  Yeah, there are a lot of misconceptions about how containers work, and partly that’s due (IMO and experience) to the earlier days of containers, before we had things like standard formats and interfaces. It’s also partly complicated by higher level abstractions like Kubernetes, Mesos, Racher, et al. So at the minimum, pushing these workloads to containers with existing codebases would likely just replace the “certified on RHEL X” with “certified for use in RHEL X containers on RHEL X hypervisors, managed by container orchestration platform X version Y” and so on. Epic has been working on an implementation of their client application (Hyperspace) as a web app (Hyperweb), and that’s probably where we’ll see any new abstraction from incumbent players with large install bases for quite a long time. Edit: typo
  - oneplane 3y ago
    As long as you have a complete userland in your image and a compatible kernel and no extra CAP or binds, it really doesn't matter at all what else the host is doing. That's true for most mainstream architectures and distros of the past 5 years (and more, but I can't be bothered to look it up). If you add orchestration tools in to the mix, that's a different story, but even then you can argue that the orchestrator is your API and not the host OS. A bare kubeadm setup vs. EKS vs. AKS vs. GKE all have the same base API, which means as long as you don't extend past this API it doesn't matter where you run. Again, that breaks as soon as you start including hard dependencies on StorageClass configurations for PVCs, CNIs or CSIs. The only real reason a specific host OS matters is the case where someone writes software against the specific userland of that OS. If you use the same syscalls that are available everywhere, the host OS no longer matters as long as it complies. In a way, if you didn't need the host OS userland, you could just build a single static binary. Or if the software really has special needs, build it as a unikernel. As a concept the 'we build the software, you install the OS' deal is critically flawed and a bad point to abstract an interface between a third party vendor and a server they don't supply.

hedora 3y ago

Containers have to be compatible with the host operating system kernel. In my experience, getting containers to run on non-mainstream Linux distributions is a complete crapshoot.

oneplane 3y ago
That's why I mentioned kernel and runtime compatibility.
pxc 3y ago
What non-mainstream distros did you run into this with? Was it Docker containers or desktop containers like Flatpak? I thought with Docker the only thing you depend on from the host system was the kernel.
- hedora 3y ago
  Devuan and Synology come to mind. Devuan's kernels lead to segfaults that I didn't successfully diagnose. Synology's are missing some obscure system call that non-obsolete OpenSSL needs. SmartOS (an Illumnos / solaris variant) is also a bit flaky, though was more stable than those two the last time I checked. edit: Also, switching between RHEL and Ubuntu sometimes leads to VDSO mismatches, which can cause severe performance regressions. I've seen that with and without containers.
TheNewsIsHere 3y ago
Yeah, there are a lot of misconceptions about how containers work, and partly that’s due (IMO and experience) to the earlier days of containers, before we had things like standard formats and interfaces. It’s also partly complicated by higher level abstractions like Kubernetes, Mesos, Racher, et al. So at the minimum, pushing these workloads to containers with existing codebases would likely just replace the “certified on RHEL X” with “certified for use in RHEL X containers on RHEL X hypervisors, managed by container orchestration platform X version Y” and so on. Epic has been working on an implementation of their client application (Hyperspace) as a web app (Hyperweb), and that’s probably where we’ll see any new abstraction from incumbent players with large install bases for quite a long time. Edit: typo
- oneplane 3y ago
  As long as you have a complete userland in your image and a compatible kernel and no extra CAP or binds, it really doesn't matter at all what else the host is doing. That's true for most mainstream architectures and distros of the past 5 years (and more, but I can't be bothered to look it up). If you add orchestration tools in to the mix, that's a different story, but even then you can argue that the orchestrator is your API and not the host OS. A bare kubeadm setup vs. EKS vs. AKS vs. GKE all have the same base API, which means as long as you don't extend past this API it doesn't matter where you run. Again, that breaks as soon as you start including hard dependencies on StorageClass configurations for PVCs, CNIs or CSIs. The only real reason a specific host OS matters is the case where someone writes software against the specific userland of that OS. If you use the same syscalls that are available everywhere, the host OS no longer matters as long as it complies. In a way, if you didn't need the host OS userland, you could just build a single static binary. Or if the software really has special needs, build it as a unikernel. As a concept the 'we build the software, you install the OS' deal is critically flawed and a bad point to abstract an interface between a third party vendor and a server they don't supply.