4 ms·

By "bare metal" you mean as a regular process in the OS instead of as a docker container? Then it should really make no difference in terms of performance. A p

By "bare metal" you mean as a regular process in the OS instead of as a docker container? Then it should really make no difference in terms of performance. A process in a docker container is also a process in the host OS, but with some network and user permission stuff on top. When I read the title I thought you meant bare metal as with no OS layer, just a go app running in the processor, which I think would be pretty cool to see.

jandrewrogers 5y ago
For any software that is I/O intensive, there is a large difference in performance between a container and an ordinary process, as well as a number of other sharp edges. Containers are only approximately equivalent for CPU intensive workloads.
- zozbot234 5y ago
  How so? Are you running actual containers (i.e. on Linux bare-metal host) as opposed to docker VM's on a foreign OS?
- pests 5y ago
  How is this true? A container _is_ an ordinary process. https://iximiuz.com/en/posts/not-every-container-has-an-operating-system-inside/#container-is-just-a-processes https://iximiuz.com/en/posts/not-every-container-has-an-oper... (we just discussed another article by the same author a few days ago on slimmer containers) "The container process is isolated (namespaces) from the rest of the system and restricted from both the resource consumption (cgroups) and security (capabilities, AppArmor, Seccomp) standpoints. But in the end, this is still a regular process, same as any other process on the host system."
  - jandrewrogers 5y ago
    In a normal process (or VM in many cases) you can take direct control of the physical hardware via standard syscalls and mechanisms, which is idiomatic for a lot of high-performance infrastructure software. These appear to stop working or have unexpected (and suboptimal) behavior if the same code is executed in a container. Note that configurations modifying syscall behavior is not unique to containers. There are other ways, only rarely encountered in the wild, to configure your Linux system to inadvertently break the same required syscall behavior. Linux makes no guarantees about the behaviors of syscalls. Nonetheless, high-performance software is necessarily designed under the assumption of specific behaviors that are almost always true on bare metal and modern virtual machines when configured sensibly. AFAIK there is no way to configure a container to ensure the same behavior, which is the problem. VMs in the early days had many of the same issues. It may be sorted eventually (like with VMs) but for now it causes severe regressions in many codes.
- sliken 5y ago
  I think you are either confusing virtual machines with containers, or using containers where any OS differences are transparently hidden by starting a VPS with a container in it. Containers have no overhead, you use the system kernel to do I/O for you, just like any other process.
  - jandrewrogers 5y ago
    There is a litany of ways in which syscalls behave differently when using containers that degrade performance for high-performance software relative to bare metal (or virtual machines, ironically). Many people have observed and tried to debug these issues. There might be a way to empirically make containers work correctly for this type of software but no one has been able to discover how, and not for lack of trying. No process serious about I/O performance uses the system kernel for I/O. Doing it in user space is integer factor higher throughput, hence why Linux has APIs to make this possible for software that needs it. These work perfectly on bare metal and virtual machines with negligible loss of performance in the case of the latter. That aside, containers are currently (inexplicably) missing other capabilities that have been assumed and idiomatic for high-performance software for decades, such as strict hardware resource control and discovery. You can sort of work around this with gross hacks but that doesn’t scale well. Basically, containers made things that were easy for decades stop working or become complicated, and these things were highly valuable for some types of software so discarding them is not an option.

jandrewrogers 5y ago

For any software that is I/O intensive, there is a large difference in performance between a container and an ordinary process, as well as a number of other sharp edges. Containers are only approximately equivalent for CPU intensive workloads.

zozbot234 5y ago
How so? Are you running actual containers (i.e. on Linux bare-metal host) as opposed to docker VM's on a foreign OS?
pests 5y ago
How is this true? A container _is_ an ordinary process. https://iximiuz.com/en/posts/not-every-container-has-an-operating-system-inside/#container-is-just-a-processes https://iximiuz.com/en/posts/not-every-container-has-an-oper... (we just discussed another article by the same author a few days ago on slimmer containers) "The container process is isolated (namespaces) from the rest of the system and restricted from both the resource consumption (cgroups) and security (capabilities, AppArmor, Seccomp) standpoints. But in the end, this is still a regular process, same as any other process on the host system."
- jandrewrogers 5y ago
  In a normal process (or VM in many cases) you can take direct control of the physical hardware via standard syscalls and mechanisms, which is idiomatic for a lot of high-performance infrastructure software. These appear to stop working or have unexpected (and suboptimal) behavior if the same code is executed in a container. Note that configurations modifying syscall behavior is not unique to containers. There are other ways, only rarely encountered in the wild, to configure your Linux system to inadvertently break the same required syscall behavior. Linux makes no guarantees about the behaviors of syscalls. Nonetheless, high-performance software is necessarily designed under the assumption of specific behaviors that are almost always true on bare metal and modern virtual machines when configured sensibly. AFAIK there is no way to configure a container to ensure the same behavior, which is the problem. VMs in the early days had many of the same issues. It may be sorted eventually (like with VMs) but for now it causes severe regressions in many codes.
sliken 5y ago
I think you are either confusing virtual machines with containers, or using containers where any OS differences are transparently hidden by starting a VPS with a container in it. Containers have no overhead, you use the system kernel to do I/O for you, just like any other process.
- jandrewrogers 5y ago
  There is a litany of ways in which syscalls behave differently when using containers that degrade performance for high-performance software relative to bare metal (or virtual machines, ironically). Many people have observed and tried to debug these issues. There might be a way to empirically make containers work correctly for this type of software but no one has been able to discover how, and not for lack of trying. No process serious about I/O performance uses the system kernel for I/O. Doing it in user space is integer factor higher throughput, hence why Linux has APIs to make this possible for software that needs it. These work perfectly on bare metal and virtual machines with negligible loss of performance in the case of the latter. That aside, containers are currently (inexplicably) missing other capabilities that have been assumed and idiomatic for high-performance software for decades, such as strict hardware resource control and discovery. You can sort of work around this with gross hacks but that doesn’t scale well. Basically, containers made things that were easy for decades stop working or become complicated, and these things were highly valuable for some types of software so discarding them is not an option.