4 ms·

I still don't get the benefit of running Go binaries in containers. Totally get it for Rails, Python, etc, where the program needs a consistent set of supportin

I still don't get the benefit of running Go binaries in containers. Totally get it for Rails, Python, etc, where the program needs a consistent set of supporting libraries. But Go doesn't need that. Especially now we can embed whole file systems into the actual binary. I've been looking at going in the other direction and using a Go binary with a unikernel; the machine just runs the binary and nothing else. I haven't got this working to my satisfaction yet - it works but there's still too much infrastructure, and deployment is still "fun". But I think this is way more interesting for Go deployments than using containers.

tail_exchange 3y ago
Containers can give you better isolation between the application and the host, as well as making horizontal scaling easier. It's also a must if you are using a system like Kubernetes. If you are running multiple applications in the same host, you also have control over resource limits.
- jeffbee 3y ago
  Kubernetes requires a platform runtime that answers its requests, but there's no law enforcement agency that will prevent you from using a custom runtime that ignores the very existence of Linux control groups.
  - tail_exchange 3y ago
    Yes, that is true. Though if you are using Google's or Amazon's managed Kubernetes services, I think you need to use Docker.
    - jen20 3y ago
      Certainly EKS is less managed than it appears, I’m quite confident that a node running something implementing the kubelet API convincingly would work. They changed recently (1.23 maybe) from Docker to containerd.
mholt 3y ago
As the author of Caddy, I have often wondered why people run it in containers. The feedback I keep hearing is basically workflow/ecosystem lock-in. Everything else uses containers, so the stuff that doesn't need containers needs them now, too.
- marcus_holmes 3y ago
  Love your work :) Good to know I'm not totally out there on this :)
- wonrax 3y ago
  For my personal site I run Caddy in a Docker container along with other containers using a compose file. By doing it this way, getting things up when moving to a new instance is as simple as running `docker compose up`. Also making changes to the config or upgrading Caddy version on deployments is the same as a other services since they're all containers. So it's easy to add CI/CD and have it re-deploy Caddy whenever the config changes and there's no need for extra GitHub Actions .yaml's. Setup as code like this also documents all the dependencies and I think it might be helpful in the future. Having said that, for serious business, this setup doesn't make sense. It possibly takes more work to operate as container when the gateway runs on a dedicated instance.
- nurple 3y ago
  I find putting http proxies in containers to be a very effective method of building interesting dynamic L7 dataplanes on orchestrators like k8s. Packaging applications (particularly modern static SPAs), with a webserver embedded, is also a very intuitive way of plugging them into the leaves of this topology while abstracting away a lot of the connectivity policy from the app. Of course there's also the well-known correlation between the quality of your k8s deployment and the number of proxies it hosts. /s
- fhuici 3y ago
  We just gave this a go (pun intended) on Unikraft / kraft.cloud , Caddy runs fine on it!
wahern 3y ago
Go programs still make use of, e.g., /etc/resolv.conf and /etc/ssl/certs/ca-certificates.crt. These aren't strictly necessary, but using them makes it easier--more consistent, more transparent--to configure these resource across different languages and frameworks.
- marcus_holmes 3y ago
  I use the semi-official Acme package [0], which handles all that really well. I haven't touched SSL or TLS config for years. I mean, I might be an outlier, but this seems pretty standard for Go deployments these days. [0] https://pkg.go.dev/golang.org/x/crypto/acme https://pkg.go.dev/golang.org/x/crypto/acme
throwaway892238 3y ago
Here's what a container gives you: - Isolation of networking - Isolation of process namespace - Isolation of filesystem namespace - CPU and Memory limit enforcement - A near-universal format for packaging, distributing, and running applications, with metadata, and support for multiple architectures - A simple method for declaring a multi-step build process - Other stuff I don't remember You're certainly welcome to go without them, but over time, everybody ends up needing at least one of the features containers bring. I get the aversion to adding more complexity, but complexity has this annoying habit of being useful.
- marcus_holmes 3y ago
  A VM provides the first 4 anyway - if you're deploying to a cloud instance then having these in the container is redundant. If you're deploying to bare metal then it's possibly useful, but only if you're deploying multiple containers to the same machine. Go doesn't need a format for packaging - it's one file. It's becoming common practice to embed everything else into the binary. (side note: I haven't done this with env files yet, and tend to deploy them separately, but I don't see any reason why we don't do this and produce binaries targetted at the specific deployment environments. I might give it a go). I kinda prefer makefiles for the build stuff, or even just a script. The whole process of creating a Docker instance, pushing source files to it, trigging go build and then pulling back the binary seems redundant; there's no advantage of doing this in a container over doing it on the local machine. And it's a lot faster on the local machine. Talking to people, it appears to be as mholt said: everyone just does everything in containers so apparently we do this too.
  - dilyevsky 3y ago
    Are you suggesting to setup a separate VM for each process that may only require like 0.25 cpu? Another thing you can’t do with VMs is oversubscribe (at least not with cloud ones)
    - lmm 3y ago
      You can't have both oversubscription and isolation, almost by definition. If you want isolation, VMs are great. If you want oversubscription, OSes are still better than container runtimes at managing competing processes on the same host.
cedws 3y ago
A lot of companies are Kubernetes based now so containers are the default delivery mechanism for all software.
deleted 3y ago
[deleted]
fhuici 3y ago
Hi, one of the maintainers of the Unikraft LF OSS project here (www.unikraf.org), clearly in agreement :) . We regularly run Go workloads, and even use Dockerfiles to build Go and other projects that we then turn into minimal unikernels. We also have a closed beta/free (unikernel) cloud platform if people want to try, sign up is at kraft.cloud .

tail_exchange 3y ago

Containers can give you better isolation between the application and the host, as well as making horizontal scaling easier. It's also a must if you are using a system like Kubernetes. If you are running multiple applications in the same host, you also have control over resource limits.

jeffbee 3y ago
Kubernetes requires a platform runtime that answers its requests, but there's no law enforcement agency that will prevent you from using a custom runtime that ignores the very existence of Linux control groups.
- tail_exchange 3y ago
  Yes, that is true. Though if you are using Google's or Amazon's managed Kubernetes services, I think you need to use Docker.
  - jen20 3y ago
    Certainly EKS is less managed than it appears, I’m quite confident that a node running something implementing the kubelet API convincingly would work. They changed recently (1.23 maybe) from Docker to containerd.

mholt 3y ago

As the author of Caddy, I have often wondered why people run it in containers. The feedback I keep hearing is basically workflow/ecosystem lock-in. Everything else uses containers, so the stuff that doesn't need containers needs them now, too.

marcus_holmes 3y ago
Love your work :) Good to know I'm not totally out there on this :)
wonrax 3y ago
For my personal site I run Caddy in a Docker container along with other containers using a compose file. By doing it this way, getting things up when moving to a new instance is as simple as running `docker compose up`. Also making changes to the config or upgrading Caddy version on deployments is the same as a other services since they're all containers. So it's easy to add CI/CD and have it re-deploy Caddy whenever the config changes and there's no need for extra GitHub Actions .yaml's. Setup as code like this also documents all the dependencies and I think it might be helpful in the future. Having said that, for serious business, this setup doesn't make sense. It possibly takes more work to operate as container when the gateway runs on a dedicated instance.
nurple 3y ago
I find putting http proxies in containers to be a very effective method of building interesting dynamic L7 dataplanes on orchestrators like k8s. Packaging applications (particularly modern static SPAs), with a webserver embedded, is also a very intuitive way of plugging them into the leaves of this topology while abstracting away a lot of the connectivity policy from the app. Of course there's also the well-known correlation between the quality of your k8s deployment and the number of proxies it hosts. /s
fhuici 3y ago
We just gave this a go (pun intended) on Unikraft / kraft.cloud , Caddy runs fine on it!

wahern 3y ago

Go programs still make use of, e.g., /etc/resolv.conf and /etc/ssl/certs/ca-certificates.crt. These aren't strictly necessary, but using them makes it easier--more consistent, more transparent--to configure these resource across different languages and frameworks.

marcus_holmes 3y ago
I use the semi-official Acme package [0], which handles all that really well. I haven't touched SSL or TLS config for years. I mean, I might be an outlier, but this seems pretty standard for Go deployments these days. [0] https://pkg.go.dev/golang.org/x/crypto/acme https://pkg.go.dev/golang.org/x/crypto/acme

throwaway892238 3y ago

Here's what a container gives you: - Isolation of networking - Isolation of process namespace - Isolation of filesystem namespace - CPU and Memory limit enforcement - A near-universal format for packaging, distributing, and running applications, with metadata, and support for multiple architectures - A simple method for declaring a multi-step build process - Other stuff I don't remember You're certainly welcome to go without them, but over time, everybody ends up needing at least one of the features containers bring. I get the aversion to adding more complexity, but complexity has this annoying habit of being useful.

marcus_holmes 3y ago
A VM provides the first 4 anyway - if you're deploying to a cloud instance then having these in the container is redundant. If you're deploying to bare metal then it's possibly useful, but only if you're deploying multiple containers to the same machine. Go doesn't need a format for packaging - it's one file. It's becoming common practice to embed everything else into the binary. (side note: I haven't done this with env files yet, and tend to deploy them separately, but I don't see any reason why we don't do this and produce binaries targetted at the specific deployment environments. I might give it a go). I kinda prefer makefiles for the build stuff, or even just a script. The whole process of creating a Docker instance, pushing source files to it, trigging go build and then pulling back the binary seems redundant; there's no advantage of doing this in a container over doing it on the local machine. And it's a lot faster on the local machine. Talking to people, it appears to be as mholt said: everyone just does everything in containers so apparently we do this too.
- dilyevsky 3y ago
  Are you suggesting to setup a separate VM for each process that may only require like 0.25 cpu? Another thing you can’t do with VMs is oversubscribe (at least not with cloud ones)
  - lmm 3y ago
    You can't have both oversubscription and isolation, almost by definition. If you want isolation, VMs are great. If you want oversubscription, OSes are still better than container runtimes at managing competing processes on the same host.

cedws 3y ago

A lot of companies are Kubernetes based now so containers are the default delivery mechanism for all software.

deleted 3y ago

[deleted]

fhuici 3y ago

Hi, one of the maintainers of the Unikraft LF OSS project here (www.unikraf.org), clearly in agreement :) . We regularly run Go workloads, and even use Dockerfiles to build Go and other projects that we then turn into minimal unikernels. We also have a closed beta/free (unikernel) cloud platform if people want to try, sign up is at kraft.cloud .