5 ms·

I've been wanting to use Traefik for a long time but there's this security issue[0] that's almost two(!) years old now that's been keeping me from deploying it

I've been wanting to use Traefik for a long time but there's this security issue[0] that's almost two(!) years old now that's been keeping me from deploying it in production. As far as I can tell, there's still no out-of-the-box solution that's not overly complicated and won't come back to haunt me a year or two from now. [0] https://github.com/traefik/traefik/issues/4174 https://github.com/traefik/traefik/issues/4174 [1] https://doc.traefik.io/traefik/providers/docker/#docker-api-access https://doc.traefik.io/traefik/providers/docker/#docker-api-...

emilevauge 6y ago
This security issue is not that simple to manage as you probably know. It's mainly due to the fact that there is now way to have authorization on the the docker API. This is not the case on Kubernetes for example where you have RBAC to prevent this kind of issue. We have described this in detail in our documentation, and you have many solutions/workarounds to address this: https://doc.traefik.io/traefik/providers/docker/#docker-api-access https://doc.traefik.io/traefik/providers/docker/#docker-api-...
- Spivak 6y ago
  Yeah, I'm surprised that this is such a sticking point. There's nothing that anyone who isn't Docker Inc. can do to fix the problem that, by default, Docker is all or nothing. It would be nice if Docker could expose a read-only endpoint but c'est la vie. The only solution I've seen/used that wasn't convoluted or brittle is running a little daemon to just shovel container metadata into Consul and going from there.
- codethief 6y ago
  > This security issue is not that simple to manage as you probably know. I do think it's simple to manage: As I already mentioned elsewhere, it wouldn't be necessary for the network-facing part of Traefik to talk to the Docker API. There could be a second Traefik container (w/o network access) running a binary called, say, traefik-config-generator whose only task it is to talk to the Docker socket and generate a config and write that config to a shared volume. EDIT: Oh, I just realized you're the founder of Traefik! Thank you so much for your work! I would really appreciate your opinion on my suggestion – even if you think it's complete BS. :)
3np 6y ago
You've probably discounted this for some reason already, but why not use something more built for service discovery - e.g. Consul Catalog / k8s / etcd?
- codethief 6y ago
  I was looking for a dead simple solution whose deployment wouldn't require much more than a simple `docker-compose`. :)
kayson 6y ago
This is very easily solved by using a proxy for the docker socket: https://github.com/Tecnativa/docker-socket-proxy https://github.com/Tecnativa/docker-socket-proxy https://github.com/traefik/traefik/issues/4174#issuecomment- https://github.com/traefik/traefik/issues/4174#issuecomment- Create a private network that only connects Traefik and the proxy, and limit Traefik's access to only the GET requests it needs to operate. Now the socket is only exposed to a local container.
- imtringued 6y ago
  This is such an obvious solution. I am seriously wondering why this isn't integrated into docker directly.
- codethief 6y ago
  This just adds another layer of indirection. While it improves security, it is not the same as fixing the issue in the first place and making sure that no network-facing part of the system runs as root.
TheDong 6y ago
That so called "security issue" is silly. You don't have to deploy traefik with docker. If you want traefik to monitor new docker containers to add routes for them, of course traefik needs to talk to the docker api to do so. The docker api has no way to control access such that it's not equivalent to root access. However, there's no real vulnerability. I'm happy to provide you a url hosted by traefik with the docker integration enabled, no docker socket proxy, etc, and if you can manage to actually escalate permissions, I'll give you 500 bucks. But, of course, you can't. That security issue is just a "defense in depth" issue, and it's an issue for docker, not traefik. This would be like saying "traefik uses the linux kernel api to open files, but the linux kernel requires traefik validate what goes into that api or else it could allow file path traversal"... But traefik does validate filepaths and so no one makes that complaint. Similarly, traefik does validate that only safe docker api calls are made and works hard to prevent any sort of remote code execution, so the issue is not a security issue, but a defense in depth proposal that is really a feature request for the docker project.
- imtringued 6y ago
  >The docker api has no way to control access such that it's not equivalent to root access. I've thought this was a mistake many years ago. The fact that the docker daemon is running with root privileges is also something they should have solved a long time ago. Docker is pretty pathetic when it comes to security.
- codethief 6y ago
  > You don't have to deploy traefik with docker. Sure but in my case that was the whole idea. > If you want traefik to monitor new docker containers to add routes for them, of course traefik needs to talk to the docker api to do so. Yes, but it wouldn't be necessary for the network-facing part of Traefik to talk to the Docker API. There could be a second container (w/o network access) whose only task it is to talk to the Docker socket and generate a config and write that config to a shared volume. > However, there's no real vulnerability. In the present situation Traefik (with Docker integration) is effectively running as root. I don't think it's up for debate that this is much worse than just running Traefik as a normal user (outside Docker). Besides, most users expect applications running in Docker containers to be more secure – not less secure – than running them on the bare system. > This would be like saying "traefik uses the linux kernel api to open files, but the linux kernel requires traefik validate what goes into that api or else it could allow file path traversal"... But traefik does validate filepaths and so no one makes that complaint. No. This would be like saying "Traefik has full access to the kernel and the entire OS and the only thing preventing a hacker from exploiting this is Traefik validating incoming network requests." Do you also run your other web servers as root? > Similarly, traefik does validate that only safe docker api calls are made This is completely irrelevant. Once a hacker is inside the Traefik process (i.e. can execute code under Traefik's PID), he can access the Docker socket and therefore the entire system as she/he pleases.
deleted 6y ago
[deleted]

emilevauge 6y ago

This security issue is not that simple to manage as you probably know. It's mainly due to the fact that there is now way to have authorization on the the docker API. This is not the case on Kubernetes for example where you have RBAC to prevent this kind of issue. We have described this in detail in our documentation, and you have many solutions/workarounds to address this: https://doc.traefik.io/traefik/providers/docker/#docker-api-access https://doc.traefik.io/traefik/providers/docker/#docker-api-...

Spivak 6y ago
Yeah, I'm surprised that this is such a sticking point. There's nothing that anyone who isn't Docker Inc. can do to fix the problem that, by default, Docker is all or nothing. It would be nice if Docker could expose a read-only endpoint but c'est la vie. The only solution I've seen/used that wasn't convoluted or brittle is running a little daemon to just shovel container metadata into Consul and going from there.
codethief 6y ago
> This security issue is not that simple to manage as you probably know. I do think it's simple to manage: As I already mentioned elsewhere, it wouldn't be necessary for the network-facing part of Traefik to talk to the Docker API. There could be a second Traefik container (w/o network access) running a binary called, say, traefik-config-generator whose only task it is to talk to the Docker socket and generate a config and write that config to a shared volume. EDIT: Oh, I just realized you're the founder of Traefik! Thank you so much for your work! I would really appreciate your opinion on my suggestion – even if you think it's complete BS. :)

3np 6y ago

You've probably discounted this for some reason already, but why not use something more built for service discovery - e.g. Consul Catalog / k8s / etcd?

codethief 6y ago
I was looking for a dead simple solution whose deployment wouldn't require much more than a simple `docker-compose`. :)

kayson 6y ago

This is very easily solved by using a proxy for the docker socket: https://github.com/Tecnativa/docker-socket-proxy https://github.com/Tecnativa/docker-socket-proxy https://github.com/traefik/traefik/issues/4174#issuecomment- https://github.com/traefik/traefik/issues/4174#issuecomment- Create a private network that only connects Traefik and the proxy, and limit Traefik's access to only the GET requests it needs to operate. Now the socket is only exposed to a local container.

imtringued 6y ago
This is such an obvious solution. I am seriously wondering why this isn't integrated into docker directly.
codethief 6y ago
This just adds another layer of indirection. While it improves security, it is not the same as fixing the issue in the first place and making sure that no network-facing part of the system runs as root.

TheDong 6y ago

That so called "security issue" is silly. You don't have to deploy traefik with docker. If you want traefik to monitor new docker containers to add routes for them, of course traefik needs to talk to the docker api to do so. The docker api has no way to control access such that it's not equivalent to root access. However, there's no real vulnerability. I'm happy to provide you a url hosted by traefik with the docker integration enabled, no docker socket proxy, etc, and if you can manage to actually escalate permissions, I'll give you 500 bucks. But, of course, you can't. That security issue is just a "defense in depth" issue, and it's an issue for docker, not traefik. This would be like saying "traefik uses the linux kernel api to open files, but the linux kernel requires traefik validate what goes into that api or else it could allow file path traversal"... But traefik does validate filepaths and so no one makes that complaint. Similarly, traefik does validate that only safe docker api calls are made and works hard to prevent any sort of remote code execution, so the issue is not a security issue, but a defense in depth proposal that is really a feature request for the docker project.

imtringued 6y ago
>The docker api has no way to control access such that it's not equivalent to root access. I've thought this was a mistake many years ago. The fact that the docker daemon is running with root privileges is also something they should have solved a long time ago. Docker is pretty pathetic when it comes to security.
codethief 6y ago
> You don't have to deploy traefik with docker. Sure but in my case that was the whole idea. > If you want traefik to monitor new docker containers to add routes for them, of course traefik needs to talk to the docker api to do so. Yes, but it wouldn't be necessary for the network-facing part of Traefik to talk to the Docker API. There could be a second container (w/o network access) whose only task it is to talk to the Docker socket and generate a config and write that config to a shared volume. > However, there's no real vulnerability. In the present situation Traefik (with Docker integration) is effectively running as root. I don't think it's up for debate that this is much worse than just running Traefik as a normal user (outside Docker). Besides, most users expect applications running in Docker containers to be more secure – not less secure – than running them on the bare system. > This would be like saying "traefik uses the linux kernel api to open files, but the linux kernel requires traefik validate what goes into that api or else it could allow file path traversal"... But traefik does validate filepaths and so no one makes that complaint. No. This would be like saying "Traefik has full access to the kernel and the entire OS and the only thing preventing a hacker from exploiting this is Traefik validating incoming network requests." Do you also run your other web servers as root? > Similarly, traefik does validate that only safe docker api calls are made This is completely irrelevant. Once a hacker is inside the Traefik process (i.e. can execute code under Traefik's PID), he can access the Docker socket and therefore the entire system as she/he pleases.

deleted 6y ago

[deleted]