5 ms·

This bit me the first time I deployed using docker. Everything but 80/443 blocked by the firewall. "Surely it is safe to run my app server on port 8000 because

This bit me the first time I deployed using docker. Everything but 80/443 blocked by the firewall. "Surely it is safe to run my app server on port 8000 because no one can access that port externally." Docker stepped in to help by making sure people could access that port.

justsomehnguy 3y ago
My favourite part in Docker is what if you have 127.0.0.1 configured as your resolver (eg you are running Unbound) then Docker helpfully configure containers to use 1.1.1.1 instead. Thanks, my ass, and give back my time trying to diagnose [a container with no diagnostic tools, because it's a lightweight container] why it can't connect to someservice.on.my.lan.
- judge2020 3y ago
  Wouldn't 127.0.0.1 be the loopback for the container itself? So you'd need to set the DNS to the gateway IP or dynamically by resolving `host.docker.internal` (assuming `--add-host=host.docker.internal:host-gateway`)
  - justsomehnguy 3y ago
    Yes. > So you'd need to set the DNS to the gateway IP This is the least friction way. Run your own resolver there to resolve names for containers, forward to the host's loopback for everything else. > or dynamically by resolving `host.docker.internal` Resolving. Name. Of the DNS server. Right?
    - judge2020 3y ago
      I mean dynamically as in a startup script that resolves that hostname, then updates the resolver to the IP it returns.
      - justsomehnguy 3y ago
        Ah. This would require adding a script to the image itself, for each image. No go.
      - watermelon0 3y ago
        Doesn't even have to be dynamic, Docker engine knows the IP of the host, so it can just directly configure `/etc/resolv.conf` with it.
        marginalia_nu 3y ago
        Might not bind on the external interface though.
  - fuzzy2 3y ago
    No, that won't work if the resolver is listening on 127.0.0.1, which will most likely be the case. Docker also has special handling for systemd-resolved: if resolv.conf points at 127.0.0.53, it will use the managed resolv.conf located in /run.
- kccqzy 3y ago
  Nobody should ever mess with a system's resolver settings. I've recently had to debug a bug with a container (probably not Docker itself) unable to reach the internet. It turns out the machine is behind a NAT64 and has IPv6 addresses only, yet something in the container helpfully reconfigures the resolver to be 8.8.8.8 which of course is unreachable.
  - Bluecobra 3y ago
    Ubuntu on WSL is really bad about this and you have to end up using chattr to make /etc/resolv.conf immutable to prevent it from fiddling around with it. Really makes you appreciate Slackware/FreeBSD.
  - patrakov 3y ago
    Running anything on NAT64 without 464XLAT is a broken setup.
    - deleted 3y ago
      [deleted]
    - raffraffraff 3y ago
      I have to admit with some embarrassment that I've never deployed an ipv6-only infrastructure, so I've never had to figure this stuff out. 25+ years doing Linux/infra and it's still arcane and avoidable. I don't even know what 464XLAT is, and have to look it up. > 464XLAT allows private IPv4 clients to connect to an IPv4 host over an IPv6 network. It only supports client-server, not peer-to-peer communication or inbound connections. ...A brief description for anyone else who didn't know.
berkle4455 3y ago
Interesting. I guess I've been lucky as I use externally managed stateful and stateless rules (aws security groups + network acls)
teekert 3y ago
This is such scary stuff. I love the ease of docker (compose) but stuff like this makes me miss the old days. No surprises. Now you can deploy a huge stack of stuff but it may work differently from expectations.
- ActorNightly 3y ago
  Its scary stuff because the common theme of software dev/deployment is to follow common patterns without actually understanding what goes on under the hood.
  - deleted 3y ago
    [deleted]
  - nine_zeros 3y ago
    And no one follows what happens under the hood because companies don't value it at all (until something goes wrong, at which point they fire the scapegoat)
michaelcampbell 3y ago
That issue bit me with a dockerized mysql db. I think because I was not familiar with `ufw` at the time that after I set it up I tested to ensure you couldn't get to the db. Surprise enused.
hackyhacky 3y ago
I agree this is surprising behavior. Other comments have placed blame at the feet of Docker, iptables, or Ufw. However the easiest workaround is to tell docker to publish the ports ONLY on 127.0.0.1, making them inaccessible from elsewhere: - ports: 127.0.0.1:8000:8000
- heywoodlh 3y ago
  I have found that another long-term solution is to switch to rootless Docker[0] or Podman for local dev. That way iptables isn't mangled with because you don't elevate to a privileged user to use it. Not being privileged has some downsides, but I think the tradeoffs are worth it. Docker Desktop and Rancher Desktop also is great on Linux for this reason, although, they both require a desktop environment. (Not disagreeing with your point -- yours is a great/quick workaround -- it just seems that many people don't know about rootless Docker) [0] https://docs.docker.com/engine/security/rootless/ https://docs.docker.com/engine/security/rootless/
- jvda 3y ago
  I don‘t get this. What‘s the point of publishing them on some port if you don‘t that to be.. well, public?
  - hackyhacky 3y ago
    Docker lets you publish services for consumption by other services in your application. You don't necessarily want all of those services to be accessible to the whole world. Database, for example.

justsomehnguy 3y ago

My favourite part in Docker is what if you have 127.0.0.1 configured as your resolver (eg you are running Unbound) then Docker helpfully configure containers to use 1.1.1.1 instead. Thanks, my ass, and give back my time trying to diagnose [a container with no diagnostic tools, because it's a lightweight container] why it can't connect to someservice.on.my.lan.

judge2020 3y ago
Wouldn't 127.0.0.1 be the loopback for the container itself? So you'd need to set the DNS to the gateway IP or dynamically by resolving `host.docker.internal` (assuming `--add-host=host.docker.internal:host-gateway`)
- justsomehnguy 3y ago
  Yes. > So you'd need to set the DNS to the gateway IP This is the least friction way. Run your own resolver there to resolve names for containers, forward to the host's loopback for everything else. > or dynamically by resolving `host.docker.internal` Resolving. Name. Of the DNS server. Right?
  - judge2020 3y ago
    I mean dynamically as in a startup script that resolves that hostname, then updates the resolver to the IP it returns.
    - justsomehnguy 3y ago
      Ah. This would require adding a script to the image itself, for each image. No go.
    - watermelon0 3y ago
      Doesn't even have to be dynamic, Docker engine knows the IP of the host, so it can just directly configure `/etc/resolv.conf` with it.
      - marginalia_nu 3y ago
        Might not bind on the external interface though.
- fuzzy2 3y ago
  No, that won't work if the resolver is listening on 127.0.0.1, which will most likely be the case. Docker also has special handling for systemd-resolved: if resolv.conf points at 127.0.0.53, it will use the managed resolv.conf located in /run.
kccqzy 3y ago
Nobody should ever mess with a system's resolver settings. I've recently had to debug a bug with a container (probably not Docker itself) unable to reach the internet. It turns out the machine is behind a NAT64 and has IPv6 addresses only, yet something in the container helpfully reconfigures the resolver to be 8.8.8.8 which of course is unreachable.
- Bluecobra 3y ago
  Ubuntu on WSL is really bad about this and you have to end up using chattr to make /etc/resolv.conf immutable to prevent it from fiddling around with it. Really makes you appreciate Slackware/FreeBSD.
- patrakov 3y ago
  Running anything on NAT64 without 464XLAT is a broken setup.
  - deleted 3y ago
    [deleted]
  - raffraffraff 3y ago
    I have to admit with some embarrassment that I've never deployed an ipv6-only infrastructure, so I've never had to figure this stuff out. 25+ years doing Linux/infra and it's still arcane and avoidable. I don't even know what 464XLAT is, and have to look it up. > 464XLAT allows private IPv4 clients to connect to an IPv4 host over an IPv6 network. It only supports client-server, not peer-to-peer communication or inbound connections. ...A brief description for anyone else who didn't know.

berkle4455 3y ago

Interesting. I guess I've been lucky as I use externally managed stateful and stateless rules (aws security groups + network acls)

teekert 3y ago

This is such scary stuff. I love the ease of docker (compose) but stuff like this makes me miss the old days. No surprises. Now you can deploy a huge stack of stuff but it may work differently from expectations.

ActorNightly 3y ago
Its scary stuff because the common theme of software dev/deployment is to follow common patterns without actually understanding what goes on under the hood.
- deleted 3y ago
  [deleted]
- nine_zeros 3y ago
  And no one follows what happens under the hood because companies don't value it at all (until something goes wrong, at which point they fire the scapegoat)

michaelcampbell 3y ago

That issue bit me with a dockerized mysql db. I think because I was not familiar with `ufw` at the time that after I set it up I tested to ensure you couldn't get to the db. Surprise enused.

hackyhacky 3y ago

I agree this is surprising behavior. Other comments have placed blame at the feet of Docker, iptables, or Ufw. However the easiest workaround is to tell docker to publish the ports ONLY on 127.0.0.1, making them inaccessible from elsewhere: - ports: 127.0.0.1:8000:8000

heywoodlh 3y ago
I have found that another long-term solution is to switch to rootless Docker[0] or Podman for local dev. That way iptables isn't mangled with because you don't elevate to a privileged user to use it. Not being privileged has some downsides, but I think the tradeoffs are worth it. Docker Desktop and Rancher Desktop also is great on Linux for this reason, although, they both require a desktop environment. (Not disagreeing with your point -- yours is a great/quick workaround -- it just seems that many people don't know about rootless Docker) [0] https://docs.docker.com/engine/security/rootless/ https://docs.docker.com/engine/security/rootless/
jvda 3y ago
I don‘t get this. What‘s the point of publishing them on some port if you don‘t that to be.. well, public?
- hackyhacky 3y ago
  Docker lets you publish services for consumption by other services in your application. You don't necessarily want all of those services to be accessible to the whole world. Database, for example.