8 ms·

mDNS, Avahi and Docker non-root containers

72 points by gnanesh 6y ago

L3viathan 6y ago
> If it is IPv6, it's nearly impossible to remember the address. Why should that be the case? If you have 1000 nodes, you could organize your network such that node one gets prefix::00:01, node two gets prefix::00:02, node 138 gets prefix::01:38, etc. There is so much space in IPv6 that you can group things more logically than was ever doable in the IPv4 world.
giancarlostoro 6y ago
I'm a big fan of Zeroconf / MDNS / Avahi / Bonjour (it has many names due to the differing packages that can be used for it across platforms). It's a great way to make any type of service discoverable, whether it's a web service, or what have you. Having had used something like Eureka from Netflix I think mdns is way simpler, especially on Linux, if you can install Avahi all you need to do is write an XML file to a directory and you're broadcasting a service immediately.
- amaccuish 6y ago
  Yes I just wish microsoft hadn't complicated it with LLMNR. All sorts of things use mDNS now (chromecast, spotify, printers, thermostats). And finally windows will start to support mDNS so hopefully we can finally settle on one good autodiscovery protocol and get rid of the horror that is SSDP.
  - giancarlostoro 6y ago
    I believe Microsoft finally added mdns to Windows 10, you might have to install it as a component yourself though. It's not yet available under Win32 though by the sound of it.
    - ComputerGuru 6y ago
      Here’s some documentation: https://docs.microsoft.com/en-us/uwp/api/Windows.Networking.ServiceDiscovery.Dnssd?redirectedfrom=MSDN&view=winrt-19041 https://docs.microsoft.com/en-us/uwp/api/Windows.Networking.... Lots of new core APIs added in Windows 10 aren’t available via Win32 even if they have nothing to do with packages, unfortunately. It’s extremely convoluted but you can use the Windows.Sdk, Contracts, or CsWinRT packages to interop with the WinRT APIs from a C++, C#, or now even rust application but depending on which APIs you consume you may need to use a Windows Application Project appx/msix wrapper to package your code and associate it with a package ID unlocking the APIs for use. (I have a hello world project demonstrating a WPF .NET Core project hosting a custom control consisting of a single text block that displays your current location via the Geolocator API and it requires four different individually user-created projects in the Visual Studio solution and a minimum half-a-dozen NuGet dependencies just to make it happen. It’s a terrible, terrible mess.)
      - giancarlostoro 6y ago
        It seems like they want to consolidate all those APIs with Project Reunion https://github.com/microsoft/ProjectReunion https://github.com/microsoft/ProjectReunion So I'm hoping this improved things in the future.
        ComputerGuru 6y ago
        lol, I forgot to mention: this is with Project Reunion. WinUI 3, CsWinRt, etc.
    - goont 6y ago
      There is an API exposed in Windns.h but it doesn't work very well. It's awkward to use and doesn't pick up on services being added and removed. See `DnsService*`, etc in https://docs.microsoft.com/en-us/windows/win32/api/windns/ https://docs.microsoft.com/en-us/windows/win32/api/windns/.
      - Fredej 6y ago
        I believe that's deprecated. The right thing to do - though very well hidden - is to use device enumeration: https://docs.microsoft.com/en-us/windows/uwp/devices-sensors/enumerate-devices-over-a-network https://docs.microsoft.com/en-us/windows/uwp/devices-sensors... This uses mDNS under the hood and works pretty well now. The basics is that you create a device watcher and set it up to look for devices of a specific kind. In this case, you're looking for network-based devices with a specific service type. It's however somewhat of a pain to get running as there's a lot of details you need to know / figure out before you can get running.
        goont 6y ago
        Have you shipped anything using that API? I'm just a bit weary of wasting time on another half-baked solution. And are you sure the Windns.h API is deprecated? I could have sworn it only showed up last year.
        Fredej 6y ago
        I'm in the process of doing so. We had some issues with it initially but it seemed that it got better since a couple of updates ago. I vastly prefer it to having to ship mdnsresponder alongside my application though. I'm basing my "deprecated" comment on this: https://docs.microsoft.com/en-us/uwp/api/Windows.Networking.ServiceDiscovery.Dnssd?redirectedfrom=MSDN&view=winrt-19041 https://docs.microsoft.com/en-us/uwp/api/Windows.Networking.... However that looks to be different from the windns.h stuff. To be honest, I wasn't aware of windns.h and just assumed it was the above link you were talking about. It's probably worth taking a look at :)
        6y ago
- ken 6y ago
  Zeroconf is fantastic for discovering services, but hardly anyone supports it. Apple removed it from Safari, 3 years ago [1]. It's been an open feature request in Chromium for 11 years [2], and in Firefox for 18 years [3]. The only major database I know which supports it is Postgres (it's disabled by default) [4]. On MySQL, it's an open work item, untouched for 13 years [5]. Last I checked, no major web framework broadcasts its service using Zeroconf. Zeroconf service broadcast is so easy to implement (really) and such a tremendous help to users that I don't understand why developers aren't falling over themselves to implement it. [1]: https://apple.stackexchange.com/questions/299700 https://apple.stackexchange.com/questions/299700 [2]: https://bugs.chromium.org/p/chromium/issues/detail?id=13573 https://bugs.chromium.org/p/chromium/issues/detail?id=13573 [3]: https://bugzilla.mozilla.org/show_bug.cgi?id=173804 https://bugzilla.mozilla.org/show_bug.cgi?id=173804 [4]: https://www.postgresql.org/docs/9.1/runtime-config-connection.html https://www.postgresql.org/docs/9.1/runtime-config-connectio... [5]: https://dev.mysql.com/worklog/?sc=id&sd=ASC&pg=3 https://dev.mysql.com/worklog/?sc=id&sd=ASC&pg=3
  - apearson 6y ago
    The browser support you listed is for listing out the available sites. The browsers still support mdns for name resolution. Zeroconf is used in Google Cast and Airplay for service discovery.
    - ken 6y ago
      As Apple's Bonjour page points out, addressing and naming is only 2/3 of it. If that's all you support, then you still depend on users knowing the name, and typing it with no mistakes -- not exactly "zero configuration". 1980's AppleTalk was better than that. How do browsers 'support mDNS for name resolution'? Isn't that a feature of the OS? I can "ping NAS.local" (without a DNS server) just as easily as I can go to "http://NAS.local" http://NAS.local", and ping.c doesn't appear to have any mDNS code.
      - apearson 6y ago
        Right there still a need for apps and browsers to surface mDNS services but the parent comment made it seem like the browsers can't use mDNS at all. You're are correct, mDNS name resolution is an OS feature and to the browser/program/app it makes no difference.
- arminiusreturns 6y ago
  I'm the opposite. I strongly dislike these services as I feel they are not needed in many situations and open up the vulnerability potential, especially if your system is ever on a network not controlled by you (laptop at cafe, etc). I turn them all off/disable/uninstall them.
  - ken 6y ago
    If you're on an untrusted network, run a firewall. An open network port isn't much safer just because you're not announcing it. People looking to attack your local web service at Starbucks probably know how to portscan.
    - arminiusreturns 6y ago
      The point is that instead of an open port for a service you close all the ports you dont need open, and then firewall what you do need. I'm saying stuff like avahi isn't needed in my case so I turn it off and of course thats not the only measure so I run a firewall amongst other things...
pottertheotter 6y ago
This is timely! I installed Wireshark last night for fun and have been looking at traffic on my network. There's quite a bit from mDNS, which is all new to me.
- voltagex_ 6y ago
  I can highly recommend taking a look at the traffic going across your network as well. With an OpenWRT or Ubiquiti router, you may even be able to mirror your WAN port to something Wireshark can see - it's definitely eye-opening.
  - pottertheotter 6y ago
    I've been thinking about getting a Ubiquiti Dream Machine Pro or a similar setup and would love to look at the traffic there. Right now I just have an Orbi router with two satellites.
    - voltagex_ 6y ago
      I'm sure there are cheaper options with a port mirroring function, but one of the entry level Ubiquiti switches plus a controller running in a VM would work, too. Hmm, I should definitely investigate cheaper options.
      - pottertheotter 6y ago
        Yeah, I've thought about other options, including running a lot of things in a VM, because of that. Plus I usually like to tinker and learn. But I have so many other things on my plate right now that I'm not sure I want to take that on too. So having something that's prosumer that I don't have to mess with as much would be nice. I just wish I lived somewhere where I could find more second-hand networking equipment!
  - sigjuice 6y ago
    Or even easier, just plug in a laptop directly without a router.
teddyh 6y ago
http://zeroconf.org/ http://zeroconf.org/ http://www.dns-sd.org/ http://www.dns-sd.org/ http://www.multicastdns.org/ http://www.multicastdns.org/
viraptor 6y ago
I made a project to solve this issue from the other end in my home network: https://gitlab.com/viraptor/docker_mdns https://gitlab.com/viraptor/docker_mdns You can run the app on the host with docker containers and traefik and any service.local domain configured in labels will be announced via mdns. Services themselves don't have to be mdns aware and will be registered/removed as they come up/down. It works without traefik too, but you have to include the port then as well.
ac29 6y ago
"But still, on most of the local systems, we don't have a DNS server and we have to remember the IP addresses of the systems." Perhaps I'm misunderstanding this, but why would your systems not have access to a local DNS server? My routers respond just fine to DNS queries for "hostname.local"-like requests without using something like mDNS.
- rlpb 6y ago
  This only works if your router arranges this through its DHCP server, and even then only if everything on your network uses DHCP and the DNS server supplied by it. This is not widely the case, and with devices that roam across wifi networks it's likely not even true for your devices. mDNS provides peer-to-peer discovery that does not require any supporting network infrastructure.
  - ac29 6y ago
    There's no reason this requires DHCP, or even for the DNS server to be on the router. You could assign static IPs everywhere and still have the DNS server perform lookups to its hosts file.
    - rlpb 6y ago
      Yes, of course. I took it as a given that any alternative solution would have to be automatic and so manual configuration of hosts wasn't an option. If you think it is an option, then the answer to your question is obvious: mDNS does it automatically, so you don't have to maintain the mapping and it works for automatic IP address assignments.
badrabbit 6y ago
This is such a pet-peeve for me. A noisy network service turned on by default where most users won't use it,but all users are exposed to the network through it. (Edit: I mean in most Linux Distros not in this example, of course I have no problem with anyone intentionally doing anything to their own system) In my opinion, modern systems shouldn't run services in case something needs them, they should be dynamically loaded and unloaded (isn't this what dbus is for??). If Chrome needs mDNS it can ask for it. If network resources are accessed at login, the login processes can request avahi to be loaded and then unloaded once they're done. What really feels like insult to injury is how systemd was forced on unwilling users because of features like this, yet avahi,cups,etc... Just sit there idle even if you didn't start any application that has a need for them. I think maybe I am seeing it from the wrong perspective?
- rlpb 6y ago
  > A noisy network service turned on by default... It's not installed by default in the author's example; they specifically install it in the Docker image. > In my opinion, modern systems shouldn't run services in case something needs them, they should be dynamically loaded and unloaded (isn't this what dbus is for??). That's exactly how it works in Debian today. > ...how systemd was forced on unwilling users because of features like this... systemd wasn't forced on anyone. If you're an "unwilling user", use a distribution that doesn't use systemd! But systemd does support socket activation, which is the exact feature you're claiming doesn't exist!
  - badrabbit 6y ago
    I meant on distros not on OP's example. > systemd wasn't forced on anyone. If you're an "unwilling user", use a distribution that doesn't use systemd! But systemd does support socket activation, which is the exact feature you're claiming doesn't exist! It was forced on existing users. If someone gives you the choice between moving to a different house or accepting something and staying where you are, if you don't like that thing you just got forced out of your house,if you accept that thing you were forced into accepting it. If someone puts a gun to your head and asks for your money, the fact that you can chose to die does not mean your money was not taken from you by force when you opt to live. I did not claim socket activation does not exist, I even mentiones dbus as a way. I said it was not being used, services are just left running. Even GPG has a service that just idles.

72 points by gnanesh 6y ago

> If it is IPv6, it's nearly impossible to remember the address. Why should that be the case? If you have 1000 nodes, you could organize your network such that node one gets prefix::00:01, node two gets prefix::00:02, node 138 gets prefix::01:38, etc. There is so much space in IPv6 that you can group things more logically than was ever doable in the IPv4 world.

giancarlostoro 6y ago

I'm a big fan of Zeroconf / MDNS / Avahi / Bonjour (it has many names due to the differing packages that can be used for it across platforms). It's a great way to make any type of service discoverable, whether it's a web service, or what have you. Having had used something like Eureka from Netflix I think mdns is way simpler, especially on Linux, if you can install Avahi all you need to do is write an XML file to a directory and you're broadcasting a service immediately.

amaccuish 6y ago
Yes I just wish microsoft hadn't complicated it with LLMNR. All sorts of things use mDNS now (chromecast, spotify, printers, thermostats). And finally windows will start to support mDNS so hopefully we can finally settle on one good autodiscovery protocol and get rid of the horror that is SSDP.
- giancarlostoro 6y ago
  I believe Microsoft finally added mdns to Windows 10, you might have to install it as a component yourself though. It's not yet available under Win32 though by the sound of it.
  - ComputerGuru 6y ago
    Here’s some documentation: https://docs.microsoft.com/en-us/uwp/api/Windows.Networking.ServiceDiscovery.Dnssd?redirectedfrom=MSDN&view=winrt-19041 https://docs.microsoft.com/en-us/uwp/api/Windows.Networking.... Lots of new core APIs added in Windows 10 aren’t available via Win32 even if they have nothing to do with packages, unfortunately. It’s extremely convoluted but you can use the Windows.Sdk, Contracts, or CsWinRT packages to interop with the WinRT APIs from a C++, C#, or now even rust application but depending on which APIs you consume you may need to use a Windows Application Project appx/msix wrapper to package your code and associate it with a package ID unlocking the APIs for use. (I have a hello world project demonstrating a WPF .NET Core project hosting a custom control consisting of a single text block that displays your current location via the Geolocator API and it requires four different individually user-created projects in the Visual Studio solution and a minimum half-a-dozen NuGet dependencies just to make it happen. It’s a terrible, terrible mess.)
    - giancarlostoro 6y ago
      It seems like they want to consolidate all those APIs with Project Reunion https://github.com/microsoft/ProjectReunion https://github.com/microsoft/ProjectReunion So I'm hoping this improved things in the future.
      - ComputerGuru 6y ago
        lol, I forgot to mention: this is with Project Reunion. WinUI 3, CsWinRt, etc.
  - goont 6y ago
    There is an API exposed in Windns.h but it doesn't work very well. It's awkward to use and doesn't pick up on services being added and removed. See `DnsService*`, etc in https://docs.microsoft.com/en-us/windows/win32/api/windns/ https://docs.microsoft.com/en-us/windows/win32/api/windns/.
    - Fredej 6y ago
      I believe that's deprecated. The right thing to do - though very well hidden - is to use device enumeration: https://docs.microsoft.com/en-us/windows/uwp/devices-sensors/enumerate-devices-over-a-network https://docs.microsoft.com/en-us/windows/uwp/devices-sensors... This uses mDNS under the hood and works pretty well now. The basics is that you create a device watcher and set it up to look for devices of a specific kind. In this case, you're looking for network-based devices with a specific service type. It's however somewhat of a pain to get running as there's a lot of details you need to know / figure out before you can get running.
      - goont 6y ago
        Have you shipped anything using that API? I'm just a bit weary of wasting time on another half-baked solution. And are you sure the Windns.h API is deprecated? I could have sworn it only showed up last year.
        Fredej 6y ago
        I'm in the process of doing so. We had some issues with it initially but it seemed that it got better since a couple of updates ago. I vastly prefer it to having to ship mdnsresponder alongside my application though. I'm basing my "deprecated" comment on this: https://docs.microsoft.com/en-us/uwp/api/Windows.Networking.ServiceDiscovery.Dnssd?redirectedfrom=MSDN&view=winrt-19041 https://docs.microsoft.com/en-us/uwp/api/Windows.Networking.... However that looks to be different from the windns.h stuff. To be honest, I wasn't aware of windns.h and just assumed it was the above link you were talking about. It's probably worth taking a look at :)
        6y ago
ken 6y ago
Zeroconf is fantastic for discovering services, but hardly anyone supports it. Apple removed it from Safari, 3 years ago [1]. It's been an open feature request in Chromium for 11 years [2], and in Firefox for 18 years [3]. The only major database I know which supports it is Postgres (it's disabled by default) [4]. On MySQL, it's an open work item, untouched for 13 years [5]. Last I checked, no major web framework broadcasts its service using Zeroconf. Zeroconf service broadcast is so easy to implement (really) and such a tremendous help to users that I don't understand why developers aren't falling over themselves to implement it. [1]: https://apple.stackexchange.com/questions/299700 https://apple.stackexchange.com/questions/299700 [2]: https://bugs.chromium.org/p/chromium/issues/detail?id=13573 https://bugs.chromium.org/p/chromium/issues/detail?id=13573 [3]: https://bugzilla.mozilla.org/show_bug.cgi?id=173804 https://bugzilla.mozilla.org/show_bug.cgi?id=173804 [4]: https://www.postgresql.org/docs/9.1/runtime-config-connection.html https://www.postgresql.org/docs/9.1/runtime-config-connectio... [5]: https://dev.mysql.com/worklog/?sc=id&sd=ASC&pg=3 https://dev.mysql.com/worklog/?sc=id&sd=ASC&pg=3
- apearson 6y ago
  The browser support you listed is for listing out the available sites. The browsers still support mdns for name resolution. Zeroconf is used in Google Cast and Airplay for service discovery.
  - ken 6y ago
    As Apple's Bonjour page points out, addressing and naming is only 2/3 of it. If that's all you support, then you still depend on users knowing the name, and typing it with no mistakes -- not exactly "zero configuration". 1980's AppleTalk was better than that. How do browsers 'support mDNS for name resolution'? Isn't that a feature of the OS? I can "ping NAS.local" (without a DNS server) just as easily as I can go to "http://NAS.local" http://NAS.local", and ping.c doesn't appear to have any mDNS code.
    - apearson 6y ago
      Right there still a need for apps and browsers to surface mDNS services but the parent comment made it seem like the browsers can't use mDNS at all. You're are correct, mDNS name resolution is an OS feature and to the browser/program/app it makes no difference.
arminiusreturns 6y ago
I'm the opposite. I strongly dislike these services as I feel they are not needed in many situations and open up the vulnerability potential, especially if your system is ever on a network not controlled by you (laptop at cafe, etc). I turn them all off/disable/uninstall them.
- ken 6y ago
  If you're on an untrusted network, run a firewall. An open network port isn't much safer just because you're not announcing it. People looking to attack your local web service at Starbucks probably know how to portscan.
  - arminiusreturns 6y ago
    The point is that instead of an open port for a service you close all the ports you dont need open, and then firewall what you do need. I'm saying stuff like avahi isn't needed in my case so I turn it off and of course thats not the only measure so I run a firewall amongst other things...

pottertheotter 6y ago

This is timely! I installed Wireshark last night for fun and have been looking at traffic on my network. There's quite a bit from mDNS, which is all new to me.

voltagex_ 6y ago
I can highly recommend taking a look at the traffic going across your network as well. With an OpenWRT or Ubiquiti router, you may even be able to mirror your WAN port to something Wireshark can see - it's definitely eye-opening.
- pottertheotter 6y ago
  I've been thinking about getting a Ubiquiti Dream Machine Pro or a similar setup and would love to look at the traffic there. Right now I just have an Orbi router with two satellites.
  - voltagex_ 6y ago
    I'm sure there are cheaper options with a port mirroring function, but one of the entry level Ubiquiti switches plus a controller running in a VM would work, too. Hmm, I should definitely investigate cheaper options.
    - pottertheotter 6y ago
      Yeah, I've thought about other options, including running a lot of things in a VM, because of that. Plus I usually like to tinker and learn. But I have so many other things on my plate right now that I'm not sure I want to take that on too. So having something that's prosumer that I don't have to mess with as much would be nice. I just wish I lived somewhere where I could find more second-hand networking equipment!
- sigjuice 6y ago
  Or even easier, just plug in a laptop directly without a router.

teddyh 6y ago

http://zeroconf.org/ http://zeroconf.org/ http://www.dns-sd.org/ http://www.dns-sd.org/ http://www.multicastdns.org/ http://www.multicastdns.org/

viraptor 6y ago

I made a project to solve this issue from the other end in my home network: https://gitlab.com/viraptor/docker_mdns https://gitlab.com/viraptor/docker_mdns You can run the app on the host with docker containers and traefik and any service.local domain configured in labels will be announced via mdns. Services themselves don't have to be mdns aware and will be registered/removed as they come up/down. It works without traefik too, but you have to include the port then as well.

ac29 6y ago

"But still, on most of the local systems, we don't have a DNS server and we have to remember the IP addresses of the systems." Perhaps I'm misunderstanding this, but why would your systems not have access to a local DNS server? My routers respond just fine to DNS queries for "hostname.local"-like requests without using something like mDNS.

rlpb 6y ago
This only works if your router arranges this through its DHCP server, and even then only if everything on your network uses DHCP and the DNS server supplied by it. This is not widely the case, and with devices that roam across wifi networks it's likely not even true for your devices. mDNS provides peer-to-peer discovery that does not require any supporting network infrastructure.
- ac29 6y ago
  There's no reason this requires DHCP, or even for the DNS server to be on the router. You could assign static IPs everywhere and still have the DNS server perform lookups to its hosts file.
  - rlpb 6y ago
    Yes, of course. I took it as a given that any alternative solution would have to be automatic and so manual configuration of hosts wasn't an option. If you think it is an option, then the answer to your question is obvious: mDNS does it automatically, so you don't have to maintain the mapping and it works for automatic IP address assignments.

badrabbit 6y ago

This is such a pet-peeve for me. A noisy network service turned on by default where most users won't use it,but all users are exposed to the network through it. (Edit: I mean in most Linux Distros not in this example, of course I have no problem with anyone intentionally doing anything to their own system) In my opinion, modern systems shouldn't run services in case something needs them, they should be dynamically loaded and unloaded (isn't this what dbus is for??). If Chrome needs mDNS it can ask for it. If network resources are accessed at login, the login processes can request avahi to be loaded and then unloaded once they're done. What really feels like insult to injury is how systemd was forced on unwilling users because of features like this, yet avahi,cups,etc... Just sit there idle even if you didn't start any application that has a need for them. I think maybe I am seeing it from the wrong perspective?

rlpb 6y ago
> A noisy network service turned on by default... It's not installed by default in the author's example; they specifically install it in the Docker image. > In my opinion, modern systems shouldn't run services in case something needs them, they should be dynamically loaded and unloaded (isn't this what dbus is for??). That's exactly how it works in Debian today. > ...how systemd was forced on unwilling users because of features like this... systemd wasn't forced on anyone. If you're an "unwilling user", use a distribution that doesn't use systemd! But systemd does support socket activation, which is the exact feature you're claiming doesn't exist!
- badrabbit 6y ago
  I meant on distros not on OP's example. > systemd wasn't forced on anyone. If you're an "unwilling user", use a distribution that doesn't use systemd! But systemd does support socket activation, which is the exact feature you're claiming doesn't exist! It was forced on existing users. If someone gives you the choice between moving to a different house or accepting something and staying where you are, if you don't like that thing you just got forced out of your house,if you accept that thing you were forced into accepting it. If someone puts a gun to your head and asks for your money, the fact that you can chose to die does not mean your money was not taken from you by force when you opt to live. I did not claim socket activation does not exist, I even mentiones dbus as a way. I said it was not being used, services are just left running. Even GPG has a service that just idles.