5 ms·

A major gotcha for using file-based UNIX sockets for communicating with containers is that UNIX sockets have far shorter path length limits than anything else.

A major gotcha for using file-based UNIX sockets for communicating with containers is that UNIX sockets have far shorter path length limits than anything else. This bit me when I implemented Nomad's Consul Connect integration and used a UNIX socket to allow Envoy in a container to contact Consul on the host: https://github.com/hashicorp/nomad/issues/6516 https://github.com/hashicorp/nomad/issues/6516

quotemstr 7y ago
It'd be easy enough to fix in the kernel. The challenge would be overcoming the chorus of "no". It's hard to add fixes even for clear and long-standing problems when you have to respond to "do we REALLY need this change?" with an overwhelming amount of evidence.
- duncaen 7y ago
  The size for `sockaddr_storage` is not defined by POSIX, but `sockaddr_un` is defined, and you can't just change the `sun_path` to a pointer, so to increase `sun_path` you would have to increase the `sockaddr_storage` struct size. This comes with other downsides, first is incompatibility with other OSes, most OSes seem to be around 104-109 bytes. Second new problem would be that with any larger value each socket call would have to copy more data, even if they are not unix sockets. If you change it to something that looks sensible like `PATH_MAX` you end up increasing memory requirements for any application that works with sockets.
  - quotemstr 7y ago
    The relevant functions take size parameters. We don't have to limit ourselves to sockaddr_storage.
- Quekid5 7y ago
  ... or: The danger of defining things via C structs. (I'm being a teeny tiny bit facetious. It could theoretically be done reasonably via C structs, but the history of POSIX, etc. is littered with historical accidents that are causing weird and avoidable problems to this day.)
cerberusss 7y ago
Very interesting, I didn't know that. On my macOS system, there are easily a dozen paths in /var that exceed 100 chars. For example becase they have a UUID in them.

quotemstr 7y ago

It'd be easy enough to fix in the kernel. The challenge would be overcoming the chorus of "no". It's hard to add fixes even for clear and long-standing problems when you have to respond to "do we REALLY need this change?" with an overwhelming amount of evidence.

duncaen 7y ago
The size for `sockaddr_storage` is not defined by POSIX, but `sockaddr_un` is defined, and you can't just change the `sun_path` to a pointer, so to increase `sun_path` you would have to increase the `sockaddr_storage` struct size. This comes with other downsides, first is incompatibility with other OSes, most OSes seem to be around 104-109 bytes. Second new problem would be that with any larger value each socket call would have to copy more data, even if they are not unix sockets. If you change it to something that looks sensible like `PATH_MAX` you end up increasing memory requirements for any application that works with sockets.
- quotemstr 7y ago
  The relevant functions take size parameters. We don't have to limit ourselves to sockaddr_storage.
Quekid5 7y ago
... or: The danger of defining things via C structs. (I'm being a teeny tiny bit facetious. It could theoretically be done reasonably via C structs, but the history of POSIX, etc. is littered with historical accidents that are causing weird and avoidable problems to this day.)

cerberusss 7y ago

Very interesting, I didn't know that. On my macOS system, there are easily a dozen paths in /var that exceed 100 chars. For example becase they have a UUID in them.