4 ms·

How do the two processes listen to the same port?

loeg 5y ago
There’s an ioctl for this on FreeBSD and Linux — SO_REUSEPORT. You could also just leave the listening socket open when exec’ing the new httpd, or send it with a unix domain socket.
ctrlrsf 5y ago
Using socket option SO_REUSEPORT allows multiple processes to bind to same port.
- VWWHFSfQ 5y ago
  is this what it's actually doing though? It doesn't say the reuseport option to the listen directive is required for this.
- riyadparvez 5y ago
  Is there any restrictions on this option? Eg only children of the same parent process are allowed to bind to same port. Otherwise how does the packet distribution work? And how does the response from that port work?
  - garaetjjte 5y ago
    >So long as the first server sets this option before binding its socket, then any number of other servers can also bind to the same port if they also set the option beforehand. [...] To prevent unwanted processes from hijacking a port that has already been bound by a server using SO_REUSEPORT, all of the servers that later bind to that port must have an effective user ID that matches the effective user ID used to perform the first bind on the socket. https://lwn.net/Articles/542629/ https://lwn.net/Articles/542629/
markbnj 5y ago
This article on how haproxy uses SO_REUSEPORT goes into some more detail: https://www.haproxy.com/blog/truly-seamless-reloads-with-haproxy-no-more-hacks/ https://www.haproxy.com/blog/truly-seamless-reloads-with-hap...
nullify88 5y ago
Once the USR2 signal is received the master process forks, the child process inherits the parents file descriptors including listen(). One process stops accepting connections creating a queue in the kernel. The new process takes over and starts accepting connections. You can follow the trail by searching for ngx_exec_new_binary in the nginx repo.
- krab 5y ago
  Just to add - Nginx normally spawns several worker processes that all process connections to the same port.
  - nullify88 5y ago
    Correct but to clarify, only the master process binds to the ports. The master process creates socketpairs to the workers for interprocess communication. The workers accept connections over the shared socket. https://www.nginx.com/blog/socket-sharding-nginx-release-1-9-1/ https://www.nginx.com/blog/socket-sharding-nginx-release-1-9... Page also has an example of how SO_REUSEPORT effects flow.
    - krab 5y ago
      Oh, thanks! I didn't know that. I supposed it worked by inheriting the listening socket but I didn't check.

loeg 5y ago

There’s an ioctl for this on FreeBSD and Linux — SO_REUSEPORT. You could also just leave the listening socket open when exec’ing the new httpd, or send it with a unix domain socket.

ctrlrsf 5y ago

Using socket option SO_REUSEPORT allows multiple processes to bind to same port.

VWWHFSfQ 5y ago
is this what it's actually doing though? It doesn't say the reuseport option to the listen directive is required for this.
riyadparvez 5y ago
Is there any restrictions on this option? Eg only children of the same parent process are allowed to bind to same port. Otherwise how does the packet distribution work? And how does the response from that port work?
- garaetjjte 5y ago
  >So long as the first server sets this option before binding its socket, then any number of other servers can also bind to the same port if they also set the option beforehand. [...] To prevent unwanted processes from hijacking a port that has already been bound by a server using SO_REUSEPORT, all of the servers that later bind to that port must have an effective user ID that matches the effective user ID used to perform the first bind on the socket. https://lwn.net/Articles/542629/ https://lwn.net/Articles/542629/

markbnj 5y ago

This article on how haproxy uses SO_REUSEPORT goes into some more detail: https://www.haproxy.com/blog/truly-seamless-reloads-with-haproxy-no-more-hacks/ https://www.haproxy.com/blog/truly-seamless-reloads-with-hap...

nullify88 5y ago

Once the USR2 signal is received the master process forks, the child process inherits the parents file descriptors including listen(). One process stops accepting connections creating a queue in the kernel. The new process takes over and starts accepting connections. You can follow the trail by searching for ngx_exec_new_binary in the nginx repo.

krab 5y ago
Just to add - Nginx normally spawns several worker processes that all process connections to the same port.
- nullify88 5y ago
  Correct but to clarify, only the master process binds to the ports. The master process creates socketpairs to the workers for interprocess communication. The workers accept connections over the shared socket. https://www.nginx.com/blog/socket-sharding-nginx-release-1-9-1/ https://www.nginx.com/blog/socket-sharding-nginx-release-1-9... Page also has an example of how SO_REUSEPORT effects flow.
  - krab 5y ago
    Oh, thanks! I didn't know that. I supposed it worked by inheriting the listening socket but I didn't check.