4 ms·

Software should be written under the assumption that any given port can already be in use.

TomSwirly 5y ago
Most software has zero need for this. My company's product contains a web server. We don't bother to check to see if some other program is using port 80 or 443, we just use it. The reason is that we own the machine, we install the software, and we know there is nothing running on these ports; and more, if something is on these ports, we have no way to recover, so terminating abnormally is the right answer.
the_mitsuhiko 5y ago
I'm not aware of any software that is not written that way.
- tsimionescu 5y ago
  Well, apparently Flask or other web dev environments fail to start if their default port is already in use...
  - tristor 5y ago
    That's a misunderstanding of sockets and ports. A port alone is not sufficient for anything, to send traffic across an IP network you need a socket, a socket is tuple of a port and an IP address. When a SERVER "binds" a port, what it's doing is opening a socket for listening against the address specified for binding (generally 0.0.0.0 known as INADDR_ANY). Sockets cannot collide, therefore you cannot bind more than one service running on a host to the same socket, which means if both are using INADDR_ANY, they cannot use the same port. Nothing prevents you from having multiple IPs on a host, each bound by IP to the same port (which would be separate sockets). For CLIENT applications, they aren't just utilizing sockets, they're utilizing connections, and a connection is a tuple of two sockets. You can have many many many connections utilizing the same port, as long as they don't collide exactly with the socket on the other side in their signature. There are various ways to handle this, but a typical one is that servers do a hand-off during connection initialization to a temporary port assigned to that connection. So, Flask (a server), is saying you can't bind it to listen on a socket which is previously bound because it would cause a collision and violate the no-collision rule for sockets. But it doesn't in any way mean that port can't be overloaded by different sockets being bound to the same port.
    - dcow 5y ago
      That’s not the point at all.
    - tsimionescu 5y ago
      Thank you for explaining to me how Berkley sockets work (though you forgot about the protocol identifier, you can absolutely have two sockets listening on 127.0.0.1:8000 as long as one is using TCP port 8000 and the other UDP port 8000). > There are various ways to handle this, but a typical one is that servers do a hand-off during connection initialization to a temporary port assigned to that connection. I don't think this is right. Servers don't normally use additional ports. Instead, a process can actually create multiple TCP (streaming) sockets with the same local address:local port combination, but different remote address:remote port combinations (each TCP socket is uniquely identified by a four-tuple). The point though was that once Flask realizes that it can't bind the address:TCP socket, it shouldn't just give up without even telling users about other options.
      - tristor 5y ago
        > I don't think this is right. Servers don't normally use additional ports. Instead, a process can actually create multiple TCP (streaming) sockets with the same local address:local port combination, but different remote address:remote port combinations (each TCP socket is uniquely identified by a four-tuple). It's not the only way to address this issue, but it is a common pattern for servers to do a TCP handoff to an ephemeral port during connection initialization. One of the reasons that this is done is because it allows you to have /multiple/ connections from the same client to the same server, which is useful for a number of reasons. You're correct that each TCP connection (which is two sockets, a socket is IP + port) four-tuple with a different remote IP or remote port is considered a separate identifiable entity, so in some cases that means the client is responsible for providing an ephemeral port that should be connected back on. It depends on a variety of factors. If neither side does this, then you are limited to a single connection for that protocol for a single client. If you'd like to investigate some common servers that use connection handoff and ephemeral ports look at nginx and haproxy, both do this and can also suffer from ephemeral port exhaustion on heavily loaded servers because of it (there are mitigation strategies as well).
  - masklinn 5y ago
    I fail to see what other behaviour they could have. Picking a random port when you have been told to pick a specific port (even implicitly) makes no sense. What if I start flask telling it to use 8000 and it’s already taken by an other program I’m running (something I do semi-regularly as I run tests concurrently), just pick a random port unasked and notify no one about it?
    - checkyoursudo 5y ago
      > notify no one about it But why this part? Python -m http.server tells me which port it is using or gives me "OSError: [Errno 48] Address already in use". In fact, I just tried python -m http.server $((10000 + RANDOM % 10000)) and it told me what port it started using.
    - tsimionescu 5y ago
      Given how much confusion the current behavior seems to have caused, the best course of action seems to have been a simple: "Listening on port $PORT. Warning: failed to bind localhost:5000. Start process with --port=$X to specify a different port" That would be much more helpful behavior and messaging than simply failing, especially since it seems people don't know that they can control the port, and start looking to disabling OS services rather than changing a port...
      - the_mitsuhiko 5y ago
        It might be something to consider, but there are definitely some negative consequences to silently starting on another port, particularly when using automation and tests. It's not fun to have flask silently start on another port and then an integration test hits a completely unrelated service that happens to occupy that port.
        checkyoursudo 5y ago
        Totally agree, but if you're running tests, presumably you are looking for errors in the first place, right? So it doesn't have to fail silently, does it? The testing tools failing for whatever reason seems important to know as well.
  - the_mitsuhiko 5y ago
    If the suggestion is that Flask should use a random other available port: that could be done, but generally people usually prefer to manually pick a different port themselves. Having a dev server come up on different ports all the time is not a great user experience.
- marcellus23 5y ago
  Are you being intentionally obtuse for some reason?
- dev_tty01 5y ago
  We wouldn't be having this conversation if the web tools were written that way. If the web tool is already running, Airplay will move to another port. The reverse is apparently not true.

Most software has zero need for this. My company's product contains a web server. We don't bother to check to see if some other program is using port 80 or 443, we just use it. The reason is that we own the machine, we install the software, and we know there is nothing running on these ports; and more, if something is on these ports, we have no way to recover, so terminating abnormally is the right answer.

the_mitsuhiko 5y ago

I'm not aware of any software that is not written that way.

tsimionescu 5y ago
Well, apparently Flask or other web dev environments fail to start if their default port is already in use...
- tristor 5y ago
  That's a misunderstanding of sockets and ports. A port alone is not sufficient for anything, to send traffic across an IP network you need a socket, a socket is tuple of a port and an IP address. When a SERVER "binds" a port, what it's doing is opening a socket for listening against the address specified for binding (generally 0.0.0.0 known as INADDR_ANY). Sockets cannot collide, therefore you cannot bind more than one service running on a host to the same socket, which means if both are using INADDR_ANY, they cannot use the same port. Nothing prevents you from having multiple IPs on a host, each bound by IP to the same port (which would be separate sockets). For CLIENT applications, they aren't just utilizing sockets, they're utilizing connections, and a connection is a tuple of two sockets. You can have many many many connections utilizing the same port, as long as they don't collide exactly with the socket on the other side in their signature. There are various ways to handle this, but a typical one is that servers do a hand-off during connection initialization to a temporary port assigned to that connection. So, Flask (a server), is saying you can't bind it to listen on a socket which is previously bound because it would cause a collision and violate the no-collision rule for sockets. But it doesn't in any way mean that port can't be overloaded by different sockets being bound to the same port.
  - dcow 5y ago
    That’s not the point at all.
  - tsimionescu 5y ago
    Thank you for explaining to me how Berkley sockets work (though you forgot about the protocol identifier, you can absolutely have two sockets listening on 127.0.0.1:8000 as long as one is using TCP port 8000 and the other UDP port 8000). > There are various ways to handle this, but a typical one is that servers do a hand-off during connection initialization to a temporary port assigned to that connection. I don't think this is right. Servers don't normally use additional ports. Instead, a process can actually create multiple TCP (streaming) sockets with the same local address:local port combination, but different remote address:remote port combinations (each TCP socket is uniquely identified by a four-tuple). The point though was that once Flask realizes that it can't bind the address:TCP socket, it shouldn't just give up without even telling users about other options.
    - tristor 5y ago
      > I don't think this is right. Servers don't normally use additional ports. Instead, a process can actually create multiple TCP (streaming) sockets with the same local address:local port combination, but different remote address:remote port combinations (each TCP socket is uniquely identified by a four-tuple). It's not the only way to address this issue, but it is a common pattern for servers to do a TCP handoff to an ephemeral port during connection initialization. One of the reasons that this is done is because it allows you to have /multiple/ connections from the same client to the same server, which is useful for a number of reasons. You're correct that each TCP connection (which is two sockets, a socket is IP + port) four-tuple with a different remote IP or remote port is considered a separate identifiable entity, so in some cases that means the client is responsible for providing an ephemeral port that should be connected back on. It depends on a variety of factors. If neither side does this, then you are limited to a single connection for that protocol for a single client. If you'd like to investigate some common servers that use connection handoff and ephemeral ports look at nginx and haproxy, both do this and can also suffer from ephemeral port exhaustion on heavily loaded servers because of it (there are mitigation strategies as well).
- masklinn 5y ago
  I fail to see what other behaviour they could have. Picking a random port when you have been told to pick a specific port (even implicitly) makes no sense. What if I start flask telling it to use 8000 and it’s already taken by an other program I’m running (something I do semi-regularly as I run tests concurrently), just pick a random port unasked and notify no one about it?
  - checkyoursudo 5y ago
    > notify no one about it But why this part? Python -m http.server tells me which port it is using or gives me "OSError: [Errno 48] Address already in use". In fact, I just tried python -m http.server $((10000 + RANDOM % 10000)) and it told me what port it started using.
  - tsimionescu 5y ago
    Given how much confusion the current behavior seems to have caused, the best course of action seems to have been a simple: "Listening on port $PORT. Warning: failed to bind localhost:5000. Start process with --port=$X to specify a different port" That would be much more helpful behavior and messaging than simply failing, especially since it seems people don't know that they can control the port, and start looking to disabling OS services rather than changing a port...
    - the_mitsuhiko 5y ago
      It might be something to consider, but there are definitely some negative consequences to silently starting on another port, particularly when using automation and tests. It's not fun to have flask silently start on another port and then an integration test hits a completely unrelated service that happens to occupy that port.
      - checkyoursudo 5y ago
        Totally agree, but if you're running tests, presumably you are looking for errors in the first place, right? So it doesn't have to fail silently, does it? The testing tools failing for whatever reason seems important to know as well.
- the_mitsuhiko 5y ago
  If the suggestion is that Flask should use a random other available port: that could be done, but generally people usually prefer to manually pick a different port themselves. Having a dev server come up on different ports all the time is not a great user experience.
marcellus23 5y ago
Are you being intentionally obtuse for some reason?
dev_tty01 5y ago
We wouldn't be having this conversation if the web tools were written that way. If the web tool is already running, Airplay will move to another port. The reverse is apparently not true.