5 ms·

How useful is dynamic upstream which is new to me?

mysterydip 4y ago
Sounds like it would allow automatic handling of either failed backend nodes or adding new capacity over time without restarting the proxy, but I could be reading it wrong.
- mholt 4y ago
  Basically this, yeah. You can add or take down backends without having to keep web server config in sync and reload it. (Though config reloads are pretty light anyways.)
  - paulgb 4y ago
    Does it have implications for long-lived connections like SSE and websockets? It seems like restarts would have been disruptive to those but now might not be?
    - mholt 4y ago
      Possibly; the long-lived connections eventually have to end when reloads (note: restarts and reloads are different) happen, to free up resources -- otherwise they leak. (The grace period is configurable and can even be disabled.)
      - eliaspro 4y ago
        This bit us in a recent project where we have very long ongoing downloads, where even a very high grace period wouldn't really help. I wonder whether it's technically generally feasible (possibly depends on net/http2's capabilities) to handover the connection from the old to the new server instance, comparable to what systemd does in terms of FD passing for socket-activated services.
        francislavoie 4y ago
        The problem is that existing connections are inside of async functions (goroutines) that have references to the old config. AFAIK, there's no way in Go's stdlib to kill those goroutines while keeping the connection and handing it to another fresh goroutine. To be clear, the new config is running at the same time as the old is still tearing down; all new connections get handled by the new config. Listeners are shared and transferred between servers. We're all ears, though, if someone with deep knowledge of Go's internals can point us in the right direction.
mholt 4y ago
If your backends are registered via SRV records (e.g. Consul), this will be very useful for you! It also works similarly for A/AAAA records. This feature is modular, so other methods of retrieving backends can be implemented as well.

Sounds like it would allow automatic handling of either failed backend nodes or adding new capacity over time without restarting the proxy, but I could be reading it wrong.

mholt 4y ago
Basically this, yeah. You can add or take down backends without having to keep web server config in sync and reload it. (Though config reloads are pretty light anyways.)
- paulgb 4y ago
  Does it have implications for long-lived connections like SSE and websockets? It seems like restarts would have been disruptive to those but now might not be?
  - mholt 4y ago
    Possibly; the long-lived connections eventually have to end when reloads (note: restarts and reloads are different) happen, to free up resources -- otherwise they leak. (The grace period is configurable and can even be disabled.)
    - eliaspro 4y ago
      This bit us in a recent project where we have very long ongoing downloads, where even a very high grace period wouldn't really help. I wonder whether it's technically generally feasible (possibly depends on net/http2's capabilities) to handover the connection from the old to the new server instance, comparable to what systemd does in terms of FD passing for socket-activated services.
      - francislavoie 4y ago
        The problem is that existing connections are inside of async functions (goroutines) that have references to the old config. AFAIK, there's no way in Go's stdlib to kill those goroutines while keeping the connection and handing it to another fresh goroutine. To be clear, the new config is running at the same time as the old is still tearing down; all new connections get handled by the new config. Listeners are shared and transferred between servers. We're all ears, though, if someone with deep knowledge of Go's internals can point us in the right direction.

mholt 4y ago

If your backends are registered via SRV records (e.g. Consul), this will be very useful for you! It also works similarly for A/AAAA records. This feature is modular, so other methods of retrieving backends can be implemented as well.