3 ms·

How do you handle deployments of realtime back ends which needs state in memory?

jakewins 2y ago
In general you do it by doing a failover behind some variation of a reverse proxy. If you can start new instances quickly and clients can handle short delays you can do it by just stopping the old deployment and starting the new one, booting off of the snapshotted state from the prior deployment. If you need “instant” you do it by implementing some form of catchup and then fail over. It is a lot easier to do this if you have a dedicated component that ”does” the failover, rather than having the old and new deployments try to solve it bilaterally. Could just be a script ran by a human, or something like a k8s operator if you do this a lot
cess11 2y ago
On BEAM/OTP you can control how state is handled at code updates. Finicky but you can. In most other contexts you'd externalise state to a data store like Redis or RDBMS, and spawn one, kill one or do blue-green in the nebula behind your load balancer constellation.
- mtndew4brkfst 2y ago
  BEAM hot code deploys are a vanity tactic in most business verticals, not worth the time and energy to most - nor are they something you can feasibly implement at a polyglot organization. Rolling deploys behind an off the shelf load-balancer setup is perfectly adequate for an overwhelming number of the businesses who'll adopt Elixir for web dev.
zknill 2y ago
The other commenters have mentioned doing deploys behind a proxy, which is fine, but eventually you're going to have to re-deploy the component that terminates the client connections (i.e. websocket server or SSE, etc). From the client's perspective, there's not a lot of difference between the server dropping the connection (on redeploy) or the connection being dropped for some other transient reason. That is to say, with a decent client side handling of connection state you just incrementally rollout your new servers and each server terminates its connections triggering reconnects from the clients. The hardest part is often maintaining continuity on some stream of events. That is; picking up exactly where you were before the connection dropped. You need some mechanism for the client to report the event it last received, and some way to "rewind" back to that point on the stream.
paulgb 2y ago
I work on an open source project for doing just that: https://plane.dev https://plane.dev It follows the model used by Figma where each active document gets its own process on a cluster of machines, and requests are routed to the document server. The advantage of this over something like pubsub is that you have a single authoritative server that can persist the document, instead of having to coordinate multiple sources of truth. (We also provide a managed version of Plane at https://jamsocket.com https://jamsocket.com)

In general you do it by doing a failover behind some variation of a reverse proxy. If you can start new instances quickly and clients can handle short delays you can do it by just stopping the old deployment and starting the new one, booting off of the snapshotted state from the prior deployment. If you need “instant” you do it by implementing some form of catchup and then fail over. It is a lot easier to do this if you have a dedicated component that ”does” the failover, rather than having the old and new deployments try to solve it bilaterally. Could just be a script ran by a human, or something like a k8s operator if you do this a lot

cess11 2y ago

On BEAM/OTP you can control how state is handled at code updates. Finicky but you can. In most other contexts you'd externalise state to a data store like Redis or RDBMS, and spawn one, kill one or do blue-green in the nebula behind your load balancer constellation.

mtndew4brkfst 2y ago
BEAM hot code deploys are a vanity tactic in most business verticals, not worth the time and energy to most - nor are they something you can feasibly implement at a polyglot organization. Rolling deploys behind an off the shelf load-balancer setup is perfectly adequate for an overwhelming number of the businesses who'll adopt Elixir for web dev.

zknill 2y ago

The other commenters have mentioned doing deploys behind a proxy, which is fine, but eventually you're going to have to re-deploy the component that terminates the client connections (i.e. websocket server or SSE, etc). From the client's perspective, there's not a lot of difference between the server dropping the connection (on redeploy) or the connection being dropped for some other transient reason. That is to say, with a decent client side handling of connection state you just incrementally rollout your new servers and each server terminates its connections triggering reconnects from the clients. The hardest part is often maintaining continuity on some stream of events. That is; picking up exactly where you were before the connection dropped. You need some mechanism for the client to report the event it last received, and some way to "rewind" back to that point on the stream.

paulgb 2y ago

I work on an open source project for doing just that: https://plane.dev https://plane.dev It follows the model used by Figma where each active document gets its own process on a cluster of machines, and requests are routed to the document server. The advantage of this over something like pubsub is that you have a single authoritative server that can persist the document, instead of having to coordinate multiple sources of truth. (We also provide a managed version of Plane at https://jamsocket.com https://jamsocket.com)