3 ms·
> We will soon be sending some time series metrics over NATS to use internally for scheduling. For what purpose? > Re. handling failure There are several ope
by hayst4ck 4y ago
> We will soon be sending some time series metrics over NATS to use internally for scheduling.
For what purpose?
> Re. handling failure
There are several operations that should be near seamless and very well thought out/handled reasonably including for the pieces of Plane itself.
Push new code
Roll back code
Push Canary code
shutdown -r now
add machine
remove machine
cluster wide restart
And for persistent data:
replace master
add replica
backup/restore backup
It seems like just about any product is going to have to implement some version of those things so it seems like there should be very well thought out story for each of them under the various conditions that prevent standard architectures.
A single point of failure is an extremely convenient architecture, but it is also a brittle and pain causing architecture that will resist scaling and the clean-ness of operations is probably the best window to assess that.
As far as architecture itself goes, why choose to use DNS rather than header based information/cookies? Why let the client choose the backend rather than hiding that as an infrastructure side implementation detail?