3 ms·
> When you log into all of your nodes after a network snafu and discover that every single node has attempted to perform a full state transfer, involving the de
by zzzeek 9y ago
> When you log into all of your nodes after a network snafu and discover that every single node has attempted to perform a full state transfer, involving the destruction of the data directory...
I support customers that are using Galera at Red Hat and while we have seen lots of network snafus and situations bringing individual nodes back online, I've never seen all nodes attempt to state transfer each other like that nor have I ever seen a data directory "destroyed". The situations with Galera in 100% of cases involve operators acting too hastily and not understanding what's going on as they do it.
- clon 9y agoBy destroyed I mean the data directory is wiped as a first step towards a full state transfer. By the time we restored stable shell access to the nodes, after a lot of intermittent networking, all 3 nodes had indeed wiped themselves. The cluster was no more. There may have been of course also something to do with SeveralNines CC that we were trialling at the time. Perhaps it was trying to do something hasty as you put it. In any case, a lot of moving parts to understand fully.
- zzzeek 9y agoGaleras SST is going to rsync the files over but I can't imagine how you got an SST to initiate from a wiped data directory that would somehow wipe another one. Would love to see a reproducer for the condition you describe.