3 ms·
That's not a risk unique to Erlang or Mnesia. There's all sorts of reasons nodes may fail, some where data persisted on that node is temporarily unavailable (ne
by toast0 4y ago
That's not a risk unique to Erlang or Mnesia. There's all sorts of reasons nodes may fail, some where data persisted on that node is temporarily unavailable (network, power, cabling, OS/application fault, etc), some where it is permanently unavailable (storage failure, node reassigned, operator error, confiscation, etc).
In my experience, Mnesia works best if you can ensure an orderly sequence of online nodes, but unless you use ram_copies tables, the data is persisted to disc, with journaling, so it's recoverable as long as you have an orderly sequence of nodes in general.
By orderly sequence, I mean that you don't lose access to a node thatis the sole holder of data. On hard provisioned systems, two geoseparated pairs of two nodes should be more than sufficient, but don't get them all storage from the same batch and turn on everything at once. You might want to add a fifth node somewhere if you want to pursue a majority transaction setup. If your nodes are more ephemeral, you probably want three (or more?) per location so you have a better chance of data migration within the location. If you can provision network storage with good persistance forecasts, you could look into storing snapshots (and maybe journals) there, but that's not an out of the box experience.
As much as possible, you want to put nodes that store redundant data in different failure domains; not in the same rack attached to the same switch and the same PDU. Of course, there's connectivity benefits of being close too, so you have to weigh the risks with the benefits.