3 ms·
Question: would a data lake require backup ? Would you be keeping backups of the lake itself, or would you: - assume processes that dump into the lake also d
by rsync 11y ago
Question: would a data lake require backup ? Would you be keeping backups of the lake itself, or would you:
- assume processes that dump into the lake also dump elsewhere, for backup purposes
- assume that some other process pulls from the lake (and possibly orders or sanitizes the data) and then backs it up
?
- rtkwe 11y agoThe safest way is to backup the data lake. To rely on external backups is really risky because you're counting on every source to have a proper working backup and a way for you to access it if you need to rebuild the lake. On top of that the time for every team to get their backup to you and to load it back into the lake is another factor. Really counting on other people/teams to have backups of your data is asking for pain.