3 ms·
I came to the same conclusion about sharding files from the get go. I had a similar experience where my host wouldn't accept to synchronize a directory with mor
by jobigoud 10y ago
I came to the same conclusion about sharding files from the get go. I had a similar experience where my host wouldn't accept to synchronize a directory with more than 5000 files in it. (Software producing the files is my own note-taking application where each page is stored as an individual file to speed up backup of only the pages that changes).
- vog 10y agoWouldn't 5000 files of note-taking easily fit into RAM? Why bother dealing with thousands of small files? Also, rsync does quite a good job with intransferring multi-MB (or even larger) files efficiently while copying only what's actually changed.
- jobigoud 10y agoI feel more secure by dealing with many individual files. For example in case of a conflict during sync, I can visually diff between the two pages very easily. This happens once in a while. Also for example if more than 5% of files have changed, the sync is stopped and asks for manual confirmation to prevent catastrophies.