3 ms·
(Excuse me if this is obvious, I have limited time, but this article grabbed my attention; Fascinating). How do you handle writes? It seems like an interrupted
by smarm52 2y ago
(Excuse me if this is obvious, I have limited time, but this article grabbed my attention; Fascinating).
How do you handle writes? It seems like an interrupted write process could corrupt a section of text, which could be difficult to recover from.
Given the example at "breckyunits.com", I don't see hashing information associated with each item.
Are you depending on git to prevent such errors from corrupting individual items? If so, then I would be concerned about gits propensity for data corruption [1, 2, 3, 4].
I wonder if adding some ZFS-like hashing and integrity checks would be helpful. Then, as it's one big file, it seems to act like a TAR archive [5], where you append to the end, but have to scan through the previous content to find what you want. If that's the case, then it may be viable to do copy-on-write [6], where information is never modified, but instead referenced with a key, and later modifications supersede older versions.
(Again apologies if this is redundant, I just had the thought and had to get it down. XD)
[1] https://superuser.com/questions/1253830/does-git-prevent-data-degradation https://superuser.com/questions/1253830/does-git-prevent-dat...
[2] https://superuser.com/questions/1635797/what-if-git-repository-gets-corrupted https://superuser.com/questions/1635797/what-if-git-reposito...
[3] https://stackoverflow.com/questions/tagged/corruption?tab=Frequent https://stackoverflow.com/questions/tagged/corruption?tab=Fr...
[4] https://www.reddit.com/r/git/comments/oq9wph/power_outage_in_the_middle_of_commit_index_file/ https://www.reddit.com/r/git/comments/oq9wph/power_outage_in...
[5] https://en.wikipedia.org/wiki/Tar_(computing) https://en.wikipedia.org/wiki/Tar_(computing)
[6] https://en.wikipedia.org/wiki/Copy-on-write https://en.wikipedia.org/wiki/Copy-on-write