Definitely the latter. There are many of other products which implement "keep track of partial file contents to deduplicate", for example casync borg and resti

Definitely the latter. There are many of other products which implement "keep track of partial file contents to deduplicate", for example casync borg and restic. None of them use sqlite for metadata storage, which kinda makes sense - in author's schema, each block uses (fileid, block_index, hash) tuple, which is 2/3 wasted space, as fileids are mostly same (if files are big) and block_index is always increasing. A custom data structure would be both faster and more efficient.