2 ms·
Briefly, there are two main hash functions in use in noms. sha-2 is used to compute the hash of individual chunks. This is the classic use of hashing in conten
by aboodman 10y ago
Briefly, there are two main hash functions in use in noms.
sha-2 is used to compute the hash of individual chunks. This is the classic use of hashing in content-addressed systems.
We also use a rolling hash to compute chunk boundaries. We do this in the typical way that tools like bup, camlistore, rsync, and others do for large files.
But our observation was that if you squint your eyes, a merkle tree looks a little like a b-tree. So we use a rolling hash to break up huge lists, maps, and sets into trees where nodes are roughly 4KB. So it's a kind of self-balancing, probabilistic, deterministic b-tree thing.