3 ms·

> The index file stores a list of every file at HEAD, along with the object ID for its blob and some metadata. This list of files is stored as a flat list and G

> The index file stores a list of every file at HEAD, along with the object ID for its blob and some metadata. This list of files is stored as a flat list and Git parses the index into an array. I'm surprised that the index is not hierarchical, like tree objects in Git's object storage. With tree objects (https://git-scm.com/book/en/v2/Git-Internals-Git-Objects#_tree_objects https://git-scm.com/book/en/v2/Git-Internals-Git-Objects#_tr...), each level of the hierarchy is a separate object. So you would only need to load directories that are interesting to you. You could use a single hash compare to determine that two directories are identical without actually recursing into them. In particular, I can't understand why you would need a full list of all files to create a commit. If your commit is known not to touch certain directories, it should be able to simply refer to the existing tree object without loading or expanding it. I guess that's what this sparse-index work is doing. I'm just surprised it didn't already work that way.

arxanas 5y ago
It makes more sense if you think of the index as a structure meant specifically to speed up `git status` operations. (It was originally called the "dircache"! See https://github.com/git/git/commit/5adf317b31729707fad4967c1aef6cdba43d0dd3 https://github.com/git/git/commit/5adf317b31729707fad4967c1a...) We desperately want to reduce the number of file accesses we have to make, so directly using the object database and a tree object (or similar structures) would more than double file accesses. There's performance-related metadata in the index which isn't in tree objects. For example, the modified-time of a given file exists in its index entry, which can be used to avoid reading the file from disk if it seems to be unmodified. If you have to do a disk lookup to decide whether to read a file from disk, then the overhead is potentially as much as the operation itself. There's also semantic metadata, such as which stage the file is in (for merge conflict resolution). It's worth noting that you can turn on the cache tree extension (https://git-scm.com/docs/index-format#_cache_tree https://git-scm.com/docs/index-format#_cache_tree) in order to speed up commit operations. It doesn't replace objects in the index with trees, but it does keep ranges of the index cached, if they're known to correspond to a tree.

arxanas 5y ago

It makes more sense if you think of the index as a structure meant specifically to speed up `git status` operations. (It was originally called the "dircache"! See https://github.com/git/git/commit/5adf317b31729707fad4967c1aef6cdba43d0dd3 https://github.com/git/git/commit/5adf317b31729707fad4967c1a...) We desperately want to reduce the number of file accesses we have to make, so directly using the object database and a tree object (or similar structures) would more than double file accesses. There's performance-related metadata in the index which isn't in tree objects. For example, the modified-time of a given file exists in its index entry, which can be used to avoid reading the file from disk if it seems to be unmodified. If you have to do a disk lookup to decide whether to read a file from disk, then the overhead is potentially as much as the operation itself. There's also semantic metadata, such as which stage the file is in (for merge conflict resolution). It's worth noting that you can turn on the cache tree extension (https://git-scm.com/docs/index-format#_cache_tree https://git-scm.com/docs/index-format#_cache_tree) in order to speed up commit operations. It doesn't replace objects in the index with trees, but it does keep ranges of the index cached, if they're known to correspond to a tree.