3 ms·

My holy grail implementation would be a "partial clone" that downloads desired files like normal, but creates stubs for selected files that are not stored on th

My holy grail implementation would be a "partial clone" that downloads desired files like normal, but creates stubs for selected files that are not stored on the device but downloaded on-demand upon opening them, like the OneDrive Files On-Demand [1] or Google Drive File Stream. [1]: https://support.microsoft.com/en-us/office/save-disk-space-with-onedrive-files-on-demand-for-windows-10-0e6860d3-d9f3-4971-b321-7092438fb38e https://support.microsoft.com/en-us/office/save-disk-space-w...

minimalist 4y ago
Have you looked into git-annex?
neerajsi 4y ago
Have you seen https://github.com/microsoft/VFSForGit https://github.com/microsoft/VFSForGit? It's used by the Windows team to manage a monorepo containing most Windows source code. Unfortunately that approach was put in maintenance mode since it didn't seem like it would be supportable on macOS.
- dafelst 4y ago
  This has been superceded by Scalar (https://github.com/microsoft/scalar https://github.com/microsoft/scalar) and again merged into Microsoft's fork of git (https://github.com/microsoft/git https://github.com/microsoft/git) It supports neat stuff like partial clone which seems like a pretty big deal.
  - infogulch 4y ago
    Interesting! It seems some of Scalar from late 2021 has already made it into the official git project's contrib dir [0]. It looks like Scalar is mostly an opinionated way to configure git [1] using git partial-clone among other features. Git partial-clone looks almost perfect, except it only downloads and displays files explicitly added to the git sparse-checkout list. I want some "magic" vfs shenanigans that lets me view and browse the full repo exactly as if the full repo where checked out, but when I open a directory or file the contents are downloaded on-demand. [0]: https://github.com/git/git/tree/master/contrib/scalar https://github.com/git/git/tree/master/contrib/scalar [1]: https://github.com/microsoft/git/blob/vfs-2.37.0/Documentation/scalar/philosophy.md https://github.com/microsoft/git/blob/vfs-2.37.0/Documentati...
    - infogulch 4y ago
      It seems win32 has specific support for this kind of tech at a couple different layers, one that's low level filesystem virtualization that's like FUSE I guess [1], and another that's higher level and exposes sync status of files via explorer and other win32 apis [2] which is what I assume OneDrive, Dropbox, Google Drive, etc use. [1]: https://docs.microsoft.com/en-us/windows/win32/projfs/projected-file-system https://docs.microsoft.com/en-us/windows/win32/projfs/projec... [2]: https://docs.microsoft.com/en-us/windows/win32/cfapi/build-a-cloud-file-sync-engine https://docs.microsoft.com/en-us/windows/win32/cfapi/build-a...

minimalist 4y ago

Have you looked into git-annex?

neerajsi 4y ago

Have you seen https://github.com/microsoft/VFSForGit https://github.com/microsoft/VFSForGit? It's used by the Windows team to manage a monorepo containing most Windows source code. Unfortunately that approach was put in maintenance mode since it didn't seem like it would be supportable on macOS.

dafelst 4y ago
This has been superceded by Scalar (https://github.com/microsoft/scalar https://github.com/microsoft/scalar) and again merged into Microsoft's fork of git (https://github.com/microsoft/git https://github.com/microsoft/git) It supports neat stuff like partial clone which seems like a pretty big deal.
- infogulch 4y ago
  Interesting! It seems some of Scalar from late 2021 has already made it into the official git project's contrib dir [0]. It looks like Scalar is mostly an opinionated way to configure git [1] using git partial-clone among other features. Git partial-clone looks almost perfect, except it only downloads and displays files explicitly added to the git sparse-checkout list. I want some "magic" vfs shenanigans that lets me view and browse the full repo exactly as if the full repo where checked out, but when I open a directory or file the contents are downloaded on-demand. [0]: https://github.com/git/git/tree/master/contrib/scalar https://github.com/git/git/tree/master/contrib/scalar [1]: https://github.com/microsoft/git/blob/vfs-2.37.0/Documentation/scalar/philosophy.md https://github.com/microsoft/git/blob/vfs-2.37.0/Documentati...
  - infogulch 4y ago
    It seems win32 has specific support for this kind of tech at a couple different layers, one that's low level filesystem virtualization that's like FUSE I guess [1], and another that's higher level and exposes sync status of files via explorer and other win32 apis [2] which is what I assume OneDrive, Dropbox, Google Drive, etc use. [1]: https://docs.microsoft.com/en-us/windows/win32/projfs/projected-file-system https://docs.microsoft.com/en-us/windows/win32/projfs/projec... [2]: https://docs.microsoft.com/en-us/windows/win32/cfapi/build-a-cloud-file-sync-engine https://docs.microsoft.com/en-us/windows/win32/cfapi/build-a...