3 ms·

Without the virtual file system, does this work by hashing the files? That would make it very slow to use on larger files. If I was to tag my movie collection,

Without the virtual file system, does this work by hashing the files? That would make it very slow to use on larger files. If I was to tag my movie collection, would TMSU have to load every byte of every movie in order to show the file tags?

mplewis 12y ago
You may want to check out the repair command. It seems to show that TMSU keeps track of file locations on disk. If you move a file, it can find it if it hasn't been modified. If you move and modify a file, it can't find it and will mark it missing. https://github.com/oniony/TMSU/blob/master/src/tmsu/cli/repair.go https://github.com/oniony/TMSU/blob/master/src/tmsu/cli/repa...
- ars 12y ago
  I wonder why he doesn't use EA's where available (or does he?)
  - albertzeyer 12y ago
    What is EA?
    - Someone 12y ago
      Probably Extended Attributes (http://en.m.wikipedia.org/wiki/Extended_file_attributes http://en.m.wikipedia.org/wiki/Extended_file_attributes) They stick better with files, as long as your tools know about them and handle them fine. Weakness is that they can (will) get lost in tools that do not. There are many of them: zip-unzip, tar-untar, http requests, ftp, etc.
      - albertzeyer 12y ago
        Ah. Also, depending on the FS, they are probably too limited, e.g. in size (only 4kb for ext2/3/4).
      - dingaling 12y ago
        > Weakness is that they can (will) get lost in tools that do not. Including, frustratingly, Linux's NFS implementation. A WONTFIX apparently.
- joosters 12y ago
  Ah, I see, thanks. The FAQ also explains some of the strengths and weaknesses of this approach: https://github.com/oniony/TMSU/wiki/FAQ https://github.com/oniony/TMSU/wiki/FAQ
rgro 12y ago
It seems that this also hashes the files to produce a fingerprint. Check out the fingerprinter [1] for the raw details on how it's done, but basically, if the file is below the "sparseFingerprintThreshold" of 5 MB, the whole file is hashed. Any file above is "sparsely hashed" where the first 512 KB, middle 512 KB, and end 512 KB are hashed and combined to produce the fingerprint for that file. It shouldn't produce any performance concerns on large files, so it should be safe to use for your movie collection. [1] https://github.com/oniony/TMSU/blob/master/src/tmsu/common/fingerprint/fingerprinter.go https://github.com/oniony/TMSU/blob/master/src/tmsu/common/f...

mplewis 12y ago

You may want to check out the repair command. It seems to show that TMSU keeps track of file locations on disk. If you move a file, it can find it if it hasn't been modified. If you move and modify a file, it can't find it and will mark it missing. https://github.com/oniony/TMSU/blob/master/src/tmsu/cli/repair.go https://github.com/oniony/TMSU/blob/master/src/tmsu/cli/repa...

ars 12y ago
I wonder why he doesn't use EA's where available (or does he?)
- albertzeyer 12y ago
  What is EA?
  - Someone 12y ago
    Probably Extended Attributes (http://en.m.wikipedia.org/wiki/Extended_file_attributes http://en.m.wikipedia.org/wiki/Extended_file_attributes) They stick better with files, as long as your tools know about them and handle them fine. Weakness is that they can (will) get lost in tools that do not. There are many of them: zip-unzip, tar-untar, http requests, ftp, etc.
    - albertzeyer 12y ago
      Ah. Also, depending on the FS, they are probably too limited, e.g. in size (only 4kb for ext2/3/4).
    - dingaling 12y ago
      > Weakness is that they can (will) get lost in tools that do not. Including, frustratingly, Linux's NFS implementation. A WONTFIX apparently.
joosters 12y ago
Ah, I see, thanks. The FAQ also explains some of the strengths and weaknesses of this approach: https://github.com/oniony/TMSU/wiki/FAQ https://github.com/oniony/TMSU/wiki/FAQ

rgro 12y ago

It seems that this also hashes the files to produce a fingerprint. Check out the fingerprinter [1] for the raw details on how it's done, but basically, if the file is below the "sparseFingerprintThreshold" of 5 MB, the whole file is hashed. Any file above is "sparsely hashed" where the first 512 KB, middle 512 KB, and end 512 KB are hashed and combined to produce the fingerprint for that file. It shouldn't produce any performance concerns on large files, so it should be safe to use for your movie collection. [1] https://github.com/oniony/TMSU/blob/master/src/tmsu/common/fingerprint/fingerprinter.go https://github.com/oniony/TMSU/blob/master/src/tmsu/common/f...