3 ms·
I've been looking for something like this (and thinking about developing something if I can't find a satisfactory solution) to use across many different hosts t
by rufugee 11y ago
I've been looking for something like this (and thinking about developing something if I can't find a satisfactory solution) to use across many different hosts to identify duplicate files, etc. I've got media spread across many different linux and os x machines. Can crab handle this?
- Qantourisc 11y agoWell a workaround is mounting and using joins ? Now, be careful the example find duplicate file names (with equal size). Not duplicate files ! Those would require check of the contents of the file. Also there are tools like fdupe, same problem regarding remote hosts though. Some tools use xattr to store hashes. Some might use DB's. (With xattr tools, you just run the tool on the remote host first, then on the local host, if you need to save bandwith.) I however don't have the perfect answer.
- cogs 11y agoThe example does check the contents, by using the SHA-1 of the files
- rufugee 11y agoYeah, ideally what I want is a daemon which hooks into libevent or something similar, and each time a file changes or is created, calculates a checksum and updates other metadata, and then finally provides this information back to a central queryable database.
- cogs 11y agoWe don't have any linux builds at the moment, if there's demand we'll make some. We can scan mounted drives, and Crab has a command line switch to treat names as case sensitive or not. SHA-1 calls are probably not practical across a network to compare file contents, even scans can be a bit slow, but you might do something by comparing file sizes.