30 ms·
I use and test assorted duplicate finders regularly. fdupes is the classic (going way way back) but it's really very slow, not worth using anymore. The four I
by jjav 5y ago
I use and test assorted duplicate finders regularly.
fdupes is the classic (going way way back) but it's really very slow, not worth using anymore.
The four I know are worth trying these days (depending on data set, hardware, file arrangement and other factors, any one of these might be fastest for a specific use case) are:
https://github.com/jbruchon/jdupes https://github.com/jbruchon/jdupes
https://github.com/pauldreik/rdfind https://github.com/pauldreik/rdfind
https://github.com/jvirkki/dupd https://github.com/jvirkki/dupd
https://github.com/sahib/rmlint https://github.com/sahib/rmlint
Had not encountered fclones before, will give it a try.