Skip to content

Finding duplicates

with find

https://www.maketecheasier.com/remove-duplicate-files-linux/

find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | \
  uniq -w32 --all-repeated=separate | tee /root/dupes

rdfind

https://ostechnix.com/how-to-find-and-delete-duplicate-files-in-linux/

jdupes

https://codeberg.org/jbruchon/jdupes

  • up to 7 times faster than fdupes !

fdupes

Searches the given path for duplicate files. Such files are found by comparing file sizes and MD5 signatures, followed by a byte-by-byte comparison.

  • Slow on large directories

Usage:

sudo apt install fdupes
fdupes  -r .

fslint

  • not included anymore in recent debian/ubuntu repos