From 2e218d38854bc5934f3935bc7aa6eab598770bdb Mon Sep 17 00:00:00 2001 From: Francois Fleuret Date: Thu, 8 Apr 2010 09:31:31 +0200 Subject: [PATCH] Cosmetics. --- finddup.1 | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/finddup.1 b/finddup.1 index e262386..c86df1e 100644 --- a/finddup.1 +++ b/finddup.1 @@ -66,20 +66,24 @@ files with same inode are considered as different None known, probably many. Valgrind does not complain though. -The current algorithm is dumb, that is it does not use any hashing of -the file content. I tried md5 on the whole file, which is not -satisfactory because files are often never read entirely hence the md5 -can not be properly computed. I also tried XOR of the first 4, 16 and -256 bytes with rejection as soon as one does not match. Did not help -either. +The current algorithm is dumb, as it does not use any hashing of the +file content. + +Here are the things I tried, which did not help at all: (1) Computing +md5s on the whole files, which is not satisfactory because files are +often never read entirely hence the md5s can not be properly computed, +(2) computing XOR of the first 4, 16 and 256 bytes with rejection as +soon as one does not match, (3) reading parts of the files of +increasing sizes so that rejection could be done with a small fraction +when possible, (4) using mmap instead of open/read. .SH "WISH LIST" The format of the output should definitely be improved. Not clear how. Their could be some fancy option to link two instances of the command -running on different machines to reduce network disk accesses. Again, -this may not help much, for the reason given above. +running on different machines to reduce network disk accesses. This +may not help much though. .SH "EXAMPLES" -- 2.20.1