You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I haven't checked the code yet but judging by the results shown by iostat, czkawka performs hashing on full files having the same size. This can be improved by introducing 2-passes hashing. First pass can read and hash arbitrary block size (64kB, 1MB, etc) starting from beginning of file. The second pass could continue running hashing on full file if previous step determined that files might be the same.
This could speed up duplicate search especially on network attached volumes with large amount of files to compare.
The text was updated successfully, but these errors were encountered:
Hi there,
I haven't checked the code yet but judging by the results shown by iostat, czkawka performs hashing on full files having the same size. This can be improved by introducing 2-passes hashing. First pass can read and hash arbitrary block size (64kB, 1MB, etc) starting from beginning of file. The second pass could continue running hashing on full file if previous step determined that files might be the same.
This could speed up duplicate search especially on network attached volumes with large amount of files to compare.
The text was updated successfully, but these errors were encountered: