SHA checksum for entire folder

[kb113355]

Hello,
is this possible to create one sha1 checksum for folder with subfolders and files? I have more than 1000 files and I don't want list of 1000 files with checksum.

[Dalai]

Sure. Set the cursor on a directory of your choice (or select any number of directories) and select menu Files > Create Checksum Files. Enter the path of the checksum file (if you want to change it) and select the checksum type. Confirm the dialog to start the checksum generation.

[kb113355]

Thank you.
I did that and the result was file with separate checksum for every file("Create separate checksumfiles for each directory" is unchecked). I want have one checksum for entire directory.

[Dalai]

Make sure the option "Create separate checksum files for each file" is also unchecked.

[white]

2Dalai
He wants one single hash for the folder and all of its contents. Similar to creating a tar file and creating one hash for the tar file.

[Dalai]

Yes, the request can be read that way. However, that's just not possible, neither with TC nor with any other tool. A directory doesn't have a single checksum, but the files within a directory do, i.e. one checksum per file in the directory. Example listing of such a checksum file:

Code: Select all

999556ead55d84d46ab132ea58ee74b9 *history.txt
14cbe39e6484f97bdeaed64e4fa9e047 *readme.htm
50614479110f92b4f3fa4fcd21d6081c *res\default.css

What would a single checksum per directory even represent? Just the directory listing? The metadata? If so, can the checksum stay the same when the directory tree is transferred to a different location and the directory timestamps, NTFS ADS and similar data change?

[white]

If he makes sure the checksum file is always sorted the same way, I guess he could create a checksum file for the checksum file

[beb]

Yes, the request can be read that way. However, that's just not possible, neither with TC nor with any other tool. A directory doesn't have a single checksum...

A directory doesn't have a single checksum, but the directory content does.
Therefore, it is absolutely possible to compute the checksums for the whole directory(ies) [content] in one way or another.
I was giving an example of that over here:
viewtopic.php?p=446869#p446869

Regarding, kb113355's specifics here, that script can be modified as follows:

- for the predefined set of folders:

Code: Select all

$question = @(
# put here each path where the folders in question reside
"$env:Temp\probe\hashFolder\subfolder1",
"$env:Temp\probe\hashFolder\subfolder2",
"$env:Temp\probe\hashFolder\subfolder3",
"$env:Temp\probe\hashFolder\subfolder4",
"$env:Temp\probe\hashFolder\subfolder5"
)

foreach ($folder in $question) {
$proxy = Get-ChildItem $folder -Recurse | Get-FileHash -Algorithm SHA1| select -ExpandProperty Hash | Out-String
$hash = (Get-FileHash -Algorithm SHA1 -InputStream ([IO.MemoryStream]::new([char[]]$proxy))).Hash
"SHA1: {0} {1}" -f $hash.ToLower(),$folder|Write-Host -f Cyan
}
pause

- for the subfolders within the predefined root folder:

Code: Select all

# put here the root path where the subfolders in question reside
$question = "$env:Temp\probe\hashFolder"

$root = Get-ChildItem $question -recurse -directory

foreach ($folder in $root) {
$proxy = Get-ChildItem $folder -Recurse | Get-FileHash -Algorithm SHA1| select -ExpandProperty Hash | Out-String
$hash = (Get-FileHash -Algorithm SHA1 -InputStream ([IO.MemoryStream]::new([char[]]$proxy))).Hash
"SHA1: {0} {1}" -f $hash.ToLower(),$folder|Write-Host -f DarkCyan
}
pause

- for the subfolders within the current directory (ones that have 'unwelcome' in their names will be excluded):

Code: Select all

# for each subfolder in the current directory (except ones which include 'unwelcome' in their names):
Get-ChildItem -directory|Where {$_.Name -notlike "*unwelcome*"}|foreach{
$proxy = Get-ChildItem $_ -Recurse | Get-FileHash -Algorithm SHA1| select -ExpandProperty Hash | Out-String
$hash = (Get-FileHash -Algorithm SHA1 -InputStream ([IO.MemoryStream]::new([char[]]$proxy))).Hash
"SHA1: {0} {1}" -f $hash.ToLower(),$_|Write-Host -f Green
}
pause

Here's the example result of the script.
Image: https://i.imgur.com/UeUlVQA.png

As we can see in the example, a pair of two folders (subfolder1, subfolder2), and a group of three folders (subfolder3, subfolder4, subfolder5) have identical SHA1 (hence, in the first approach -- the identical digital content, which in case of sha1 is still more likely than not, in everyday life).

Of course, the script can be wrapped in the TotalCommander interface (custom command, button, menu entry, whatever a user prefers).

Notes:

The PowerShell script here utilizes a "master hash" approach (calculates hashes of all files within a directory and uses them to create the "master hash" -- "hash of hashes").
And it can be pretty reasonably sufficient for some use cases.

If not only the data content itself but location does matter as well, a user can add [relative]paths\file.names into the calculation to decide if the structures are the same. Dates? Add them there too.

Master hash of files hashes = implies the identical data set.
Master hash of files hashes and of [relative]paths\file.names hashes = implies the identical data set and structure
Master hash of files hashes and of [relative]paths\file.names hashes and of their dates/times hashes = implies something even more identical if needed.

Get-FileHash accepts the following cryptographic hash algorithms: MD5, SHA1, SHA256 (default), SHA384, SHA512.

[kb113355]

Thank you for help. The powershell script is very good.

[beb]

Thank you for help. The powershell script is very good.

You are welcome.
Thank you for the feedback.