Add BLAKE2 to checksum methods

[lelik007]

2ghisler(Author)
Yes, I disabled Hyper-Threading on purpose, I wanted to check if we're able to forcefully enable multi-threading with CrcBlake3BlockSize=1024 in wincmd.ini, so yes, we're able to do this.

And as for me I'd like when I have more time to play more with the buffer size.

[ghisler(Author)]

If you want to test without the disk cache, you can use this trick:
1. Go to Configuration - Option - Copy/Delete
2. Set the copy mode to "Also use big file copy mode"
3. Start copying the file you want to test with F5
4. Immediately cancel copying
This mode opens the file in mode "no buffering", which has the side effect of removing the file from the disk cache.
After doing this, the Blake3 function will read the file from disk, which is considerably slower than reading it from memory.

[lelik007]

2ghisler(Author)
I understood, and actually I forgot to mention that with i-2600K HT TC uses multi-threading by default.
But I meant BLAKE3 buffer, which is 1024 Megabytes in this case, but it's more for the researching purposes than for practical,
I think 1024 Megabytes is OK.

[white]

2ghisler(Author)
I did some testing now:

Code: Select all

Blake3 hash creation with TC 11.55rc7 64-bit

Hardware:
	CPU i7-8700
	32GB memory
	NVMe SSD

Tested with 7 applications open, 24,5GB memory available

Testfile: 
	4GB Windows 10 iso file 

Timings (no cache/cache):
	default (CrcBlake3BlockSize=1024): 5s / 1s
	CrcBlake3BlockSize=0:              2s / 2s
	CrcBlake3BlockSize=1:              6s / 3s
	CrcBlake3BlockSize=2:              5s / 3s
	CrcBlake3BlockSize=2048            error reading the file

Testfile: 
	12GB file (containing 3 copies of previous test file)

Timings (no cache/cache):
	default (CrcBlake3BlockSize=1024): 14s / 3s
	CrcBlake3BlockSize=0:               8s / 6s
	CrcBlake3BlockSize=1:              17s / 9s
	CrcBlake3BlockSize=2:              20s / 6s
	CrcBlake3BlockSize=2048:           error reading the file

[ghisler(Author)]

Interesting results - so the multi-threaded function is always a lot faster when cached, but your SSD is somehow slowing it all down, as if it has a high latency when switching between different blocks compared to serial reading. On my own system, the multi-threaded function is about 2-3 times faster even uncached on NVMe, and has the exact same speed on SATA SSDs.

In conclusion, someone who uses this function a lot will have to do their own benchmarks because every system is different.

[lelik007]

2white

CrcBlake3BlockSize=2048

2048 is the limit. 2047 is the maximum value you can use.
If you have some time and patience you could test how the BLAKE3 buffer size plays it's role, like:
CrcBlake3BlockSize=256
CrcBlake3BlockSize=512
CrcBlake3BlockSize=1536
I mean will it be faster, slower or just the same.

2ghisler(Author)

In conclusion, someone who uses this function a lot will have to do their own benchmarks because every system is different.

So true, because right now I can't tell apart when multi-threaded BLAKE3 is faster on NVMe or when it's slower.