Out of curiosity I tried the --no-mmap switch and got very strange results:
After reboot:
1st run with b3sum v155 and v180:
~58 s
2nd and n runs:
~93s
CPU: ~10%
Basically no MEM increase
First I thought it's because of thermal throttling, but the SSD kept at 50-55°C.
And caching should display the opposite results.
No idea what's going on there.
Add BLAKE2 to checksum methods
Moderators: Hacker, petermad, Stefan2, white
Re: Add BLAKE2 to checksum methods
2ZoSTeR
1-st run is like usual single-threaded BLAKE3, 2-nd... this I can't explain.
I curious what if you define --num-threads 2 and --num-threads 4 - will it be any difference, because you define x2 threads?
1-st run is like usual single-threaded BLAKE3, 2-nd... this I can't explain.
I curious what if you define --num-threads 2 and --num-threads 4 - will it be any difference, because you define x2 threads?
Re: Add BLAKE2 to checksum methods
b3sum_v180.exe --num-threads 2
reboot
1: 135,22 s
2: 135,63 s
3: 138,09 s
b3sum_v180.exe --num-threads 4
reboot
1: 88,58 s
2: 92,47 s
3: 92,54 s
The SSD "Active Time" goes up on the 2nd run, but the transfer speed goes down...
Even if the b3sum.exe is not optimal, I'm wondering what's happening, hardware or Windows?
No improvement after a few minutes pause or Defender real-time protection off, just after a reboot.
Two runs with TC are exactly the same, performance counter wise.
reboot
1: 135,22 s
2: 135,63 s
3: 138,09 s
b3sum_v180.exe --num-threads 4
reboot
1: 88,58 s
2: 92,47 s
3: 92,54 s
The SSD "Active Time" goes up on the 2nd run, but the transfer speed goes down...
Even if the b3sum.exe is not optimal, I'm wondering what's happening, hardware or Windows?
No improvement after a few minutes pause or Defender real-time protection off, just after a reboot.
Two runs with TC are exactly the same, performance counter wise.
Re: Add BLAKE2 to checksum methods
2ZoSTeR
I only ready to give a very non-standard approach.
Download this:
https://github.com/Cyan4973/xxHash/releases/download/v0.8.3/xxhsum_win64_v0_8_3.zip:
and run xxhsum.exe -H2 file
Because it's the maximum possible speed with any type of hashing, this should be 6 GiB/s, at least this is how your NVMe rated.
Maybe it's the drive doesn't perform at its peak?
I only ready to give a very non-standard approach.
Download this:
https://github.com/Cyan4973/xxHash/releases/download/v0.8.3/xxhsum_win64_v0_8_3.zip:
and run xxhsum.exe -H2 file
Because it's the maximum possible speed with any type of hashing, this should be 6 GiB/s, at least this is how your NVMe rated.
Maybe it's the drive doesn't perform at its peak?
Re: Add BLAKE2 to checksum methods
Xxhsum is at least consistent between multiple runs and a reboot (2.1 GB/s).
So this has to be a b3sum.exe issue and maybe we shouldn't spam the thread with this any further.
CrystalDiskMark:
[Read]
SEQ 1MiB (Q= 8, T= 1): 7453.595 MB/s [ 7108.3 IOPS] < 1125.00 us>
SEQ 128KiB (Q= 32, T= 1): 7440.779 MB/s [ 56768.6 IOPS] < 563.11 us>
RND 4KiB (Q= 32, T=16): 5733.378 MB/s [1399750.5 IOPS] < 365.23 us>
RND 4KiB (Q= 1, T= 1): 94.838 MB/s [ 23153.8 IOPS] < 43.10 us>
So this has to be a b3sum.exe issue and maybe we shouldn't spam the thread with this any further.
CrystalDiskMark:
[Read]
SEQ 1MiB (Q= 8, T= 1): 7453.595 MB/s [ 7108.3 IOPS] < 1125.00 us>
SEQ 128KiB (Q= 32, T= 1): 7440.779 MB/s [ 56768.6 IOPS] < 563.11 us>
RND 4KiB (Q= 32, T=16): 5733.378 MB/s [1399750.5 IOPS] < 365.23 us>
RND 4KiB (Q= 1, T= 1): 94.838 MB/s [ 23153.8 IOPS] < 43.10 us>
Re: Add BLAKE2 to checksum methods
2ZoSTeR
Right now I don't know any SW or a working sample of multi-threaded via TBB BLAKE3 С code just to give and compare.
We've already got single-threaded BLAKE3 С code as BLAKEX64.DLL so we'll go along with it until ghisler(Author) figures out how to deal with TBB. Or does he want it at all. Anyway I don't see that newer version performs slower than the older one and it's good.
All I've done is attracted Christian's attention to the possibility for BLAKE3 to become multi-treaded, but it's not ready to use code, as everybody understands now.
We shouldn't anyway. The truth is that I can't tell why your results are so inconsistent.we shouldn't spam
Right now I don't know any SW or a working sample of multi-threaded via TBB BLAKE3 С code just to give and compare.
We've already got single-threaded BLAKE3 С code as BLAKEX64.DLL so we'll go along with it until ghisler(Author) figures out how to deal with TBB. Or does he want it at all. Anyway I don't see that newer version performs slower than the older one and it's good.
All I've done is attracted Christian's attention to the possibility for BLAKE3 to become multi-treaded, but it's not ready to use code, as everybody understands now.