Verify after copy

[knnknn]

Don't take so personally junior, it clouds your judgement. You have been banging on about your CRC problem from day one - if that's not a fetish then what is?

What you call "fetish" is actually called... a forum thread.

It's not about you, it's about trying to understand why you have the temerity to insist that everyone should have to suffer a massive performance hit just because ONE person had a few problems with his dodgy hardware.

Noone has to suffer just because TC implements an option. Stop blowing a checkbox out of proportion.

Apart from being in a minority of one you haven't come up with any convincing information to shed light on the queries raised thus far, have you?

What has being in the minority have to do with anything? How many % of users need to Encode Files and yet TC offers this function.

Especially since there are already ALTERNATIVE COPIERS exactly for that purpose. Thus there is a market.

That's simply not good enough, you know. The fact remains that you have been brandishing evidence derived from a faulty system. That fact alone would make anyone's eyebrows shoot up in any situation and fail your case immediately.

CRC _IS_ for faulty systems.

[knnknn]

(1) You already have a malfunctioning system.
(2) What makes you think it would be kind enough to hand you a CRC go/no go indication whether or not there was a fault? It could just as easily have kept silent and allowed bad data to be written for the next two months, no? It would be sheer chance to catch it in the act given the intermittent nature of the problem.
(3) As I said before, in such a situation all bets are off and you simply cannot rely on anything you may or may not see. I don't know if you've ever had to diagnose an intermittently failing PSU but its symptoms can range all the way from mysterious memory parity check errors to spontaneous reboots to HDDs and fans briefly failing to spin up - you name it, it can and will happen. Even your Bios can go haywire. A software verify function running on that kind of flaky system cannot sensibly be relied on for protection and would only give a false sense of security. You could never be sure if it was lying to you or or telling you the truth.

Since you participate in Ghisler's beta tests, you're doubtless all too familiar with how even the most innocuous changes to the program can often have completely unforeseen side effects. So irrespective of the coding effort and the default off condition, does it make sense to add even more cruft to the pot when the number of likely users can be numbered on the fingers of one hand, figuratively speaking? There is of course no limit to the number of functions that can be added to TC (the notorious CD burner being one). The question is what overwhelming benefit would that bring apart from software bloat?

Funny read.

I still do not understand where the performance hit should come from.

If enabled, the CRC verification function will obviously be doing its sums with every byte that passes across the I/O interface. Depending on how paranoid a polynomial you wish to employ, that must have an impact on your computer's performance to a greater or lesser degree.

Secondly, each file being accessed falls into the embrace of your all-singing, all-dancing modern antivirus suite. To see the possible impact of AV on even an ordinary copy function, look at the benchmark here. Especially interesting is the hit taken by the system when reading and writing 1000s of small files. Add the two functions together and I'm sure you can predict the likely outcome even if you're not Paul the psychic octopus.

I can already hear the weeping and gnashing of teeth as users who unwittingly enable verification wonder why their PC is crawling on its knees when they use their networks.

What are you talking about?

CRC checks merely double the copy time. Just copy some files with TeraCopy and stop inventing nonsense about "PCs crawling on knees".

[Hacker]

SQUIRE,

What makes you think it would be kind enough to hand you a CRC go/no go indication whether or not there was a fault? It could just as easily have kept silent and allowed bad data to be written for the next two months, no? It would be sheer chance to catch it in the act given the intermittent nature of the problem.

I do not quite understand. Good data is written, good CRC - everything fine. Bad data written, good CRC - improbable. Good data written, bad CRC - suspicious. Bad data written, bad CRC - suspicious. Which of these four possible scenarios you consider a problem, or a detriment compared to the current situation?

A software verify function running on that kind of flaky system cannot sensibly be relied on for protection and would only give a false sense of security. You could never be sure if it was lying to you or or telling you the truth.

Again, I am not getting your point. This is not a function that would tell anyone "your PSU is faulty". This is a function which should raise your suspicion if some data is not copied correctly.
I am still not sure of your point, but it seems to be "even though bad data might be copied, you still might often get a misleading 'everything is OK' CRC check result". I really disagree with that. Hashing functions are designed specifically to prevent that, but I am quite sure you must know that already.

does it make sense to add even more cruft to the pot when the number of likely users can be numbered on the fingers of one hand, figuratively speaking?

Well, that is up to Christian to decide. I for myself do not think I would need such function (at least not often), on the other hand, I though so about descript.ions, too (which were intensively supported only by about three users on the forum) and now I use them quite often.

The question is what overwhelming benefit would that bring apart from software bloat?

Honestly I really would not consider this more bloat than the VerifyZip option I mentioned above.

If enabled, the CRC verification function will obviously be doing its sums with every byte that passes across the I/O interface.

But, why would anyone enable a function he does not wish to use?

unwittingly enable verificatio

Ah, is that the point? Well, they might be tipped off by some dialog saying "CRC check in progress" after each copy operation or so. Anyways, this could be an INI option for instance.

To sum up your position:
- system comes to a crawl when function is (accidentally) enabled
- false sense of security (we still have to reach some common understanding on this one)
- software bloat / unnecessary function / dangerous side effects(?)

Did I get the list about right?

Roman

[SQUIRE]

@ Hacker:

Bad data written, good CRC - improbable.

Please explain why you think that should be so on a PC with an intermittent power problem?

Hashing functions are designed specifically to prevent that...

Indeed. What happens when you compute your hash on a failing machine, bearing in mind that not only is the MOBO unstable but also the HDD's own controller cannot be relied upon for exactly the same reason? Bad data written, good CRC - possible?

Well, that is up to Christian to decide.

Of course. I have to say I would be surprised if he did insert such an unnecessary capability though.

But, why would anyone enable a function he does not wish to use?

Heh, I'm pretty surprised you would say that after all the time you've spent responding to puzzled users asking questions about standard copy/default method/explorer method, the eternal PASV enable/disable problem, copying streams, breadcrumb and network delay controls and countless more hidden switches in wincmd.ini.

Whatever. Plenty of good arguments have already been made by other posters to shoot down this proposal Roman, so let's not reinvent the wheel. The paranoid will never be satisfied whatever security measures you provide. It's enough for me that no manufacturer considers this tenuous argument to have much merit, otherwise you can rest assured that something would have been done to fix it decades ago.

[HolgerK]

<OT>

CRC checks merely double the copy time

This makes me wonder why you are complaining about TC's copy and delete operations speed: Why is Alt*p Sal*m*nder so much faster?

Btw: Was there any indicator from a SMART monitoring tool regarding the errors caused by PSU-failure?
</OT>

It wouldn't be possible with the new default copy method anyway, because it doesn't allow to calculate a checksum or so during copying...

2Hacker
If in consequence an enabled verification automatically would switch to the old copy method, i would vote against such a feature, because of the know problems with some faulty drivers.

Add a fifths scenario:
Bad data written because of verification is turned on.

Regards
Holger

[knnknn]

Plenty of good arguments have already been made by other posters to shoot down this proposal Roman

*Plenty of good arguments* against CRC?

Hmm, I must have forgotten them when I discovered the completely corrupted file (I mentioned above) that was CAUGHT by my CRC checks.

The topic is actually pretty simple:

* Copy errors do happen
* CRC checks can recognize copy errors
* TC is a file copier
* TC should have optional CRC checks

Case closed.

[SQUIRE]

Funny read.

Hilarious, for some.

I DON'T CARE WHAT CAUSED IT. That is not the point. The point is that you MOVE GIGABYTES OF DATA which arrive CORRUPTED...

CRC checks merely double the copy time.

Nice. Now look through the forum and count the number of complaints about slow copying. Enough said?

You have been banging on your CRC validation drum for weeks now and have had plenty of excellent replies as to why that would effectively be overkill. Even Ghisler has politely answered. Your response has always been to wave them away dismissively and keep on shouting, "I want, I want!" I was mildly curious as to why you were encountering problems no one else seemed to have. Now it turns out that you have been using unreliable results from a slowly failing machine to back up your case. I'm sorry knnknn but that simply will not do. You are always entitled to your opinion but you are NOT entitled to your own facts. You need to do much better I'm afraid.

Add the two functions together and I'm sure you can predict the likely outcome even if you're not Paul the psychic octopus.

What are you talking about?

And even more importantly, you need to ask someone to explain who Paul the prophet cephalopod is...he's quite famous nowadays.

[knnknn]

<OT>

CRC checks merely double the copy time

This makes me wonder why you are complaining about TC's copy and delete operations speed: Why is Alt*p Sal*m*nder so much faster?

Btw: Was there any indicator from a SMART monitoring tool regarding the errors caused by PSU-failure?
</OT>

Completely offtopic. TC is much slower than Alt*p Salamader when copying many small files. Has nothing to do with CRC.

And of course, there was no SMART problem. I already wrote about useless SMART. From the many copy errors I encountered NONE caused any SMART message. SMART is useless in this regard.

Moreover EVEN IF you would get an instant SMART notification after a completed move you would would have lost the data anyway.

If in consequence an enabled verification automatically would switch to the old copy method, i would vote against such a feature, because of the know problems with some faulty drivers.

Add a fifths scenario:
Bad data written because of verification is turned on.

I just know that I have to use FastCopy, TeraCopy or the countless other FileCopiers to get the job done. If they can do it, so should TC.

[HolgerK]

Completely offtopic. TC is much slower than Alt*p Salamader when copying many small files. Has nothing to do with CRC.

But CRC checking has something to do with copy speed...

And of course, there was no SMART problem. I already wrote about useless SMART. From the many copy errors I encountered NONE caused any SMART message. SMART is useless in this regard.

This does not answer my question: was there a notification or not or did you even refuse to use SMART anyway?

Moreover EVEN IF you would get an instant SMART notification after a completed move you would would have lost the data anyway.

The same may be true for CRC if caching comes into my mind.

If in consequence an enabled verification automatically would switch to the old copy method, i would vote against such a feature, because of the know problems with some faulty drivers.

I just know that I have to use FastCopy, TeraCopy or the countless other FileCopiers to get the job done. If they can do it, so should TC.

Maybe these tools where not in the (un)lucky position to do such an operation while your hardware decided to fail?
Anyway: Any advanced Editor can do things like syntax highlighting; TC' build in Lister can not, but some plugins can.
Maybe a plugin writer would write a fs-plugin for military level 3/4 secure copying?
Just a thought if Christian has no time to add such a feature for you.

Regards
Holger

[knnknn]

Nice. Now look through the forum and count the number of complaints about slow copying.

Is that what you called *cough* an argument?

You have been banging on your CRC validation drum for weeks now and have had plenty of excellent replies as to why that would effectively be overkill.

They won't become "excellent replies" because you call them that.

And how would an OPTION be overkill?

Even Ghisler has politely answered.

He answered that CRC calculation is impossible with the default copy method. That's all. He could never claim that CRC is impossible per se, BECAUSE OTHER FILE COPIERS ALREADY HAVE CRC checks. Whether they do it with the "default copy method" or not is irrelevant.

I was mildly curious as to why you were encountering problems no one else seemed to have.

You say it correctly: "Noone else SEEEMED to have".

Years ago I also did not SEEM to have copy errors until I found that for months my LAN transfers had been corrupted.

Since then I started to do CRC checks and caught many bad file transfers.

Now it turns out that you have been using unreliable results from a slowly failing machine to back up your case.

That was just one of my examples. I encountered all kinds of copy errors for all kinds of reasons.

[Hacker]

SQUIRE,

Please explain why you think that should be so on a PC with an intermittent power problem?

Well, for that to happen two exactly opposite mistakes would have to happen. For instance in a 1 GB file if a bit is changed on a certain position during copy, then the same bit would have to be changed back during read from target. That a faulty PSU (or faulty RAM, or faulty HDD, or overheating CPU, or anything else) would cause mistakes at the exactly same position at two different specific times twice is highly improbable. It is as if you would read a book to me, exactly with punctuation and quotation marks etc. over a phone and the line would break up at one moment and I would hear "a" instead of "the". Then I would read the book back to you, the line would have to break up at the very same moment I was reading the "a" and you would hear "the", thinking I wrote down an exact copy of your book. Chances of that happening are infinitesimal, but please correct me if I am wrong.

What happens when you compute your hash on a failing machine, bearing in mind that not only is the MOBO unstable but also the HDD's own controller cannot be relied upon for exactly the same reason? Bad data written, good CRC - possible?

Assuming we are talking about creating a CRC from the target data, which we assume is a bad copy of the original data, then we either get correct CRC from bad data, thus not matching the CRC from good data, or we get wrong CRC from bad data, again, not matching the CRC from the good data. The chances of them matching, assuming an ideal hashing functiion, is 1/2^(number of bits in hash) [EDIT: That should have been 2^n, not 1^n of course].

Heh, I'm pretty surprised you would say that after all the time you've spent responding to puzzled users asking questions about standard copy/default method/explorer method, the eternal PASV enable/disable problem, copying streams, breadcrumb and network delay controls and countless more hidden switches in wincmd.ini.

I agree it is a possibility, however I still think that having an INI switch with such dramatic slowdown effects is not something you set and forget and after half a year you suddenly realize, "hey, it's slower than before". But that's just the way I see it.

HolgerK,

an enabled verification automatically would switch to the old copy method

I don't think Christian would implement it that way, but again, that would be up to him.

Roman

[knnknn]

Completely offtopic. TC is much slower than Alt*p Salamader when copying many small files. Has nothing to do with CRC.

But CRC checking has something to do with copy speed...

CRC checks double the copy time because you have to reread the written file.

That Alt*p Salamander is so much faster (with many small files) than TC has something to do with the way how TC constantly displays the file names during the copy operation.

This does not answer my question: was there a notification or not or did you even refuse to use SMART anyway?

There was NO NOTIFICATION whatsoever.

Moreover EVEN IF you would get an instant SMART notification after a completed move you would would have lost the data anyway.

The same may be true for CRC if caching comes into my mind.

This is already a programmer's problem. Has nothing to do with the actual need for a CRC function. And if competitors can solve the problem then so can Mr. Ghisler.

Maybe a plugin writer would write a fs-plugin for military level 3/4 secure copying?
Just a thought if Christian has no time to add such a feature for you

Yes, a plugin solution could do.

[byblo]

Maybe a plugin writer would write a fs-plugin for military level 3/4 secure copying?
Just a thought if Christian has no time to add such a feature for you

Yes, a plugin solution could do.

I'm not sure...

Personally, I'll need the CRC check also on the synchronization tool (when sychronizing like 1000 files on random folders, its hard to keep a trace at what files to CRC check later...)
I don't think sync tool can copy files from a plugin ?


BTW, I had a new case on files copied and corrupted.
While TC said files were copied successfully, I noticed the problem thanks to my usual paranoia by checking (manually...) the files CRC.

Again, I really support this suggestion since it is astonishing that for a (10 years old?) program which the main purpose is to copy/move files, doesn't have such useful/professional feature.

[knnknn]

Well, for that to happen two exactly opposite mistakes would have to happen. For instance in a 1 GB file if a bit is changed on a certain position during copy, then the same bit would have to be changed back during read from target. That a faulty PSU (or faulty RAM, or faulty HDD, or overheating CPU, or anything else) would cause mistakes at the exactly same position at two different specific times twice is highly improbable.

It's actually even less likely. Because you have to assume that the system causes READ errors.

In my experience there are hardly ever read errors because a system with read errors is unstable (crashes, audio glitches etc). Most errors are write errors (HDD) or transfer errors (cables, hubs etc).

And it gets even less likely when you have to assume that the read error happens AT THE SAME BIT and moreover AT THE SAME BIT ONLY (= with no other bits being falsely read although you have a system that causes read errors).

[knnknn]

BTW, I had a new case on files copied and corrupted.
While TC said files were copied successfully, I noticed the problem thanks to my usual paranoia by checking (manually...) the files CRC.

Oh, no! You posted in this thread. Just wait until SQUIRE insults you by "We are not interested in your faulty system" and "You are NOT entitled to your own facts" (Original quote!)