"Syncronize Dirs" on local folder and remote (FTP)

[vasm]

I have tried to syncronize local and remote folder when option "by content" is checked. And files have different size are marked as "?" on diff listing.
If "by content" checkbox is clear all right ( not eq ).


TC 7.50a has not the same behavior

[ghisler(Author)]

Can you send me a connection log, please?

TC 7.50 didn't support compare by content over ftp at all. TC 7.55 now supports it if the plugin announces (via reply to the FEAT command) that it supports one of these commands:
HASH
XCRC
XMD5
XSHA
MD5
TC then calculates the checksum of the local file, and compares it with the checksum reported by the server.

I guess that your server reports something like this, but then doesn't handle the command when TC tries to get the file checksum.

[vasm]

Sorry, but this "FTP" is by plugin SFTP4TC ( it is not real FTP, over SSH ), witch based on PuTTY (older sync algorithm was same FTP).
I think the plugin hasn't these commands.
However now diff reports empty information about files with same names: if files are identical diff results => "?", files knowingly are not identical (different sizes) diff results => "?".
Maybe leave old algorithm on sync with FSplugins? md5 hashes are different on files have different size.

By log it uses "ls [remote dir]".

[ghisler(Author)]

I see - indeed TC now shows "?" on plugins when the contents cannot be compared. I will consider to change that. Just uncheck the "content" option to avoid it.

[nick7inc]

Good day.

Topic is concern Folder synchronization of local folder with remote FTP folder via XCRC command. I found, that if CRC32 of file is like 0C54282F (starting from zero), it can make false file difference (with question mark) with some ftp servers like SERV-U, which removes leading zeros from CRC sums (here is FTP log):

Code: Select all

xcrc bak.7z
250 C54282F

Please, check others crc algorithms (MD5 and SH1) on this problem too. I have no possibility to check it by myself, my server supports only XCRC command.

Thanks you.

[Sob]

IMHO it's bug in Serv-U, CRC32 values are always written with eight characters, no other software known to me drops leading zeros. On the other hand, it'd be easy for TC to handle it.

[nick7inc]

This is not quite bug. The CRC32 is a HEX number and actually can be written without leading zeros (or even with prefix, like '0x' or '$'). It is better IMHO to handle it as number, not as string.

[Sob]

Ok, "bug" may be too strong, lets say "not usually done this way". Usually format string "%08X" for printf-like function is used to output the value. Specifically for XCRC it's hard to tell what is or is not allowed, because no real specification exists.

[nick7inc]

Usually I read from such strings by sscanf() function (or by my own polymorph function xscanf(), which can read from anywhere). And I see no problem to read both strings "250 C54282F" and "250 0C54282F" by this code:

Code: Select all

const char* test_string="250 C54282F"; 
// or 
// const char* test_string="250 0C54282F";
int res = sscanf(test_string,"%d %lx", &srv_ans, &crc);.

More over, it is easy to create multi-pattern read:

Code: Select all

#include <stdio.h>
int main (void)
{
 const char*pt []=
 {
     "%d %lx",
     "%d %08lx",
     "%d 0x%lx",
     "%d $%lx",
     0
 };

//  const char* test_string="250 C54282F";
//  const char* test_string="250 0C54282F";
//  const char* test_string="250 0xC54282F";
  const char* test_string="250 $C54282F";
  int srv_ans =0;
  unsigned long int crc=0;
  for (unsigned int i=0; pt[i];i++)
  {
      int res = sscanf(test_string,pt[i], &srv_ans, &crc);
      if (res==2) break; else srv_ans=0;
  }
if (srv_ans)
printf("Server ansver is %d, CRC is %08lX\n",srv_ans,crc);
else puts ("Error");
return(0);
}

[ghisler(Author)]

Currently TC uses the length of the returned string as one indication that the returned checksum is valid. If the length is bad, the question mark will remain, meaning "couldn't compare". I will try to support this special case, although it's very unusual.

Btw, does anyone know a sample file which gives a CRC32 of zero?

[Sob]

Not exactly zero, but with leading zeros:

Code: Select all

MIME-Version: 1.0
Content-Type: application/octet-stream; name="files.7z"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="files.7z"

N3q8ryccAAOa+7yJZQAAAAAAAAAgAAAAAAAAAP/w/XABABQwMDE1IA0KMDA1OCANCjUyMDMgDQoA
AACBMweuD8+SbmAP6+qcvwza+cR0Rnl1jAAwAaESpt9r85rlt3ZV+zm6d3Y+pfDaVEwKoXhyebRU
ZmJxOWCyQ4AgqK9y/zVtiH/AABcGGQEJTAAHCwEAASMDAQEFXQAQAAAMZwoBbQrAiAAA

[nick7inc]

2ghisler(Author)
I'll try to find some data peace with zero CRC by brute-force method. I need some time to create a suitable program and to find these data.
2Sob
Your file looks damaged. TC can't decode it.

[nick7inc]

ghisler(Author)


Here is file with zero CRC32 sum:

Code: Select all

MIME-Version: 1.0
Content-Type: application/octet-stream; name="data.zip"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="data.zip"

UEsDBAoAAgAAAJGT1zwAAAAABAAAAAQAAAAIAAAAZGF0YS5kYXSdCtltUEsBAhQACgACAAAAkZPX
PAAAAAAEAAAABAAAAAgAAAAAAAAAAAAgAAAAAAAAAGRhdGEuZGF0UEsFBgAAAAABAAEANgAAACoA
AAAAAA==

By the way. Why if I try to decode this file with TC it display warning message about illegal characters in source file (Mime)?

[nick7inc]

Another SERV-U Ftp server log for this kind of file:

Code: Select all

xcrc data.dat
250 0

[ghisler(Author)]

Thanks very much! It's good to know that it returns a zero and not an empty string, that's what I needed to know!