TC7.5rc1: Wrong FTP directory listing in UTF-8 mode

[yong]

Recently some users on a Chinese software forum reported two FTP directory listing bugs in RC1 of TC 7.5. I have been able to reproduce these bugs on Windows XP and here are the symptoms:

Bug 1:

This bug occurs when TC enables UTF-8 mode and the server in question is giving directory strings in UTF-8 too. When the user switches into a directory that contains Chinese characters the current directory bar (breadcrumb bar) shows the directory name as

0:/XXX/YYY" is current directory./*.*

XXX and YYY being some Chinese characters. The string [" is current directory.] (enclosed in square brackets for clarity) is appended to the directory name.

Bug 2:

When the file list font is changed to Fixedsys and TC logs in to the above FTP server, some Chinese directory names show a trailing black square which isn't seen either in the breadcrumb bar or in the directory history (Alt+Down). This black square is not visible if the font is something else.

These bugs were found in earlier public betas, according to the users who reported them. My guess is that something screwed up in the code that counts Unicode characters in the FTP strings. Oops!

[ghisler(Author)]

Unfortunately I cannot reproduce that with any Chinese names I tried (I used RaidenFTPd as the ftp server, which is a Chinese product, so it should know how to handle Chinese names).

I guess that it happens only with certain Chinese directory names. Can you post a directory name please which causes the problem? The forum can show it. Thanks!

[yong]

I guess that it happens only with certain Chinese directory names.

Yes, these problems only happen with some Chinese directory names, and when a directory name has a trailing black box in Fixedsys, that directory also has that ["is current directory.] string.

The FTP site I had problems with runs Gene6 FTP Server v3.8.0 (Build 34). The following is the directory strings from that server (in UTF-8):

Code: Select all

drw-rw-rw-   1 user     group           0 Oct 21  2008 北朝鲜核试验之谜
drw-rw-rw-   1 user     group           0 Feb 19  2008 北朝鲜纪实(3)
drw-rw-rw-   1 user     group           0 Jan 12  2008 采棉女-中国劳务输出的三个月
drw-rw-rw-   1 user     group           0 Dec 23  2007 从楷模到公敌-亚洲留学生眼中的日本
drw-rw-rw-   1 user     group           0 Jan 02  2008 大蒙古帝国(4)
drw-rw-rw-   1 user     group           0 Dec 21  2007 东京-终战日
drw-rw-rw-   1 user     group           0 Jun 29 19:12 返乡潮-2009年春节广州站纪实
drw-rw-rw-   1 user     group           0 Jan 12  2008 刚果河-黑暗启示录
drw-rw-rw-   1 user     group           0 Oct 21  2008 公平社会-左倾拉丁美洲的挑战
drw-rw-rw-   1 user     group           0 Dec 20  2007 故宫的至宝(26)
drw-rw-rw-   1 user     group           0 Jan 12  2008 关口知宏中国铁道大纪行-最终回
drw-rw-rw-   1 user     group           0 Dec 24  2008 话说改革开放30年
drw-rw-rw-   1 user     group           0 Jul 22  2008 激流中国(13)
drw-rw-rw-   1 user     group           0 Oct 21  2008 纪实72小时(12)
drw-rw-rw-   1 user     group           0 Oct 21  2008 柬埔寨大屠杀的真相
drw-rw-rw-   1 user     group           0 Jan 12  2008 紧急运送-人气商品越洋记
drw-rw-rw-   1 user     group           0 Dec 23  2007 满洲拓殖移民的真相-东宫铁男资料揭秘
drw-rw-rw-   1 user     group           0 Jan 07  2008 煤矿工与小煤窑
drw-rw-rw-   1 user     group           0 Dec 23  2007 明治(4)
drw-rw-rw-   1 user     group           0 Dec 21  2007 欧洲野餐计划
drw-rw-rw-   1 user     group           0 Oct 21  2008 普京的俄罗斯(4)
drw-rw-rw-   1 user     group           0 Dec 21  2007 齐奥塞斯库政权的崩溃-市民拍摄的七日革命
drw-rw-rw-   1 user     group           0 Dec 22  2007 秦始皇(3)
drw-rw-rw-   1 user     group           0 Oct 21  2008 秋叶原物语
drw-rw-rw-   1 user     group           0 Jan 12  2008 日本的垃圾去大陆
drw-rw-rw-   1 user     group           0 Oct 08  2008 日本的穷忙族(3)
drw-rw-rw-   1 user     group           0 Jan 07  2008 日本企业长盛不衰的奥秘
drw-rw-rw-   1 user     group           0 Oct 21  2008 三点同时纪实系列(3)
drw-rw-rw-   1 user     group           0 Jan 12  2008 三个代表-中国共产党的新战略
drw-rw-rw-   1 user     group           0 Dec 23  2007 上海拆迁悲喜剧
drw-rw-rw-   1 user     group           0 Feb 01 10:39 四川大地震灾区现状
drw-rw-rw-   1 user     group           0 Dec 21  2007 团结工会十年的轨迹
drw-rw-rw-   1 user     group           0 Dec 21  2007 吞没世界的巨龙和巨象(2)
drw-rw-rw-   1 user     group           0 Dec 11  2007 文革那些事儿(4)
drw-rw-rw-   1 user     group           0 Dec 21  2007 文化大革命-四十年后的证言
drw-rw-rw-   1 user     group           0 Jan 12  2008 卧龙大熊猫研究中心的六个月
drw-rw-rw-   1 user     group           0 Dec 23  2007 匈牙利动乱的真相
drw-rw-rw-   1 user     group           0 Oct 21  2008 颜色革命的背后
drw-rw-rw-   1 user     group           0 Jan 12  2008 摇曳的童心-中国浪潮下的缅甸寺塾生
drw-rw-rw-   1 user     group           0 Dec 21  2007 隐形眼镜和自由的故事
drw-rw-rw-   1 user     group           0 Nov 14  2007 映像的世纪(11)
drw-rw-rw-   1 user     group           0 Jan 12  2008 在法律与现实之间 某位北京人大代表的职责
drw-rw-rw-   1 user     group           0 Jan 07  2008 中国残留孤儿归国记(2)
drw-rw-rw-   1 user     group           0 Jun 01  2008 中国四川大地震
drw-rw-rw-   1 user     group           0 Jan 12  2008 中国通向富裕的摸索
drw-rw-rw-   1 user     group           0 Jan 12  2008 中国支配下的北朝鲜经济
drw-rw-rw-   1 user     group           0 Feb 17  2008 中日对决-制造工艺人的攻防战
drw-rw-rw-   1 user     group           0 Oct 09  2008 中日战争特集(2)
drw-rw-rw-   1 user     group           0 Jan 12  2008 中日战争相关(5)

Some directory names (for example the 6th from the top, dated Dec 21 2007) have problems. I'm sending you the FTP listing as well as FTP logs by e-mail so that you can check it yourself.

[yong]

I think I've found out what's causing this problem: your UTF-8 parsing code fails to handle the carriage return (0xD)/line feed (0xA) combination properly when it comes after some Chinese characters, an extra carriage return is inserted. The following is the FTP log as seen in vim:

Code: Select all

CWD 东京-终战日^M    <- Extra carriage return

250 CWD command successful. "/MovieZ-Down/-=NHK=-/东京-终战日" is current directory.
PWD
257 "/MovieZ-Down/-=NHK=-/东京-终战日" is current directory.
读取文件夹
PASV
227 Entering Passive Mode (10,12,6,2,96,161)
Server reports local IP -> Redirect to: 58.52.147.231
LIST
150 Data connection accepted from 10.12.6.1:41797; transfer starting.
下载
等待服务器应答...
226 Transfer ok.

[ghisler(Author)]

Thanks for the listing, I think that I could reproduce and fix it now! We will see with the upcoming RC2.

[ghisler(Author)]

Please test this with RC2, and let me know if you still encounter any directory or file names which cause problems. Please list some of the names here so I can test the problem myself. Thanks!

[yong]

It's fixed in RC2, so far so good. Thank you!

[ghisler(Author)]

Great, thanks very much!