copy/paste in Compare by contents edit mode does not work correctly

Please report only one bug per message!

Moderators: white, Hacker, petermad, Stefan2

miskox
Member
Member
Posts: 166
Joined: 2003-06-11, 06:00 UTC

copy/paste in Compare by contents edit mode does not work correctly

Post by *miskox »

(9.51, 32-bit on x86 computer XP PRO)

When I do file compare by contents (ALT-F, Y) on these two files:

Left pane: just have an empty file (err.txt), size 0 bytes.

Right pane: filename ok.txt:

Have this:
CP1250 CP852
Č ¬
Š ć
Ž ¦
č ź
š ç
ž §
These both files are not unicode, UTF-8... just good old plain .txt files.

(you can download the ok.txt file here www.kockarna.si/file.zip - this file contains both ok.txt and error.txt files (so you can have your own err.txt file to test). And also a crash info for Christian. I was doing some CTRL-A, CTRL-C,CTRL-V and it crashed twice (second time CTRL-C was not working to save the info).

(left column: are letters ČŠŽčšž in CP1250 (ANSI), right column: same letters in CP852 (ANSI).

Now compare them.

- click 'edit mode'.
- CTRL-A on right pane
- CTRL-C on right pane
- click in the left pane (set focus to left pane)
- CTRL-V to paste everything.

Save left file. Do the compare by contents. FIles are different (5 letters are changed).

Do you have different files or is there something wrong with my settings?

(XP PRO, 32-bit, x86).

Thanks.
Saso
#224551
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48012
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: copy/paste in Compare by contents edit mode does not work correctly

Post by *ghisler(Author) »

Thanks for your report. I couldn't reproduce it so far - my regional settings use Western encoding (Swiss German), and I changed the font to "Middle Eurepean" script to see what you see.

Which encoding do you use on Windows?
Author of Total Commander
https://www.ghisler.com
miskox
Member
Member
Posts: 166
Joined: 2003-06-11, 06:00 UTC

Re: copy/paste in Compare by contents edit mode does not work correctly

Post by *miskox »

Keyboard is set to Slovenian, Operation system is in English. You should probably select Eastern European coding.

Here is the INFO.BAT information (for more info see dostips.com):

Code: Select all

 INFO.BAT version 1.5
whoami.exe not found.
--------------------------------------------------------------------------------
Windows version        :  Microsoft Windows XP [Version 5.1.2600]
Product name           :  Microsoft Windows XP, 32 bit
Performance indicators :  Processor Cores: 1      Visible RAM: 2095984 kilobytes

Date/Time format       :  (dd/mm/yy)  ned 24.01.2021  11:10:47,21
__APPDIR__             :  C:\WINDOWS\system32\
ComSpec                :  C:\WINDOWS\system32\cmd.exe
PathExt                :  .COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH
Extensions             :  system: Enabled   user: Enabled 
Delayed expansion      :  system: Disabled  user: Disabled
Locale name            :         Code Pages: OEM  852    ANSI 1250
DIR  format            :  24.01.2021  11:09       805.306.368 pagefile.sys
Permissions            :  Elevated Admin=Yes, Admin group=
System  Autorun found  :      AutoRun	REG_SZ

                          Missing from the PATH environment: C:\WINDOWS\system32\WindowsPowerShell\v1.0
                          Missing from the tool collection:  certutil
                          Missing from the tool collection:  choice
                          Missing from the tool collection:  clip
                          Missing from the tool collection:  forfiles
                          Missing from the tool collection:  icacls
                          Missing from the tool collection:  powershell
                          Missing from the tool collection:  robocopy
                          Missing from the tool collection:  timeout
                          Missing from the tool collection:  whoami
Next:
- regional setting have this:
o Regional options: Standards and Formats: slovenian
o Location: Slovenia
o Languages - Text services: default input language: Slovenian, keyboard: slovenian (only one). Under Advanced (compatibility configuration (extend support of advanced text services to all programs) and system configuration (turn off advanced text services): both are unchecked)
o Supplemental language support: install files for complex and right-to-left languages: unchecked. Install files for East asian languages: unchecked
o Advanced: Language for non-Unicode programs: Slovenian. Code page conversion tables: checked are: many of them (852 and 1250 are selected (color grey - cannot be changed).

One more thing: on my another (primary) computer everything works - so this means that this computer has some settings wrong (but it is not a primary computer so it is not so important to me). So you can close this topic.

Thanks.
Saso
#224551
User avatar
elgonzo
Power Member
Power Member
Posts: 866
Joined: 2013-09-04, 14:07 UTC

Re: copy/paste in Compare by contents edit mode does not work correctly

Post by *elgonzo »

TL;DR: There also seems to be some more clipboard corruption going on that is related to clipboard operations in the file compare tool in edit mode and looks awfully like a buffer overflow. This could explain the observed crashes. Having a zero-byte file involved is not a requirement for the issue to appear, as far as i can tell. It kinda looks like the already reported issue: https://www.ghisler.ch/board/viewtopic.php?f=15&t=46075&p=319138&hilit=file+compare+crash#p319138 , but that issue has been reported as being fixed or not occuring anymore...


I have played around in both 32-bit and 64-bit versions of TC 9.51 on Win7 x64 Pro, using fresh INI files.

When playing around with the reported problem scenario in an attempt to discover more specific condiditions and patterns of this issue occuring, i noticed that the selection copied to clipboard from the file compare tool in edit mode sometimes includes additional garbage data at the end of the copied data.

This makes me think that additional to a possible text encoding issue there is also an issue of buffer overflow.

I experienced such clipboard corruptions also when using two files that are not empty and contain pure ASCII characters only.

I have not found conditions yet that reproduce this issue reliably. But i found some conditions that i feel are either necessary or improve the chances of triggering the issue:

- Files to compare are of different size.
- Files are rather smallish (< 100 bytes)
- Prior to CTRL+A CTRL+C, remove some characters/lines from the end of the file in the edit window (don't save the file)
- The file whose content is being copied into the clipboard is inside a ZIP archive.


I also experienced two occurences where the clipboard itself became inaccessible when copying and pasting within the file compare tool in edit mode. One situation involved an access violation, the other time TC didn't crash but the clipboard still became inaccessible until i ended the TC process. Both occurences were with the 32-bit version of TC, but i haven't tested extensively enough to confidently state that this would be a 32-bit TC problem only.

Particularly with the observed access violation in mind, i suspect some buffer overflow happens in the file compare tool after it opens the clipboard. Under the "right" circumstances the suspected buffer overflow causes the respective routine in the file compare tool to bail out early and not closing the clipboard handle anymore until the clipboard handle held by TC is closed and destroyed by the OS when the TC process ends.

Below a photo of the crash dialog i captured. Due to clipboard access being denied because of the clipboard being held open by the TC instance, i was unable to copy the text of TC's crash dialog or make a screenshot of it. So i had to make a photo of the crash dialog. Sorry about the picture quality :)

Image: https://imgur.com/a/3f88dpa


P.S. Let me know if you prefer tracking this access violation / suspected buffer overflow issue separately, and i will move my post over to a new topic.
Begrudgingly back to Windows... now 11... sigh... but i have no nerve for Linux desktop anymore...
miskox
Member
Member
Posts: 166
Joined: 2003-06-11, 06:00 UTC

Re: copy/paste in Compare by contents edit mode does not work correctly

Post by *miskox »

@elgonzo:
- I to had a problem that CTRL+C and PRTSCR didn't work
- mine files were not in the archive (not single one or both)
- they are larger than 100 bytes.

I did some testing (XP PRO, 32bit, 9.51 TC):

I noticed that paste operation pastes more characters:
- only one character (x0A)
- two characters (x0A0D)
- even more:

see below:

Image: http://www.kockarna.si/drupal/sites/default/files/tc_compare1.png

See: two bytes were added + END (and space after the END): x0A0A456E6420 (LF+LF+END+space)

So it really looks like previous clipboard contents is pasted. This could be reason for some crashes (buffer overruns).

How the right file was created: CTRL-A, CTRL- C in the left pane, set focus to right pane, CTRL-A, DELETE, CTRL-V.

Also: first file (left pane) has a length of 683 bytes, right pane file has 691 bytes. So x0D0A was also added to the beginning of the file (total difference of 8 bytes): 2 at the beginning and 6 at the end.

Now do this: set focus to the left pane, CTRL-A, CTRL-C, set focus to right pane, CTRL-A and right click and select 'paste here'. CRASH (not always).

Another thing: when succesfully pasted: set focus to left pane, CTRL-A, delete everything. Right click in left pane and select 'paste here' doesn't work at all.

Crash:

Code: Select all

---------------------------
Total Commander 9.51
---------------------------
Access violation at address 004027F5. Read of address 00181061.
Access violation at address 004027F5. Read of address 00181061
Windows XP SP3 5.1 (Build 2600), base: 00400000

Please report this error to the Author, with a description
of what you were doing when this error occurred!

Windows exception: C0000005
Stack trace:
004027F5

Raw:
005146D6  004324D0  00432FF9  004325D0  00433262  004335A4
00433980  >00429486  00412BD0  0042ECC1  0042ED5C  0042EF16

Press Ctrl+C to copy this report!
Continue execution?
---------------------------
Yes   No   
---------------------------
Fresh instance of TC causes crash when trying to 'paste here' when CTRL-A is in effect. CTRL+C does not work. Running a program from TC's command line doesn't work.

Hope this helps.
Saso
#224551
User avatar
Usher
Power Member
Power Member
Posts: 1675
Joined: 2011-03-11, 10:11 UTC

Re: copy/paste in Compare by contents edit mode does not work correctly

Post by *Usher »

miskox wrote: 2021-01-23, 19:26 UTC (9.51, 32-bit on x86 computer XP PRO)
FIles are different (5 letters are changed).

Do you have different files or is there something wrong with my settings?
Not confirmed when no prior edit before copy-and-paste (see @elgonzo comment). Files are identical, no crash.
However, I observed another issue. When pressing Ctrl+V in the left panel, TC inserts 4 empty lines below the first line in the right panel like that:

Code: Select all

CP1250 CP852




Č ¬
Š ć
Ž ¦
č ź
š ç
ž § 
Empty lines aren't numbered.
Windows XP Home 32-bit, Polish.
ANSI codepage windows-1250, OEM codepage DOS CP-852, keyboard Polish (programers).
TC language set to English internal, wincmd.ini in UTF-16.

@elgonzo
It may be regression caused by some of numerous fixes in compare by content code.
Andrzej P. Wozniak
Polish subforum moderator
User avatar
elgonzo
Power Member
Power Member
Posts: 866
Joined: 2013-09-04, 14:07 UTC

Re: copy/paste in Compare by contents edit mode does not work correctly

Post by *elgonzo »

Usher wrote: 2021-01-24, 19:14 UTC @elgonzo
It may be regression caused by some of numerous fixes in compare by content code.
It might be. There is an entry in the history.txt:
history.txt wrote:25.10.16 Fixed: Crash in "Compare by content" after edit, re-compare (with save), edit, then "recompare from here" (32/64)
which could indicate my observation being a regression with regard to the editing function of the file compare tool.

Although, the fact that the corruptions and crash i observed did not require re-comparing nor saving, but just some massaging with copy'n'paste might also indicate a different, unrelated bug with the editing function. Only Christian can tell...
Begrudgingly back to Windows... now 11... sigh... but i have no nerve for Linux desktop anymore...
miskox
Member
Member
Posts: 166
Joined: 2003-06-11, 06:00 UTC

Re: copy/paste in Compare by contents edit mode does not work correctly

Post by *miskox »

Here is the crash information (which couldn't be copied to clipboard nor print screened).
Image: http://www.kockarna.si/drupal/sites/default/files/tc_crash.jpg

Saso
#224551
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48012
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: copy/paste in Compare by contents edit mode does not work correctly

Post by *ghisler(Author) »

2miskox
I have tried to reproduce your problem with Slovenian locale, but failed. Here is what I tried:
1. Booted Windows XP 32-bit service pack 3 (build 2600) in Virtualbox.
2. Opened Control panel - Regional and language options - Advanced
3. Changed the "Language for non-Unicode programs" to Slovenian
4. Restarted the PC as requested
5. Started TC 9.51 with a fresh wincmd.ini: c:\totalcmd\TOTALCMD.exe /i=c:\totalcmd\test.ini
6. Compared empty.txt (empty file) with ok.txt
7. F6 edit mode
8. Switched to right panel
9. Ctrl+A, Ctrl+C
10. Clicked in left panel
11. Ctrl+V
12. Saved left file with Ctrl+S
13. Closed compare tool
14. Compared the two files again
-> "The two files have the same content!"

I was also unable to reproduce your crash report. The address is in code where text is copied from the file buffer to the clipboard buffer.
I'm not surprised that Ctrl+C no longer works after the observed crash. It happens because TC still has the clipboard open, so it cannot be opened again - only after closing TC.

===========================================================================================

Can you please test the following:
1. Start Total Commander with a fresh ini file, e.g. c:\totalcmd\TOTALCMD.exe /i=c:\totalcmd\test.ini
2. Try to reproduce the copy+paste problem
3. If it does not occur, please send me your original wincmd.ini to support at ghisler dot com
4. If it still occurs, I need more info about your setup.
4a. Do you use Windows XP Service pack 3 too? See menu Commands - System information - Software.
4b. Please compare all the regional and language settings with the computer where it works. Any differences? Same Windows?
5. I need step by step instructions to reproduce the Ctrl+C crash:
- a file with which it happens
- exactly what you selected to get the crash
Author of Total Commander
https://www.ghisler.com
miskox
Member
Member
Posts: 166
Joined: 2003-06-11, 06:00 UTC

Re: copy/paste in Compare by contents edit mode does not work correctly

Post by *miskox »

@ghisler: I will try that - I cannot promise that it will be today (evening). SP3 is installed for sure, but I cannot say that all the updates are there.

Did you perform many copy/paste operations (with different lengths of text in the clipboard)?

I will post here my findings.

Thank you.
Saso
#224551
User avatar
elgonzo
Power Member
Power Member
Posts: 866
Joined: 2013-09-04, 14:07 UTC

Re: copy/paste in Compare by contents edit mode does not work correctly

Post by *elgonzo »

ghisler(Author) wrote: 2021-01-26, 10:31 UTC I was also unable to reproduce your crash report. The address is in code where text is copied from the file buffer to the clipboard buffer.
I did observe garbage characters/bytes included at the end of the clipboard text copied from the selection, an access violation crash, and a zombified open clipboard handle on an english Win7 x64 Pro with pure ASCII files. Thus i believe the crashes themselves are not related to regional settings or text encoding issues, and i believe trying to associate these access violations with regional settings or non-ASCII files is a red herring.

Since you know that access violation originates in the code copying the file buffer to the clipboard buffer, in lack of reliable reproduction steps (which someone might still post in the future, though) a code review could be done with regard to
- wrongly sized/allocated or wrongly reused buffers
- any variables specifying buffer sizes and numbers of bytes to be copied
- in case the copy function relies on the presence of a null terminator in the soure buffer, any code or function call that sets or modifies the file buffer content for behavior or bugs that could lead to not setting or overwriting a null terminator.
Begrudgingly back to Windows... now 11... sigh... but i have no nerve for Linux desktop anymore...
miskox
Member
Member
Posts: 166
Joined: 2003-06-11, 06:00 UTC

Re: copy/paste in Compare by contents edit mode does not work correctly

Post by *miskox »

First try to start TC with empty .ini file was successful. No crashes there. Second time I was not able to start it - it always picked .ini file that is located in the "c:\Documents and Settings\saso\Application Data\GHISLER" folder. So I removed this one (moved to another folder). Now crash occured (twice):

Here is the first crash:

Code: Select all

---------------------------
Total Commander 9.51
---------------------------
Access violation at address 004027F5. Read of address 00165EC4.
Access violation at address 004027F5. Read of address 00165EC4
Windows XP SP3 5.1 (Build 2600), base: 00400000

Please report this error to the Author, with a description
of what you were doing when this error occurred!

Windows exception: C0000005
Stack trace:
004027F5

Raw:
005146D6  004324D0  00432FF9  004325D0  00433262  004335A4
00433980  >00429486  00412BD0  0042ECC1  0042ED5C  0042EF16

Press Ctrl+C to copy this report!
Continue execution?
---------------------------
Yes   No   
---------------------------
Second time when I copied the text and pasted it - pasted text was from the previous try. Very strange.

I will do more tests (it might take a while).

This post is just if this crash dump gives more clue.

Saso
#224551
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48012
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Re: copy/paste in Compare by contents edit mode does not work correctly

Post by *ghisler(Author) »

2miskox
What exactly do you try to copy to get the crash? I tried with your sample file ok.txt (Ctrl+A, Ctrl+C) but do not get any crashes.

Maybe it only happens on that PC where the other error occurs? Do you get this crash also on the other PC where there is no copy error?
Author of Total Commander
https://www.ghisler.com
User avatar
elgonzo
Power Member
Power Member
Posts: 866
Joined: 2013-09-04, 14:07 UTC

Re: copy/paste in Compare by contents edit mode does not work correctly

Post by *elgonzo »

2ghisler(Author)

It doesn't seem to matter.

Look at the screenshots of other crashes i have gotten when further trying to find ways to more reliably reproduce the issue (this time in x64 TC, with the clipboard fortunately not being blocked).

Image: https://imgur.com/a/QAo51ap
Image: https://imgur.com/a/jwSN1TO

For crying out loud, i could even reproduce the issue in a VM running a blank Win7 x64 Pro:

Image: https://imgur.com/a/X51Fm96

Note that the stack traces in all three screenshots are exactly the same, but the files involved are different. And while the involved files are different, note that their content is ASCII only with Windows-style line breaks (\r\n).

Also, i don't know whether it is just coincidence or not, but in all 3 crashes i captured i always had selected the content of the larger of the two files.



The procedure i did was always the same:

1. Start TC with fresh INI.
2. Select one of the two files in each of the two file panels.
3. Open File compare tool (Alt+F, Y)
4. Enter edit mode (Alt+M)
5. Sometimes i TAB into the right panel, sometimes i leave the left panel active.
6. ALT+A, ALT+C
7. If unlucky: no crash. If lucky: crash.
8. If no crash, close TC, and start again with step 1.

I have had to try quite a number of times, at least 40 ... 50 times, if not more, between crashes, and did not switch between different pairs of files during the test repititions. I endured this only with the help of copious amounts of coffee...


The relatively infrequent occurence of the crashes despite always executing the same sequence of steps again and again points towards some race condition. Don't use a debug build for testing, use the release build as used by the users. (It is not unusual for the runtime behavior of debug builds to differ in such a way that race conditions experienced with release builds don't happen with debug builds.) Troubleshooting this nasty will certainly be no fun for you.


If you still can't reproduce the problem on your end, i can offer you to provide the VM disk image that i used to run and crash TC's file compare tool (the one you see running in my 3rd screenshot). It is in vdmk format and compressed would be an estimated 10 .. 15 GB in size. The Windows 7 x64 OS on it is not registered/activated (as you can see in the lower right corner of the screenshot) and has VMWare tools installed in it. Other than that, there is no other software installed besides TC, so there shouldn't be any issue regarding sharing or distribution of licenses. I take it as a given that you already have VMs or can setup a VM quickly on your own, but hey, in any case, let me know if you are interested in it...
Begrudgingly back to Windows... now 11... sigh... but i have no nerve for Linux desktop anymore...
miskox
Member
Member
Posts: 166
Joined: 2003-06-11, 06:00 UTC

Re: copy/paste in Compare by contents edit mode does not work correctly

Post by *miskox »

2ghisler: currently I have something very important planned in the next two weeks or so (decision was made yesterday) so my tests will be delayed for a while.
Also: if I start TC with empty .ini copy/paste of my national characters (ČŠŽ) is OK. Looks like it was caused by damaged (?) .ini file. WIll send it to you.

2elgonzo: thank you very much for all the tests. As written above... I will try and do more tests when time permits (if required at all based on your post).

Saso
#224551
Post Reply