Wrong timestamp in RAR files modified in other time zone

Bug reports will be moved here when the described bug has been fixed

Moderators: white, Hacker, petermad, Stefan2

mikeyww
Junior Member
Junior Member
Posts: 41
Joined: 2005-07-02, 02:58 UTC

Wrong timestamp in RAR files modified in other time zone

Post by *mikeyww »

TC displays the wrong modification time for some files inside RAR archives. This occurs when the file was modified in Daylight Savings Time, while the current time zone is not in Daylight Savings Time. In such cases, TC may display a time that is shifted by one hour, while WinRAR displays the file's correct time. More information about Windows file times is available at <https://goo.gl/t3ZI2P>.

Mike
User avatar
Hacker
Moderator
Moderator
Posts: 13052
Joined: 2003-02-06, 14:56 UTC
Location: Bratislava, Slovakia

Post by *Hacker »

Hi Mike,
winmd.ini - [Configuration]
NewTimeZoneMethod= 1 Windows 7-style timezone method (default on Windows 7)
0: Old method (default on older Windows)
HTH
Roman
Mal angenommen, du drückst Strg+F, wählst die FTP-Verbindung (mit gespeichertem Passwort), klickst aber nicht auf Verbinden, sondern fällst tot um.
mikeyww
Junior Member
Junior Member
Posts: 41
Joined: 2005-07-02, 02:58 UTC

Post by *mikeyww »

Interesting, but that actually does not fix this issue. Using NewTimeZoneMethod=0 shifts the displayed mod time of both the native file and the archived file. These times were previously one hour different. They are now still one hour different, but a different hour. It's possible that the fix requires following the guidelines on the Microsoft.com Web page. Try RAR-archiving a file that was last modified in Daylight Savings Time, and you will see the problem.
User avatar
MVV
Power Member
Power Member
Posts: 8702
Joined: 2008-08-03, 12:51 UTC
Location: Russian Federation

Post by *MVV »

mikeyww,
If some timestamps are the same in WinRar and TC but some ones are different, it seems that time zone or daylight saving time information is stored somehow in RAR archive but TC misses it somehow... So perhaps an example of such archive would be much more useful than an MSDN article link, especially when we talk about something that is not related to Microsoft or Windows (RAR format).

BTW, does TC show wrong timestamps for all files of that archive or some ones are shown properly?
mikeyww
Junior Member
Junior Member
Posts: 41
Joined: 2005-07-02, 02:58 UTC

Wrong timestamp in RAR files modified in other time zone

Post by *mikeyww »

The problem will now occur for all files modified in Daylight Savings Time.

Reproducing the problem is easy: simply RAR-compress a file modified in Daylight Savings Time, and check the timestamp to see the problem. An example is also provided in <https://cogit.net/z/uloi33mu>.

Mike
User avatar
milo1012
Power Member
Power Member
Posts: 1158
Joined: 2012-02-02, 19:23 UTC

Post by *milo1012 »

This is a bug in WinRAR 5.x, and not in TC.

When I use WinRAR 4.11 to pack the file, the timestamp is saved correctly. But WinRAR 5.40 adds the mentioned 1 hour difference (you can see the changed DOS time at file offset 0x0028). I found this behavior on Windows 7. On Windows XP, both WinRAR 4.x and 5.x work correctly*.

So you probably should report this to Rarlab instead.

I guess it was caused by this bugfix:
http://www.rarlab.com/rarnew.htm wrote:Version 5.11
...
d) archived files could have 1 hour modification time error
in Windows XP;

*I tested only the older Rar 3.0 (2.9) format, which is indeed limited to local (DOS/FAT) time, but not the newer Rar 5 format which uses UTC time internally
TC plugins: PCREsearch and RegXtract
mikeyww
Junior Member
Junior Member
Posts: 41
Joined: 2005-07-02, 02:58 UTC

Post by *mikeyww »

I have already discussed with WinRAR. They provided the following explanation.

"NTFS stores file times in UTC, but WinRAR in RAR4 archive format, like many other programs, operates with local file times. When we convert time between UTC and local, Windows adjusts it to current daylight saving automatically, by adding or subtracting one hour. It adjusts all file times equally, either files were created in daylight saving period or not and it creates the problem. RAR5 archive format stores file time in UTC and I suppose, archives in the new format should have less daylight saving related issues. Unless running in Windows XP, WinRAR uses the new approach recommended by Microsoft. Other programs likely use the old style FileTimeToLocalFileTime call."

WinRAR's explanation indicates that they have assessed and understood the Windows timestamp problem, and have adjusted their software accordingly. As a result, the WinRAR program displays the correct timestamp for files inside its archives.

Mike
User avatar
milo1012
Power Member
Power Member
Posts: 1158
Joined: 2012-02-02, 19:23 UTC

Post by *milo1012 »

mikeyww wrote:I have already discussed with WinRAR. They provided the following explanation.
...
WinRAR's explanation indicates that they have assessed and understood the Windows timestamp problem, and have adjusted their software accordingly.
Your quote is just a commonplace explanation for the UTC<->non-UTC difference. From what I can see, it might not be a bug, but Rar 5.x simply trying to be "intelligent" and dropping the (former Rar 4.x) DS hour adjustment when storing the timestamp (when being on newer OS / not XP), although the archive timestamp storage is a mere DOS/FAT time field. So what you get is a local/non-UTC time already compensated for DS time. This *may* sound like a good idea, but it's bad for a main reason:
I breaks compatibility for all external applications (non-native Rar), because there is no way to know (w/o an additional flag) if the timestamp at hand was already compensated for DS or not.
So in case of TC: how is it supposed to know if the archive was created by Rar 5.x or an older version? You just have the raw 4-byte DOS time field.

Sorry, but this is not a TC bug.
Though there may be a solution by implementing a new TC option for automatic adjustment (if the user thinks that most of his rar files were created with Rar 5.x).
mikeyww wrote:As a result, the WinRAR program displays the correct timestamp for files inside its archives.
It just shows what is stored: the 4-byte DOS time field. Your so-called "correct" behavior results from the mentioned adjustment when creating the archive with Rar 5.x.
Last edited by milo1012 on 2017-02-17, 05:24 UTC, edited 1 time in total.
TC plugins: PCREsearch and RegXtract
mikeyww
Junior Member
Junior Member
Posts: 41
Joined: 2005-07-02, 02:58 UTC

Post by *mikeyww »

Thanks for the explanation.

One thing to note: Microsoft specifically recommends the following: "To account for daylight saving time when converting a file time to a local time, use the following sequence of functions instead of using FileTimeToLocalFileTime: FileTimeToSystemTime, SystemTimeToTzSpecificLocalTime, SystemTimeToFileTime". WinRAR is apparently following this. I wonder if TC is doing the same when it displays timestamps of files inside RAR archives.

Mike
User avatar
milo1012
Power Member
Power Member
Posts: 1158
Joined: 2012-02-02, 19:23 UTC

Post by *milo1012 »

mikeyww wrote:One thing to note: Microsoft specifically recommends the following:
...
WinRAR is apparently following this. I wonder if TC is doing the same.
This is a ~20 year old MSDN article. If TC would be doing anything "wrong", then we wouldn't have the correct (incorrect for Rar 5.x archives in your concept) adjustment in the first place.

Like I said, I'm not sure if this is a Rar 5.x bug or actual intended behavior. The Rar changelog didn't mention it.
TC plugins: PCREsearch and RegXtract
mikeyww
Junior Member
Junior Member
Posts: 41
Joined: 2005-07-02, 02:58 UTC

Post by *mikeyww »

Thanks again.

Mike
Eugene_Roshal
Junior Member
Junior Member
Posts: 14
Joined: 2013-05-02, 09:58 UTC
Contact:

Post by *Eugene_Roshal »

milo1012 wrote: This is a bug in WinRAR 5.x, and not in TC.

When I use WinRAR 4.11 to pack the file, the timestamp is saved correctly.
I suppose, you mean matching Total Commander under "correctly", but we need to match Windows Explorer file times. This is why in WinRAR 5.10 we switched to calling FileTimeToSystemTime, SystemTimeToTzSpecificLocalTime, SystemTimeToFileTime recommended by Microsoft instead of single FileTimeToLocalFileTime call when converting UTC to local time. Exact source code can be found in RarTime::GetLocal in UnRAR source code.

We get UTC times from disk and need to convert them to local times for RAR4 archive format.

I extracted test.rar from tctzissue170216.zip available above in this topic, opened test.rar in WinRAR 5.40, it contains test.txt with 16:40 time. I unpacked test.txt and it still has 16:40 time both in WinRAR and Explorer. I compressed test.txt again using RAR4 and RAR5 formats and it was 16:40 in both archives. When I open either the original test.rar or two new archives in Total Commander, I see 17:40 time.
milo1012 wrote: d) archived files could have 1 hour modification time error
in Windows XP;
SystemTimeToTzSpecificLocalTime based code produces 1 hour error in XP, so we had to resort to old style code in XP only. Yes, it is messy enough, but XP is on decline anyway.
milo1012 wrote: Your quote is just a commonplace explanation for the UTC<->non-UTC difference.
Relevant part of that quote was about using 3 functions instead of 1 to convert between UTC and local time.
milo1012 wrote: but Rar 5.x simply trying to be "intelligent" and dropping the (former Rar 4.x) DS hour adjustment
RAR does not drop any DS adjustments intentionally. It just uses a newer recommended way to convert times, so it matches Windows Explorer.
milo1012 wrote: So in case of TC: how is it supposed to know if the archive was created by Rar 5.x or an older version?
If you wish to display the same time as WinRAR, use conversions between UTC and local time from timefn.cpp in unrar source code. Yes, it will introduce 1 hour error for older archives, but newer archives created by WinRAR 5.1+ will be displayed correctly. Microsoft provided that new approach to convert times after we implemented first WinRAR versions, so we had to switch at some moment.
milo1012 wrote: This is a ~20 year old MSDN article. If TC would be doing anything "wrong", then we wouldn't have the correct (incorrect for Rar 5.x archives in your concept) adjustment in the first place.
Earlier WinRAR versions also used the obsolete single function based time conversion. Then we had to change it to match Explorer.

Besides, archives in RAR5 format store the file time in UTC making internal representation argument irrelevant, but Total Commander still introduces the same 1 hour difference for them. I think, TC uses unrar.dll to read RAR5 archives, so it gets the correct FILETIME from unrar.dll and 1 hour difference is likely to be introduced on very final step of converting UTC FILETIME to SYSTEMTIME to display time to user. It should be done with FileTimeToSystemTime and SystemTimeToTzSpecificLocalTime to match WinRAR and Explorer, not with FileTimeToLocalFileTime.
Last edited by Eugene_Roshal on 2017-02-17, 10:54 UTC, edited 1 time in total.
mikeyww
Junior Member
Junior Member
Posts: 41
Joined: 2005-07-02, 02:58 UTC

Post by *mikeyww »

Thanks, Eugene! My observations match yours.

I just want to clarify that all of the quotes attributed to me were actually written by others in this thread.

Everyone here seems to agree that regardless of the approach, at least some archives may show the wrong times. I agree with Eugene that the better approach would have the newer archives showing the correct times.

Mike
Eugene_Roshal
Junior Member
Junior Member
Posts: 14
Joined: 2013-05-02, 09:58 UTC
Contact:

Post by *Eugene_Roshal »

mikeyww wrote: I just want to clarify that all of the quotes attributed to me were actually written by others in this thread.
Yes, I copy pasted a wrong name :) Sorry, fixed now.
User avatar
milo1012
Power Member
Power Member
Posts: 1158
Joined: 2012-02-02, 19:23 UTC

Post by *milo1012 »

2Eugene_Roshal
Nice to see you around here personally.
Eugene_Roshal wrote:If you wish to display the same time as WinRAR, use conversions between UTC and local time from timefn.cpp in unrar source code. Yes, it will introduce 1 hour error for older archives, but newer archives created by WinRAR 5.1+ will be displayed correctly. Microsoft provided that new approach to convert times after we implemented first WinRAR versions, so we had to switch at some moment.
So your explanations fit my findings.
I'm aware of the MS decription. While it might be more correct, I still find this approach unfitting for a format with a raw DOS time field and no additional flag(s). While I'm glad that RAR5 introduced UTC, I don't see why you changed it for the older format out of the blue (considering the Rar 2.9 format as "lagacy" at some point in the future anyway), plus it wasn't explained in the changelog.

Anyway, let's see what Christian thinks about this.
And thanks for the explanations.
TC plugins: PCREsearch and RegXtract
Post Reply