[Implemented in other way] Limit file size for HTML parser in Lister 5 mode (bug and suggestion)

[Usher]

When testing media player, users sometimes test switching lister modes and press all numbers. If you press 5 for some huge video (for example, 16 GB), you must wait for parsing all the file. It may take a minute or even more.
Parsing cannot be cancelled. You should not try to press other numbers or Esc - in this case TC may crash.
That's why it should be default size limit for HTML parser (for example 16 MB, changeable in wincmd.ini).

[ghisler(Author)]

I will check whether I can make it cancellable.

[Usher]

For me the best solution is warning for large files (with user defined size limit), which gives possibility to cancel parsing.

[ghisler(Author)]

Wouldn't it be better to show a progress dialog after e.g. 2 seconds of parsing time which would allow the user to abort parsing of any file?

[Hacker]

Christian,

Wouldn't it be better to show a progress dialog after e.g. 2 seconds of parsing time which would allow the user to abort parsing of any file?

I think it would be better.

Roman

[Usher]

2ghisler(Author)
I don't think so. Have you seen any html/xhtml file greater than 100 MB? Have you ever seen email message greater than 100 MiB? Even mht or XML files with embedded Base64-encoded binaries shouldn't be so great. So for valid files progress indication is futile, and 2 seconds may be enough to switch to all Lister modes.

If the warning contains filename, the reason of the warning will be obvious.

I did check. I have found 33282 files:
1. 1 of them is 256 MiB file recovered from broken partition, only first 64 KiB are HTML content.
2. 2 copies of some XML migration log, the newer one is 158 MiB, the older one is 85 MiB. Maybe some MS Office install logs are greater?
3. All the rest is less than 16 MiB each.

There are files (compressed with brotli or gz) in the browsers' caches which have no extension, but the greatest ones (about 5 MiB compressed, about 25 MiB unpacked) contain javascript code.

[DrShark]

Have you seen any html/xhtml file greater than 100 MB? Have you ever seen email message greater than 100 MiB?Even mht or XML files with embedded Base64-encoded binaries shouldn't be so great.

One of Windows report generating functions can create quite large *.mht files, as mentioned here: https://ghisler.ch/board/viewtopic.php?p=329533#p329533.
Such a file in .html format has a comparable size.

BTW, in the past I suggested an option to control the files of which size should be passed to F4 editor:
https://ghisler.ch/board/viewtopic.php?f=14&t=72724
Maybe some solution with common/similar design could be implemented for both Lister/F3 viewer and Editor?

[Usher]

One of Windows report generating functions can create quite large *.mht files, as mentioned here: https://ghisler.ch/board/viewtopic.php?p=329533#p329533.
Such a file in .html format has a comparable size.

They are comparable with my logs from p.2. Anyone found anything greater?

BTW, in the past I suggested an option to control the files of which size should be passed to F4 editor:
https://ghisler.ch/board/viewtopic.php?f=14&t=72724
Maybe some solution with common/similar design could be implemented for both Lister/F3 viewer and Editor?

And there was my similar solution there…

Here it could be as follows:
wincmd.ini, section [Lister]
HTMLParsingLimit=16
where
-1 - don't parse at all
0- no limit
number > 0 - max size in MiB
Default value may be OS dependent (the older OS, the slower hardware), for example:
16 - Windows 9x/2000
32 -Windows XP
64 - Windows Vista/7
128 - Windows8+

Now the Warning proposition:

Code: Select all

Warning! It may take a long time to parse a large file as HTML (164 MB):
D:\a very very very\long long long\really full path\to a very very very large file\filename.ext
           Do you want to continue?
             [Yes]  [[NO]]  [Esc]

"No" is selected, Esc=NO, file extension should be always visible…

[Usher]

Wouldn't it be better to show a progress dialog after e.g. 2 seconds of parsing time which would allow the user to abort parsing of any file?

It doesn't work as expected. When I press 5 and TC displays progress bar, I can click"Cancel" - and it's OK. But then TC switches view to mode 5 and continues to display progress bar so I have to click "Cancel" again - and it's wrong. The same occurs when I press Esc to cancel operation.

[ghisler(Author)]

It seems to happen only when you press '5' while a lister plugin is shown. Then I'm getting the key down event twice. When I choose '5' from the menu or when in text mode, it doesn't happen. I will try to find a workaround.

[Usher]

21.04.21 Fixed: Lister: Press '5' while a plugin was shown -> conversion to html was done twice because the key event was received twice (64)

Still not fixed for 32-bit TC.

[ghisler(Author)]

It never happened here in 32-bit, so I only changed the 64-bit verison. I have just made some more tests with pressing '5' in 32-bit lister, then aborting the conversion with a mouse click or ESC, but I never got a second extraction. Lister was in mode 1, 2, 3, 5, 6, 7 and in various plugins (ulister, logtail, etc) when I pressed '5'.

Can you give me more details of when this happens?
1. Type of file you viewed
2. Mode in which you were when you pressed '5'
3. How you aborted the conversion

Thanks!

[Usher]

TC 32-bit in Windows XP, tested with new Media Player as in this topic: viewtopic.php?f=15&t=74262
I have described all needed steps, just see my original description:

If you press 5 for some huge video (for example, 16 GB), you must wait for parsing all the file.

Use at least 1 GB video for tests (avi, mkv, mp4, webm, etc., format is irrelevant), with 32 MB file in my case it just blinks too fast to click anything.

[ghisler(Author)]

I can confirm the problem when playing a video (or large audio track) with the internal media player. This even happens in 64-bit too.
It has a completely different reason than the nug I already fixed...

[Usher]

2ghisler(Author)

23.04.21 Fixed: Lister: Press '5' while the new media player was shown -> conversion to html was done twice even when not aborted (32/64)

Confirm fixed in TC 10b8.