Integrate NTFS Streams handling into TC

[X-Byte]

While not noticed by most, the Alternate Data Streams feature of the NTFS file system is an essential feature, representing different data streams combined into one filename.
Most users take note of that when they try to execute downloaded executable files from the internet, getting a security popup caused by the existence of an alternate data stream added to the downloaded file, defining the security zone.

There's a filesystem plugin NTFS Stream 1.0.1.59 which gives basic access to these streams.

Unfortunately it's not being developed anymore since 2009, hence only available in 32-Bit.

So I'd like to ask you, Christian Ghisler, to integrate the handling of NTFS Alternate Data Streams into a future release of Total Commander.

[samot]

A little bit sample code to create / use stream files by yourself:

Code: Select all

rem Create streams:
Echo This is stream1 > myfile.dat:stream1 
Echo This is stream2 > myfile.dat:stream2 

rem Show streams:
More < myfile.dat:stream1
More < myfile.dat:stream2

rem Shows file size = 0
dir myfile.dat

rem Shows the file with size 0 plus the streams with their size (19)
dir /r myfile.dat

• - Streams may be copied / renamed / moved as usual files but must not be edited.
  - Streams may be added to existing files
  - The most tools will show the file size as 0 (or the original size if added to an existing file)

[Horst.Epp]

While not noticed by most, the Alternate Data Streams feature of the NTFS file system is an essential feature, representing different data streams combined into one filename.
Most users take note of that when they try to execute downloaded executable files from the internet, getting a security popup caused by the existence of an alternate data stream added to the downloaded file, defining the security zone.

There's a filesystem plugin NTFS Stream 1.0.1.59 which gives basic access to these streams.

Unfortunately it's not being developed anymore since 2009, hence only available in 32-Bit.

So I'd like to ask you, Christian Ghisler, to integrate the handling of NTFS Alternate Data Streams into a future release of Total Commander.

Starting with Windows 8 NTFS Streams are no longer available !

[umbra]

2Horst.Epp
Not true. ReFS does not support NTFS streams, that's true. But it's still in beta stage - it won't be even available in client versions of Windows 8 and it MIGHT replace NTFS as a default file system in Windows 10 (or whatever it will be called).
NTFS and all its (useful?) features is not going away, at least for a while.

[Soumyanon]

Starting with Windows 8 NTFS Streams are no longer available !

I seriously doubt that. Microsoft is using this feature as well. For example, if you download a file from the internet and place it *anywhere* on your NTFS disk, you'll get an 'unsafe content' warning from Windows as soon as you attempt to open the file. You can move the file all around your NTFS-disks and you still keep that message. (Unless you check the "I trust this content" checkbox before dismissing the warning ofcourse).

How does Windows know that you downloaded this file from the internet? Even after an endless stream of move and copy actions? Simply because Internet Explorer attaches an Alternate Stream containing the zone-information to the file itself.

So I fully agree with X-Byte and like to see ADS-support included. Although I can understand why it's not in. As far as I know there is no documented way to get a list with all the files with their streams at once. You really have to open a file/directory and check it's streams. For 10 or so entries you won't notice any difference. However, you can think what impact it would have on a directory filled with several thousand files and directories. But then, TC only shows the directorysizes on request as well, so ADS-support could also be implemented the same way.

@Christian: Some hints to implement it,
Kernel32.DLL contains the functions needed:

• GetVolumeInformation that provides you with the flags to determine whether the volume supports ADS. Flag 0x00040000 (FILE_NAMED_STREAMS) has to be set.
• CreateFile / Deletefile those regular functions accept ADS-names with a little or no configuration. The first one returns a safehandle that can be used as a simple FileStream in C#. To access the ADS of a directory, the flagsAndAttributes parameter should be set to 0x02000000 (FILE_FLAG_BACKUP_SEMANTICS).
• Backup Read / Backup Seek can be used to find the ADS-streams attached to a certain file.
• FindFirstStreamW / FindNextStreamW As of Windows Vista and Win2K3 those functions can also be used to find ADS-streams.

If you want to implement ADS-support and need any help with those functions, please let me know and I'm happy to assist.

[X-Byte]

@ghisler(Author)
could you please give a statement regarding this suggestion?[/b]

[ghisler(Author)]

I'm not quite sure what you expect from me. I don't think that it would make sense to show ADS together with files in normal file lists. This would require calling a loop of FindFirstStreamW/FindNextStreamW for each file, which would be terribly slow with many files. A packer plugin (for Ctrl+PageDown) would be a possibility, but would block access to any other packer plugins for all files. A property sheet extension (Alt+Enter) would problably be the best solution, but that wouldn't have to be done by myself.

[MVV]

ghisler,
It would be great to have chance to choose which packer plugin to use for current archive (some custom command with parameter allowing to speclfy packer name/extension in em-command). It would solve plugins access problem. E.g. one could create em-command for ADS packer plugin (or for PE plugin, for AVI frames plugin or some else) and assign a hotkey for it.

[X-Byte]

@ghisler(Author)
The first step would be that TC is aware of multiple streams in a file. You could integrate this into the internal tc content plugin and offer stream counts, stream names and stream sizes per file.

A packer plugin would be the most convenient solution, as it handles all the possibles cases for ntfs streams.
Like MVV and others suggested before, it would be neccessary to have a choice, which packer plugin to use. Either in a complicated way, by using separate commands to use a specific plugin or the convenient way, by offering a popup window with matching packer plugins for the currently selected file when entering it (in case more than one plugin matches)

A Filesystem Plugin would be an alternative, which handles streams as directories, handling wouldn't be as intuitive though.

[tbeu]

There's a filesystem plugin NTFS Stream 1.0.1.59 which gives basic access to these streams.

You mixed it up. There is a WFX NTFS FileStreams 1.0 and a WCX/WDX NTFS Stream 1.0.1.59.

[Lefteous]

I think it would be great to have a function in build-in Lister which let the user toggle between the stream contents (if any).

[blind12]

Alternate streams would be a great place to hold file comments.

And maybe other relevant stuff.

Naturally, warnings when copying to non-ADS filesystem are of order.

I have no idea why MS itself never really used or supported this, or almost any other NTFS feature.

[meisl]

Alternate streams would be a great place to hold file comments.

In fact that's what Windows (XP+) is actually doing, I think.

What you see in Properties | Comments is the contents of an ADS of name "\5SummaryInformation" where the initial "\5" is to mean one byte of value 0x05. This makes it a bit hard to get to it since entering this value in cmd.exe or in TC via SHIFT+F4 seems impossible.

Some resources for investigating:
- http://www.heysoft.de/de/software/lads.php
- http://technet.microsoft.com/de-de/sysinternals/bb897440
- http://www.totalcmd.net/plugring/NTFS_diz.html
- http://www.totalcmd.net/plugring/ntfs_stream.html

I, for one, totally agree with Lefteous insofar as I'd *love* to have some lister view (F3 or CTRL-q or similar) where there are tabs named after the present ADSs and at best each providing all the stuff that exists for normal content, ie normal text, hex view, plugins view, what have you...

[white]

.. where the initial "\5" is to mean one byte of value 0x05. This makes it a bit hard to get to it since entering this value in cmd.exe .. seems impossible.

Ctrl+E or Alt+5(numeric keypad).

[meisl]

Yea, works in cmd.exe, thanks.
Example: more < test.txt:^ESummaryInformation
(^E being Ctrl-E or Alt+5; must use 'more' as 'type' doesn't understand the ADS syntax)

Unfortunately it does not work with Shift-F4 in TC (append colon ":" and stream name in upcoming dialog). There Alt+5 does produce a strange box symbol but this somehow gets replaced by a "?". The result is a new stream being created with the name "?SummaryInformation"

Btw, Tir Na Nog's "NTFS Stream" wcx/wdx plugin is pretty close to ideal. Being a wcx plugin it provides a list of the streams via Ctrl-PgDn.
Except that a) editing via F4 is not supported and b) those strange characters prohibit viewing because TC tries to write a temp file with them in the name - and fails
EDIT: It does allow deletion of selected streams, even with mangled names.