Integrate NTFS Streams handling into TC

Here you can propose new features, make suggestions etc.

Moderators: white, Hacker, petermad, Stefan2

User avatar
X-Byte
Member
Member
Posts: 173
Joined: 2004-11-29, 09:34 UTC
Location: Hessen, Germany

Integrate NTFS Streams handling into TC

Post by *X-Byte »

While not noticed by most, the Alternate Data Streams feature of the NTFS file system is an essential feature, representing different data streams combined into one filename.
Most users take note of that when they try to execute downloaded executable files from the internet, getting a security popup caused by the existence of an alternate data stream added to the downloaded file, defining the security zone.

There's a filesystem plugin NTFS Stream 1.0.1.59 which gives basic access to these streams.

Unfortunately it's not being developed anymore since 2009, hence only available in 32-Bit.

So I'd like to ask you, Christian Ghisler, to integrate the handling of NTFS Alternate Data Streams into a future release of Total Commander.
samot
Junior Member
Junior Member
Posts: 56
Joined: 2003-02-21, 11:53 UTC
Location: Koblenz, Germany
Contact:

Post by *samot »

A little bit sample code to create / use stream files by yourself:

Code: Select all

rem Create streams:
Echo This is stream1 > myfile.dat:stream1 
Echo This is stream2 > myfile.dat:stream2 

rem Show streams:
More < myfile.dat:stream1
More < myfile.dat:stream2

rem Shows file size = 0
dir myfile.dat

rem Shows the file with size 0 plus the streams with their size (19)
dir /r myfile.dat
  • - Streams may be copied / renamed / moved as usual files but must not be edited.
    - Streams may be added to existing files
    - The most tools will show the file size as 0 (or the original size if added to an existing file)
User avatar
Horst.Epp
Power Member
Power Member
Posts: 6449
Joined: 2003-02-06, 17:36 UTC
Location: Germany

Re: Integrate NTFS Streams handling into TC

Post by *Horst.Epp »

X-Byte wrote:While not noticed by most, the Alternate Data Streams feature of the NTFS file system is an essential feature, representing different data streams combined into one filename.
Most users take note of that when they try to execute downloaded executable files from the internet, getting a security popup caused by the existence of an alternate data stream added to the downloaded file, defining the security zone.

There's a filesystem plugin NTFS Stream 1.0.1.59 which gives basic access to these streams.

Unfortunately it's not being developed anymore since 2009, hence only available in 32-Bit.

So I'd like to ask you, Christian Ghisler, to integrate the handling of NTFS Alternate Data Streams into a future release of Total Commander.
Starting with Windows 8 NTFS Streams are no longer available !
umbra
Power Member
Power Member
Posts: 871
Joined: 2012-01-14, 20:41 UTC

Post by *umbra »

2Horst.Epp
Not true. ReFS does not support NTFS streams, that's true. But it's still in beta stage - it won't be even available in client versions of Windows 8 and it MIGHT replace NTFS as a default file system in Windows 10 (or whatever it will be called).
NTFS and all its (useful?) features is not going away, at least for a while.
Windows 7 Pro x64, Windows 10 Pro x64
Soumyanon
Junior Member
Junior Member
Posts: 2
Joined: 2009-01-19, 22:19 UTC

Re: Integrate NTFS Streams handling into TC

Post by *Soumyanon »

Horst.Epp wrote:Starting with Windows 8 NTFS Streams are no longer available !
I seriously doubt that. Microsoft is using this feature as well. For example, if you download a file from the internet and place it *anywhere* on your NTFS disk, you'll get an 'unsafe content' warning from Windows as soon as you attempt to open the file. You can move the file all around your NTFS-disks and you still keep that message. (Unless you check the "I trust this content" checkbox before dismissing the warning ofcourse).

How does Windows know that you downloaded this file from the internet? Even after an endless stream of move and copy actions? Simply because Internet Explorer attaches an Alternate Stream containing the zone-information to the file itself.

So I fully agree with X-Byte and like to see ADS-support included. Although I can understand why it's not in. As far as I know there is no documented way to get a list with all the files with their streams at once. You really have to open a file/directory and check it's streams. For 10 or so entries you won't notice any difference. However, you can think what impact it would have on a directory filled with several thousand files and directories. But then, TC only shows the directorysizes on request as well, so ADS-support could also be implemented the same way.

@Christian: Some hints to implement it,
Kernel32.DLL contains the functions needed:
  • GetVolumeInformation that provides you with the flags to determine whether the volume supports ADS. Flag 0x00040000 (FILE_NAMED_STREAMS) has to be set.
  • CreateFile / Deletefile those regular functions accept ADS-names with a little or no configuration. The first one returns a safehandle that can be used as a simple FileStream in C#. To access the ADS of a directory, the flagsAndAttributes parameter should be set to 0x02000000 (FILE_FLAG_BACKUP_SEMANTICS).
  • Backup Read / Backup Seek can be used to find the ADS-streams attached to a certain file.
  • FindFirstStreamW / FindNextStreamW As of Windows Vista and Win2K3 those functions can also be used to find ADS-streams.
If you want to implement ADS-support and need any help with those functions, please let me know and I'm happy to assist.
User avatar
X-Byte
Member
Member
Posts: 173
Joined: 2004-11-29, 09:34 UTC
Location: Hessen, Germany

Post by *X-Byte »

@ghisler(Author)
could you please give a statement regarding this suggestion?[/b]
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48021
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) »

I'm not quite sure what you expect from me. I don't think that it would make sense to show ADS together with files in normal file lists. This would require calling a loop of FindFirstStreamW/FindNextStreamW for each file, which would be terribly slow with many files. A packer plugin (for Ctrl+PageDown) would be a possibility, but would block access to any other packer plugins for all files. A property sheet extension (Alt+Enter) would problably be the best solution, but that wouldn't have to be done by myself.
Author of Total Commander
https://www.ghisler.com
User avatar
MVV
Power Member
Power Member
Posts: 8702
Joined: 2008-08-03, 12:51 UTC
Location: Russian Federation

Post by *MVV »

ghisler,
It would be great to have chance to choose which packer plugin to use for current archive (some custom command with parameter allowing to speclfy packer name/extension in em-command). It would solve plugins access problem. E.g. one could create em-command for ADS packer plugin (or for PE plugin, for AVI frames plugin or some else) and assign a hotkey for it.
User avatar
X-Byte
Member
Member
Posts: 173
Joined: 2004-11-29, 09:34 UTC
Location: Hessen, Germany

Post by *X-Byte »

@ghisler(Author)
The first step would be that TC is aware of multiple streams in a file. You could integrate this into the internal tc content plugin and offer stream counts, stream names and stream sizes per file.

A packer plugin would be the most convenient solution, as it handles all the possibles cases for ntfs streams.
Like MVV and others suggested before, it would be neccessary to have a choice, which packer plugin to use. Either in a complicated way, by using separate commands to use a specific plugin or the convenient way, by offering a popup window with matching packer plugins for the currently selected file when entering it (in case more than one plugin matches)

A Filesystem Plugin would be an alternative, which handles streams as directories, handling wouldn't be as intuitive though.
User avatar
tbeu
Power Member
Power Member
Posts: 1336
Joined: 2003-07-04, 07:52 UTC
Location: Germany
Contact:

Re: Integrate NTFS Streams handling into TC

Post by *tbeu »

X-Byte wrote:There's a filesystem plugin NTFS Stream 1.0.1.59 which gives basic access to these streams.
You mixed it up. There is a WFX NTFS FileStreams 1.0 and a WCX/WDX NTFS Stream 1.0.1.59.
TC plugins: Autodesk 3ds Max / Inventor / Revit Preview, FileInDir, ImageMetaData (JPG Comment/EXIF/IPTC/XMP), MATLAB MAT-file Viewer, Mover, SetFolderDate, Solid Edge Preview, Zip2Zero and more
User avatar
Lefteous
Power Member
Power Member
Posts: 9535
Joined: 2003-02-09, 01:18 UTC
Location: Germany
Contact:

Post by *Lefteous »

I think it would be great to have a function in build-in Lister which let the user toggle between the stream contents (if any).
blind12
Junior Member
Junior Member
Posts: 20
Joined: 2004-04-22, 17:06 UTC

Post by *blind12 »

Alternate streams would be a great place to hold file comments.

And maybe other relevant stuff.

Naturally, warnings when copying to non-ADS filesystem are of order.

I have no idea why MS itself never really used or supported this, or almost any other NTFS feature.
meisl
Member
Member
Posts: 171
Joined: 2013-12-17, 15:30 UTC

Post by *meisl »

Alternate streams would be a great place to hold file comments.
In fact that's what Windows (XP+) is actually doing, I think.

What you see in Properties | Comments is the contents of an ADS of name "\5SummaryInformation" where the initial "\5" is to mean one byte of value 0x05. This makes it a bit hard to get to it since entering this value in cmd.exe or in TC via SHIFT+F4 seems impossible.

Some resources for investigating:
- http://www.heysoft.de/de/software/lads.php
- http://technet.microsoft.com/de-de/sysinternals/bb897440
- http://www.totalcmd.net/plugring/NTFS_diz.html
- http://www.totalcmd.net/plugring/ntfs_stream.html

I, for one, totally agree with Lefteous insofar as I'd *love* to have some lister view (F3 or CTRL-q or similar) where there are tabs named after the present ADSs and at best each providing all the stuff that exists for normal content, ie normal text, hex view, plugins view, what have you...
User avatar
white
Power Member
Power Member
Posts: 4593
Joined: 2003-11-19, 08:16 UTC
Location: Netherlands

Post by *white »

meisl wrote:.. where the initial "\5" is to mean one byte of value 0x05. This makes it a bit hard to get to it since entering this value in cmd.exe .. seems impossible.
Ctrl+E or Alt+5(numeric keypad).
meisl
Member
Member
Posts: 171
Joined: 2013-12-17, 15:30 UTC

Post by *meisl »

Yea, works in cmd.exe, thanks.
Example: more < test.txt:^ESummaryInformation
(^E being Ctrl-E or Alt+5; must use 'more' as 'type' doesn't understand the ADS syntax)

Unfortunately it does not work with Shift-F4 in TC (append colon ":" and stream name in upcoming dialog). There Alt+5 does produce a strange box symbol but this somehow gets replaced by a "?". The result is a new stream being created with the name "?SummaryInformation"

Btw, Tir Na Nog's "NTFS Stream" wcx/wdx plugin is pretty close to ideal. Being a wcx plugin it provides a list of the streams via Ctrl-PgDn.
Except that a) editing via F4 is not supported and b) those strange characters prohibit viewing because TC tries to write a temp file with them in the name - and fails :(
EDIT: It does allow deletion of selected streams, even with mangled names.
Post Reply