This forum uses cookies. Click X button to hide this message. What is stored? 
Total Commander Forum Index Total Commander
Forum - Public Discussion and Support
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Integrate NTFS Streams handling into TC
Goto page 1, 2  Next
 
Post new topic   Reply to topic    Total Commander Forum Index -> TC suggestions (English) Printable version
View previous topic :: View next topic  
Author Message
X-Byte
Member
Member


Joined: 29 Nov 2004
Posts: 171
Location: Hessen, Germany

PostPosted: Mon May 21, 2012 5:10 am    Post subject: Integrate NTFS Streams handling into TC Reply with quote

While not noticed by most, the Alternate Data Streams feature of the NTFS file system is an essential feature, representing different data streams combined into one filename.
Most users take note of that when they try to execute downloaded executable files from the internet, getting a security popup caused by the existence of an alternate data stream added to the downloaded file, defining the security zone.

There's a filesystem plugin NTFS Stream 1.0.1.59 which gives basic access to these streams.

Unfortunately it's not being developed anymore since 2009, hence only available in 32-Bit.

So I'd like to ask you, Christian Ghisler, to integrate the handling of NTFS Alternate Data Streams into a future release of Total Commander.
Back to top
View user's profile Send private message
samot
Junior Member
Junior Member


Joined: 21 Feb 2003
Posts: 55
Location: Koblenz, Germany

PostPosted: Thu Jul 26, 2012 2:10 am    Post subject: Reply with quote

A little bit sample code to create / use stream files by yourself:

Code:

rem Create streams:
Echo This is stream1 > myfile.dat:stream1
Echo This is stream2 > myfile.dat:stream2

rem Show streams:
More < myfile.dat:stream1
More < myfile.dat:stream2

rem Shows file size = 0
dir myfile.dat

rem Shows the file with size 0 plus the streams with their size (19)
dir /r myfile.dat



    - Streams may be copied / renamed / moved as usual files but must not be edited.
    - Streams may be added to existing files
    - The most tools will show the file size as 0 (or the original size if added to an existing file)

Back to top
View user's profile Send private message Visit poster's website
Horst.Epp
Power Member
Power Member


Joined: 06 Feb 2003
Posts: 2584
Location: Germany

PostPosted: Thu Jul 26, 2012 2:35 am    Post subject: Re: Integrate NTFS Streams handling into TC Reply with quote

X-Byte wrote:
While not noticed by most, the Alternate Data Streams feature of the NTFS file system is an essential feature, representing different data streams combined into one filename.
Most users take note of that when they try to execute downloaded executable files from the internet, getting a security popup caused by the existence of an alternate data stream added to the downloaded file, defining the security zone.

There's a filesystem plugin NTFS Stream 1.0.1.59 which gives basic access to these streams.

Unfortunately it's not being developed anymore since 2009, hence only available in 32-Bit.

So I'd like to ask you, Christian Ghisler, to integrate the handling of NTFS Alternate Data Streams into a future release of Total Commander.

Starting with Windows 8 NTFS Streams are no longer available !
Back to top
View user's profile Send private message
umbra
Power Member
Power Member


Joined: 14 Jan 2012
Posts: 760

PostPosted: Thu Jul 26, 2012 3:10 am    Post subject: Reply with quote

2Horst.Epp
Not true. ReFS does not support NTFS streams, that's true. But it's still in beta stage - it won't be even available in client versions of Windows 8 and it MIGHT replace NTFS as a default file system in Windows 10 (or whatever it will be called).
NTFS and all its (useful?) features is not going away, at least for a while.
_________________
Windows 7 Pro x64, Windows 8.1 Pro x64, Windows 10 Pro x64
Back to top
View user's profile Send private message Send e-mail
Soumyanon
Junior Member
Junior Member


Joined: 19 Jan 2009
Posts: 2

PostPosted: Fri Aug 03, 2012 2:46 am    Post subject: Re: Integrate NTFS Streams handling into TC Reply with quote

Horst.Epp wrote:
Starting with Windows 8 NTFS Streams are no longer available !

I seriously doubt that. Microsoft is using this feature as well. For example, if you download a file from the internet and place it *anywhere* on your NTFS disk, you'll get an 'unsafe content' warning from Windows as soon as you attempt to open the file. You can move the file all around your NTFS-disks and you still keep that message. (Unless you check the "I trust this content" checkbox before dismissing the warning ofcourse).

How does Windows know that you downloaded this file from the internet? Even after an endless stream of move and copy actions? Simply because Internet Explorer attaches an Alternate Stream containing the zone-information to the file itself.

So I fully agree with X-Byte and like to see ADS-support included. Although I can understand why it's not in. As far as I know there is no documented way to get a list with all the files with their streams at once. You really have to open a file/directory and check it's streams. For 10 or so entries you won't notice any difference. However, you can think what impact it would have on a directory filled with several thousand files and directories. But then, TC only shows the directorysizes on request as well, so ADS-support could also be implemented the same way.

@Christian: Some hints to implement it,
Kernel32.DLL contains the functions needed:

  • GetVolumeInformation that provides you with the flags to determine whether the volume supports ADS. Flag 0x00040000 (FILE_NAMED_STREAMS) has to be set.
  • CreateFile / Deletefile those regular functions accept ADS-names with a little or no configuration. The first one returns a safehandle that can be used as a simple FileStream in C#. To access the ADS of a directory, the flagsAndAttributes parameter should be set to 0x02000000 (FILE_FLAG_BACKUP_SEMANTICS).
  • Backup Read / Backup Seek can be used to find the ADS-streams attached to a certain file.
  • FindFirstStreamW / FindNextStreamW As of Windows Vista and Win2K3 those functions can also be used to find ADS-streams.

If you want to implement ADS-support and need any help with those functions, please let me know and I'm happy to assist.
Back to top
View user's profile Send private message
X-Byte
Member
Member


Joined: 29 Nov 2004
Posts: 171
Location: Hessen, Germany

PostPosted: Fri Jan 18, 2013 3:28 am    Post subject: Reply with quote

@ghisler(Author)
could you please give a statement regarding this suggestion?[/b]
Back to top
View user's profile Send private message
ghisler(Author)
Site Admin
Site Admin


Joined: 04 Feb 2003
Posts: 34301
Location: Switzerland

PostPosted: Mon Jan 21, 2013 8:44 am    Post subject: Reply with quote

I'm not quite sure what you expect from me. I don't think that it would make sense to show ADS together with files in normal file lists. This would require calling a loop of FindFirstStreamW/FindNextStreamW for each file, which would be terribly slow with many files. A packer plugin (for Ctrl+PageDown) would be a possibility, but would block access to any other packer plugins for all files. A property sheet extension (Alt+Enter) would problably be the best solution, but that wouldn't have to be done by myself.
_________________
Author of Total Commander
http://www.ghisler.com
Back to top
View user's profile Send private message Send e-mail Visit poster's website
MVV
Power Member
Power Member


Joined: 03 Aug 2008
Posts: 7861
Location: Russian Federation

PostPosted: Mon Jan 21, 2013 8:52 am    Post subject: Reply with quote

ghisler,
It would be great to have chance to choose which packer plugin to use for current archive (some custom command with parameter allowing to speclfy packer name/extension in em-command). It would solve plugins access problem. E.g. one could create em-command for ADS packer plugin (or for PE plugin, for AVI frames plugin or some else) and assign a hotkey for it.
_________________
TCFS2 + TCFS2Tools: Full-screen mode for TC etc (forum)
TOTALCMD.NET: AskParam, CopyTree, NTLinks, Sudo, VirtualPanel…
Back to top
View user's profile Send private message Send e-mail
X-Byte
Member
Member


Joined: 29 Nov 2004
Posts: 171
Location: Hessen, Germany

PostPosted: Mon Jan 21, 2013 9:12 am    Post subject: Reply with quote

@ghisler(Author)
The first step would be that TC is aware of multiple streams in a file. You could integrate this into the internal tc content plugin and offer stream counts, stream names and stream sizes per file.

A packer plugin would be the most convenient solution, as it handles all the possibles cases for ntfs streams.
Like MVV and others suggested before, it would be neccessary to have a choice, which packer plugin to use. Either in a complicated way, by using separate commands to use a specific plugin or the convenient way, by offering a popup window with matching packer plugins for the currently selected file when entering it (in case more than one plugin matches)

A Filesystem Plugin would be an alternative, which handles streams as directories, handling wouldn't be as intuitive though.
Back to top
View user's profile Send private message
tbeu
Power Member
Power Member


Joined: 04 Jul 2003
Posts: 1193
Location: Germany

PostPosted: Mon Jan 21, 2013 10:02 am    Post subject: Re: Integrate NTFS Streams handling into TC Reply with quote

X-Byte wrote:
There's a filesystem plugin NTFS Stream 1.0.1.59 which gives basic access to these streams.
You mixed it up. There is a WFX NTFS FileStreams 1.0 and a WCX/WDX NTFS Stream 1.0.1.59.
_________________
TC plugins: Autodesk 3ds Max Preview, Blat Mailer, ImageMetaData (JPG Comment/EXIF/IPTC/XMP) , MATLAB MAT-file Viewer, SolidWorks Preview and more
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Lefteous
Power Member
Power Member


Joined: 08 Feb 2003
Posts: 9381
Location: Germany

PostPosted: Tue Jan 22, 2013 12:00 pm    Post subject: Reply with quote

I think it would be great to have a function in build-in Lister which let the user toggle between the stream contents (if any).
Back to top
View user's profile Send private message Send e-mail Visit poster's website
blind12
Junior Member
Junior Member


Joined: 22 Apr 2004
Posts: 20

PostPosted: Sun Oct 27, 2013 8:21 pm    Post subject: Reply with quote

Alternate streams would be a great place to hold file comments.

And maybe other relevant stuff.

Naturally, warnings when copying to non-ADS filesystem are of order.

I have no idea why MS itself never really used or supported this, or almost any other NTFS feature.
Back to top
View user's profile Send private message
meisl
Member
Member


Joined: 17 Dec 2013
Posts: 171

PostPosted: Thu Dec 19, 2013 6:36 pm    Post subject: Reply with quote

Quote:
Alternate streams would be a great place to hold file comments.

In fact that's what Windows (XP+) is actually doing, I think.

What you see in Properties | Comments is the contents of an ADS of name "\5SummaryInformation" where the initial "\5" is to mean one byte of value 0x05. This makes it a bit hard to get to it since entering this value in cmd.exe or in TC via SHIFT+F4 seems impossible.

Some resources for investigating:
- http://www.heysoft.de/de/software/lads.php
- http://technet.microsoft.com/de-de/sysinternals/bb897440
- http://www.totalcmd.net/plugring/NTFS_diz.html
- http://www.totalcmd.net/plugring/ntfs_stream.html

I, for one, totally agree with Lefteous insofar as I'd *love* to have some lister view (F3 or CTRL-q or similar) where there are tabs named after the present ADSs and at best each providing all the stuff that exists for normal content, ie normal text, hex view, plugins view, what have you...
Back to top
View user's profile Send private message Send e-mail
white
Power Member
Power Member


Joined: 19 Nov 2003
Posts: 2020
Location: Netherlands

PostPosted: Fri Dec 20, 2013 4:55 am    Post subject: Reply with quote

meisl wrote:
.. where the initial "\5" is to mean one byte of value 0x05. This makes it a bit hard to get to it since entering this value in cmd.exe .. seems impossible.

Ctrl+E or Alt+5(numeric keypad).
Back to top
View user's profile Send private message Send e-mail
meisl
Member
Member


Joined: 17 Dec 2013
Posts: 171

PostPosted: Fri Dec 20, 2013 7:01 am    Post subject: Reply with quote

Yea, works in cmd.exe, thanks.
Example: more < test.txt:^ESummaryInformation
(^E being Ctrl-E or Alt+5; must use 'more' as 'type' doesn't understand the ADS syntax)

Unfortunately it does not work with Shift-F4 in TC (append colon ":" and stream name in upcoming dialog). There Alt+5 does produce a strange box symbol but this somehow gets replaced by a "?". The result is a new stream being created with the name "?SummaryInformation"

Btw, Tir Na Nog's "NTFS Stream" wcx/wdx plugin is pretty close to ideal. Being a wcx plugin it provides a list of the streams via Ctrl-PgDn.
Except that a) editing via F4 is not supported and b) those strange characters prohibit viewing because TC tries to write a temp file with them in the name - and fails Sad
EDIT: It does allow deletion of selected streams, even with mangled names.
Back to top
View user's profile Send private message Send e-mail
Display posts from previous:   
Post new topic   Reply to topic    Total Commander Forum Index -> TC suggestions (English) All times are GMT - 6 Hours
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Impressum: This site is maintained by Ghisler Software GmbH

Using phpBB © 2001-2005 phpBB Group