Is TC 7.5 public beta out?

[m^2]

m^2,

Any solution that keeps comments with files is fine

Well, what's the issue then? We already have two such formats. Is it only about Unicode?

For me it's mostly about Unicode. As we want to change it already, we can introduce other changes too, but it's not necessary IMO.

Therefore reinstall will wipe them.

Come on. Reinstall wipes all the data you don't backup. Your files, your comments, your databases...

OS partition is supposed to have OS on it and nothing more. If you store other files on it then indeed you need backup, but it's only because of your bad practices, not because it has to be (or is meant to be) this way.

Editing database in a script would be a killer

Well, can't imagine parsing a text file in AHK to be difficult.

Text file seems a very bad idea for storing many comments because it has all operations in linear time. Read: it's slow.

How would you loose ADS?
They are stored on filesystem and therefore should live exactly as long as it does (unless removed by user, obviously).

Backup does not always mean copying to another NTFS drive or making partition image, so you loose ADS while formating your drive.

That's why I said that TC should use 2 formats. I thought that it's obvious, but it seems not: And should convert them when FS changes.
If you do backup: it will work. If you don't: it will work too, all files left will keep their comments.

How is average user supposed to know that there are some necessary files to backup? There never have been any.

Such "average" user does not know about descript.ion either. Little bit more advanced ones keep their wincmd.ini + few other files safe and usually know what to copy before reinstall.

And doesn't have to. Like my proposal, it just works. And is actually more robust because scripts / external tools don't have to be aware of comment structure when moving between different FSes. .xml (or a tiny database in each directory or whatever similar) will keep this robustness.

[balver]

That's why I said that TC should use 2 formats. I thought that it's obvious, but it seems not: And should convert them when FS changes.
If you do backup: it will work. If you don't: it will work too, all files left will keep their comments.

But there is a tiny detail. Converting Unicode comment from ADS to descript.ion WILL FAIL.

[Hacker]

balver,

From my side it's all about Unicode+comments portability.

Well then it should be enough to introduce a BOM in descript.ions to denote UTF8 encoding, no?

m^2,

OS partition is supposed to have OS on it and nothing more.

Well, so if you do not consider a DB part of the OS you have nothing to fear.

Text file seems a very bad idea for storing many comments because it has all operations in linear time. Read: it's slow.

OK, well, now we have text files (descript.ion / files.bbs) scattered around the drive. An XML format (adding additional overhead) has been proposed, again scattered around the drive. I have proposed a database in one location. I do not see where this is slower than either the XML proposal or the current situation. Perhaps if you are performing many operations on the file and do not keep it in memory, but I'd say that should be avoidable.

My idea is to start going into the direction of an indexed search which I would love to see in TC 8, including WDX metadata, so you could for instance search for all your EXIF-containing JPG's taken on a specific date with ISO800 and no flash. Or all your old <128 kbps MP3s whereever they might be, to replace them with higher quality versions.
This could be done with a "scattered" "database" (like it is now with the descript.ion standard) but for speed reasons it would be better to use a common database. I assume some internal tool would allow to transfer / sync metadata back and forth between a database and descript.ion-like comments. This might be as easy as one flag in the database (valid globally / valid for current dir / valid recursively). Of course, the details would be sorted out later.

Personally I do not think that introducing a new format just for the sake of Unicode support is worth it.

Roman

[balver]

Well then it should be enough to introduce a BOM in descript.ions to denote UTF8 encoding, no?

Nope, descript.ion is a standard. It's TC independent and Christian simply won't change it.

Oh, and I see your point about DB. I even like it. But it's quite unreal unless we have super big HDD/SSD/etc. Like OneDriveForLife™ (SDXC is close to it but too far on the other hand ).

[fenix_productions]

Personally I do not think that introducing a new format just for the sake of Unicode support is worth it.Roman

TC 7.5 is one of the most requested features and seeing that some parts of it does not fit is a little bit strange. TC should stick to this approach as close as it possible. Maybe even more than to breadcrumbs or Internal Associations.

Edited

But it's quite unreal unless we have super big HDD/SSD/etc. Like OneDriveForLife™ (SDXC is close to it but too far on the other hand ).

Not unreal. There are a lot of indexing tools (i.e. from Google Desktop or Locate32) and very big drives are not needed for them, as long as information about data is lot smaller than data inself, which seems to be true in most cases IMHO.

As an example: I've got almost 300GB of data (308881 files) on my HDD and Locate32's DB takes almost 31MB only. I understand that only basic files properties are stored within but how much bigger can it be and who wants all files indexed?

[Hacker]

balver,

Nope, descript.ion is a standard.

Well, CMIIW but inside the area delimited by the application identifier each application can store whatever it wants, as long as it fits the size (perhaps text-only, I admit, though not certain). Base64 anyone?

Roman

[balver]

Just thought about one thing. We could possibly live without Unicode in comments, but there is one thing, that makes new format (or format change) absolutely necessary. How do you comment Unicode-named file? You can't. Period.

2fenix_productions
Well, I must clarify myself. I meant, that I'll accept DB in one condition. If it will fit together with all my data in one drive (at best SD-sized). Hopefuly it will come up within this decade. (Of course I wouldn't complain if it suddenly was implemented - I like new features).

2Hacker
I'm afraid I'm not able to answer that. Maybe Christian will post something about it when we wakes up. It sounds possible, but I'm not the one who read descript.ion specification letter by letter.

[van Dusen]

Well then it should be enough to introduce a BOM in descript.ions to denote UTF8 encoding, no?

Nope, descript.ion is a standard. It's TC independent and Christian simply won't change it.

Mr.Ghisler has a file ID availabe for using with TC (0xC2, as received from inventors of descript.ion), which he already applys to descript.ion comments containing line breaks. Thus, Hacker's suggestion is probably valid.

[balver]

Mr.Ghisler has a file ID availabe for using with TC (0xC2, as received from inventors of descript.ion), which he already applys to descript.ion comments containing line breaks. Thus, Hacker's suggestion is probably valid.

In that case, the issue looks almost solved. But can descript.ion comments work for Unicode-named files?

[van Dusen]

balver,

Nope, descript.ion is a standard.

Well, CMIIW but inside the area delimited by the application identifier each application can store whatever it wants, as long as it fits the size (perhaps text-only, I admit, though not certain). Base64 anyone?

Roman

You are right. But additional encoding isn't necessary, IMO.

Other program info is any text the program wishes to store in its
area of the line. The text should relate specifically to the file
named on the line. It may not contain the Ctrl-D character,
carriage returns, line feeds, or nulls (ASCII 0s).

[petermad]

But can descript.ion comments work for Unicode-named files?

Most likely NOT.

Currently in TC 7.50b1 the unicode characters in the file names are replaced by the ? wildcard when stored in the descript.ion file - so the comments are not unique, but applies to any file that fits the wildcard pattern.

[fenix_productions]

Currently in TC 7.50b1 the unicode characters in the file names are replaced by the ? wildcard when stored in the descript.ion file - so the comments are not unique, but applies to any file that fits the wildcard pattern.

You're right but this is very unpleasant behaviour. Sample file with 6 Japanese characters gaines ? six times. You're making one comment but it is applied applied to all other files with unicode names with the same length.

This is also inconsistant because setting such invalid comment requires Change attributes dialogue while other methods simply give "Unfortunately this function doesn't support Unicode characters!" message. It's definitely a bug!

I will post it later (if it's not submitted).

[Lefteous]

Not really. I still think that XML is the best, DB next, ADS last

Storing metadata inside the file is the best way.

[m^2]

That's why I said that TC should use 2 formats. I thought that it's obvious, but it seems not: And should convert them when FS changes.
If you do backup: it will work. If you don't: it will work too, all files left will keep their comments.

But there is a tiny detail. Converting Unicode comment from ADS to descript.ion WILL FAIL.

I can only repeat myself:

One would be used like descript.ion now, that is one file for each directory. File structure is less important.
The other way would be alternate data streams.

2Hacker: Short text files are fine. For large ones you need something faster. Seriously, this won't work well unless with an engine that keeps it in memory in some other form and converts to text when you quit TC. Which is insecure.

2fenix:
Indexing != commenting!
With indexes, the database can be rebuild when it's lost. Comments can't. And that's critical difference IMO. I wouldn't bother if TC used a centralized database as a cache, but storing real data there is a big no-no.
We would be having a TC's own equivalent of Windows registry. Terrible idea IMO.

2Lefteous:
What do you mean?

[Lefteous]

What do you mean?

Storing metadata inside the file the metadata belong solves all problems:
- File system independent (ADS)
- No complex copy/move handling (descript.ion)
- No tracking of copy/move/delete actions necessary (database)