If a content plugin returns a (single) field of type ft_fulltextw....

[tbeu]

If a content plugin returns a (single) field of type ft_fulltextw, then in the plugin search options of TC 9.21a no operation is available on that field, see Image: https://tbeu.de/forum/wdx_ft_fulltextw64.png. Note, it works as expected for narrow full text string ft_fulltext.

[tbeu]

If this is confirmed as a bug, the content plugin interface version can be safely incremented to 2.20 since it is not yet the case for TC 9.21a. Strictly seen, currently no content plugin is able to support ft_fulltextw. See also https://www.ghisler.ch/board/viewtopic.php?f=6&t=51636.

[milo1012]

See also https://www.ghisler.ch/board/viewtopic.php?f=6&t=51636.

Did you check
https://www.ghisler.ch/board/viewtopic.php?t=17135&start=75

ft_fulltextw New in 2.11: Same as ft_fulltext, but with UTF-16 encoding. May be returned instead of ft_fulltext.

Check for version 2.11 in function ContentSetDefaultParams before returning ft_fulltextw. It should work to mix ft_fulltext and ft_fulltextw: Return ft_fulltext for the field type, and then return either ft_fulltext or ft_fulltextw for the files TC tries to search.

?
Actually I never expected the version of the wdx interface help file/manual to be identical with the interface version reported by ContentSetDefaultParams().

---

Strictly seen, currently no content plugin is able to support ft_fulltextw.

Works fine for my PCREsearch and APK-wdx plugins. I check for

Code: Select all

if((dps->PluginInterfaceVersionHi == 2 && dps->PluginInterfaceVersionLow >= 11) || (dps->PluginInterfaceVersionHi > 2))

[tbeu]

Ah, looks like a typo in the changelog then. Should read 2.11 instead of 2.20. See https://www.ghisler.ch/board/viewtopic.php?f=6&t=51636.

[tbeu]

Does PCREsearch work with the search dialog if it is ft_fulltextw? Cannot test since totalcmd.net is down.

[milo1012]

Does PCREsearch work with the search dialog if it is ft_fulltextw?

Yes, I tested it quite thorough and it works fine. You can finally search in Unicode w/o recoding issues.
I guess your bug appears if the only field returned by the plugin is ft_fulltextw, and that it's probably gone when providing an additional field for TC, but I can't test this right now.

[tbeu]

Hm, happens for all these cases:

Code: Select all

const int fieldtypes1[] = {ft_fulltextw};
const int fieldtypes2[] = {ft_fulltext, ft_fulltextw};
const int fieldtypes3[] = {ft_boolean, ft_fulltextw};
const int fieldtypes4[] = {ft_fulltextw, ft_fulltext};

[ghisler(Author)]

You should always return ft_fulltext as field type, and only when actually returning text, return ft_fulltextw.

[tbeu]

You should always return ft_fulltext as field type, and only when actually returning text, return ft_fulltextw.

Will check it, but sounds confusing, since ft_string and ft_stringw are not treated that way.

[ghisler(Author)]

The reason is that the plugin may be handling both plain text and Unicode files. By returning a field type of ft_fulltext, it tells TC to use the full text search method. The plugin can then decide when opening each individual file whether it wants to return Ansi or Unicode to TC.

[tbeu]

Thanks for clarification. ContentGetValueW works for ft_fulltextw as long as the full text is less than the buffer size. As soon, as the content plugins needs to split the buffer in chunks, it crashes with

Code: Select all

---------------------------
Total Commander 9.21a
---------------------------
Access violation at address 00703479. Write of address 006E006F.
Access violation at address 00703479. Write of address 006E006F
Windows 10 Enterprise 10.0 (Build 17134), base: 00400000

Please report this error to the Author, with a description
of what you were doing when this error occurred!

Windows exception: C0000005
Stack trace:
00703479
00437CA6  0043797E  00467408  00449EC6  0042757A  00438359
0041FECB  005DA7B5  005D5D43  00448B04  0043853F  004489CD
00448903  0044BF22  0044A9CE  >0042757A  00449F70  0042757A
0042CBD0  0042CD94  005CE665  005A7590  00590DA4  0057E9CB
0044B50B  0044A03A  0044A9CE  0042757A  00449F70  0042757A
0042CCF9  0042CD94  
Raw:
004032DA  0041F634  00437CA6  0043797E  0046001E  005C002C
0041F634  00437CA6  0043797E  00467408  004C0063  004D006C
004D006C  0046006D  00550045  004F0008  0046001E  00449EC6
0042757A  00438359  0041FECB  0046001E  00449EC6  0042757A
0044AD47  0044B104  004489CD  0044AC9D  0044AC9D  00438359
0044A03A  0044A9CE  0044A9F1  0042757A  0044A03A  0044A9CE
0044A9F1  0042757A  00449F70  0042CC46  005DB9CF  005D9EEE
005DA7B5  0046001E  0044AD2A  004489CD  0044AC9D  00437E0F
00467426  0044A03A  0044A9CE  0044A9F1  00449F70  0044A03A
0046001E  00449EC6  0042757A  00438359  00449EC6  0042757A

Press Ctrl+C to copy this report!
Continue execution?
---------------------------
Ja   Nein   
---------------------------

This is my code for wide strings:

Code: Select all

static const int fieldtypes[1] {ft_fulltext};
static std::string fullText;
static std::wstring fullTextw;

char* strlcpy(char* str1, const char* str2, int maxSize)
{
    if ((int)strlen(str2) >= maxSize - 1) {
        strncpy(str1, str2, maxSize - 1);
        str1[maxSize - 1] = 0;
    }
    else
        strcpy(str1, str2);
    return str1;
}

wchar_t* wcslcpy(wchar_t* str1, const wchar_t* str2, int maxSize)
{
    if ((int)wcslen(str2) >= maxSize - 1) {
        wcsncpy(str1, str2, maxSize - 1);
        str1[maxSize - 1] = L'\0';
    } else
        wcscpy(str1, str2);
    return str1;
}

int __stdcall ContentGetValueW(const wchar_t* FileName, int FieldIndex, int UnitIndex, void* FieldValue, int maxSize, int flags)
{
    if (FieldIndex < 1) {
        if (ft_fulltext == fieldtypes[FieldIndex]) {
            if (UnitIndex == 0) {
                // fill fullTextw here
            }
            if (fullTextw.empty()) {
                return ft_fieldempty;
            }
            if (UnitIndex == -1 || (int)fullTextw.size() <= UnitIndex) {
                fullTextw.clear();
                return ft_fieldempty;
            }
            if (UnitIndex >= 0) {
                std::wstring_view view{ fullTextw.c_str(), fullTextw.size() };
                auto block = view.substr(UnitIndex);
                wcslcpy((wchar_t*)FieldValue, block.data(), maxSize);
                return ft_fulltextw;
            }
        }
    }
    else
        return ft_nosuchfield;

    return fieldtypes[FieldIndex]; // very important!
}

Identical code on narrow string works as expected though

Code: Select all

int __stdcall ContentGetValueW(const wchar_t* FileName, int FieldIndex, int UnitIndex, void* FieldValue, int maxSize, int flags)
{
    if (FieldIndex < 1) {
        if (ft_fulltext == fieldtypes[FieldIndex]) {
            if (UnitIndex == 0) {
                // fill fullText here
            }
            if (fullText.empty()) {
                return ft_fieldempty;
            }
            if (UnitIndex == -1 || (int)fullText.size() <= UnitIndex) {
                fullText.clear();
                return ft_fieldempty;
            }
            if (UnitIndex >= 0) {
                std::string_view view{ fullText.c_str(), fullText.size() };
                auto block = view.substr(UnitIndex);
                strlcpy((char*)FieldValue, block.data(), maxSize);
                return ft_fulltext;
            }
        }
    }
    else
        return ft_nosuchfield;

    return fieldtypes[FieldIndex]; // very important!
}

Do you have an idea why TC crashes?

[ghisler(Author)]

maxSize in ContentGetValueW is in Bytes, not in characters. You are writing beyond the size of the buffer.

[tbeu]

Sure. Issue is resolved. Thank you for your support.