jsontab: plugin to view Json files

Discuss and announce Total Commander plugins, addons and other useful tools here, both their usage and their development.

Moderators: white, Hacker, petermad, Stefan2

User avatar
white
Power Member
Power Member
Posts: 4593
Joined: 2003-11-19, 08:16 UTC
Location: Netherlands

Re: jsontab: plugin to view Json files

Post by *white »

2little-brother
Can you add support for faulty files containing multiple roots? Assume they are an array and display them in <<fakeroot>> instead of <<root>>? Perhaps optional in .ini file?
little-brother
Junior Member
Junior Member
Posts: 43
Joined: 2021-09-27, 10:27 UTC

Re: jsontab: plugin to view Json files

Post by *little-brother »

The json library (parson) can't do it directly. So I should write own splitter to implement it. I'm afraid I'm not ready to do it.
User avatar
white
Power Member
Power Member
Posts: 4593
Joined: 2003-11-19, 08:16 UTC
Location: Netherlands

Re: jsontab: plugin to view Json files

Post by *white »

white wrote: 2022-03-14, 12:22 UTC 2little-brother
Can you add support for faulty files containing multiple roots? Assume they are an array and display them in <<fakeroot>> instead of <<root>>? Perhaps optional in .ini file?
little-brother wrote: 2022-03-14, 18:11 UTC The json library (parson) can't do it directly. So I should write own splitter to implement it. I'm afraid I'm not ready to do it.
The problem is:
If there is an error during parsing, the plugin fails and the file is shown in normal view. This is fine. But when there is an error after the root element, the root element is shown and all other information is simply ignored. Users viewing the file may think there isn't more information in the file and decide to delete the file. While in fact there could be lots of important information in there, for example in additional root elements.

I wrote a fix for the parson library adding one function and changing two lines. It then can handle multiple root elements in a file which will be put in a fake root array. If there is any other error anywhere in the file, the parsing fails and nothing is returned. So your plugin would no longer display the (first) root element if there is an error following the root element.

You may want to use the latest version of the parson library. It includes minor improvements, for example allowing a comma before the closing bracket of an object.

Here is how you can apply my fix to the parson library:
  • Add the declaration of the function "parse_roots" to the declaration of functions listed under /* Parser */

    Code: Select all

    /* Parser */
    static JSON_Value *  parse_roots(const char **string);
    
  • Add the function "parse_roots" to the functions listed under /* Parser */

    Code: Select all

    /* Parser */
    /* Parse one or more root elements. If there are more then one (invalid json),
       create a fake root array to store them. */
    static JSON_Value * parse_roots(const char **string) {
        JSON_Value *first_root = NULL, *next_root = NULL, *fake_root = NULL;
        JSON_Array *fake_root_array = NULL;
        first_root = parse_value(string, 0);
        if (first_root == NULL) {
            return NULL;
        }
        SKIP_WHITESPACES(string);
        /* assume null character is end of string, null character between root elements not supported */
        if (**string == '\0') { /* only one root element found, we are finished */
            return first_root;
        }
        fake_root = json_value_init_array(); /* create fake root array to hold multiple root elements */
        if (fake_root == NULL) {
            json_value_free(first_root);
            return NULL;
        }
        fake_root_array = json_value_get_array(fake_root);
        /* add info to fake root */
        if (json_array_append_string(fake_root_array, "Multiple roots from invalid JSON file!") != JSONSuccess) {
            json_value_free(first_root);
            json_value_free(fake_root);
            return NULL;
        }
        if (json_array_add(fake_root_array, first_root) != JSONSuccess) { /* add first root element */
            json_value_free(first_root);
            json_value_free(fake_root);
            return NULL;
        }
        while (strchr("{[\"ft-0123456789n",**string) != NULL ) { /* next root element */
            next_root = parse_value(string, 0); /* get next root element */
            if (next_root == NULL) {
                json_value_free(fake_root);
                return NULL;
            }
            if (json_array_add(fake_root_array, next_root) != JSONSuccess) {
                json_value_free(next_root);
                json_value_free(fake_root);
                return NULL;
            }
            SKIP_WHITESPACES(string);
            /* assume null character is end of string, null character between roots not supported */
            if (**string == '\0') { /* last root element added, we are finished */
                return fake_root;
            }
        }
        json_value_free(fake_root);
        return NULL;
    }
    
  • Change function "json_parse_string" so it calls "parse_roots" instead of "parse_value".

    Code: Select all

    //    return parse_value((const char**)&string, 0);
        return parse_roots((const char**)&string); /* parse one or more root elements */
    
  • Change function "json_parse_string_with_comments" so it calls "parse_roots" instead of "parse_value".

    Code: Select all

    //    result = parse_value((const char**)&string_mutable_copy_ptr, 0);
        result = parse_roots((const char**)&string_mutable_copy_ptr); /* parse one or more root elements */
    
No changes needed for your plugin source code.
little-brother
Junior Member
Junior Member
Posts: 43
Joined: 2021-09-27, 10:27 UTC

Re: jsontab: plugin to view Json files

Post by *little-brother »

Thanks, I'll add it soon.
Perhaps, it can be usefull for an author of parson too.
User avatar
white
Power Member
Power Member
Posts: 4593
Joined: 2003-11-19, 08:16 UTC
Location: Netherlands

Re: jsontab: plugin to view Json files

Post by *white »

2little-brother
I see your last update did use the latest parson library already, so you can ignore my remark about that ;)
little-brother wrote: 2022-03-17, 12:43 UTC Thanks, I'll add it soon.
Thanks :)
little-brother wrote: 2022-03-17, 12:43 UTC Perhaps, it can be usefull for an author of parson too.
Yes, perhaps. I will create an issue once you added it and also didn't encounter unexpected results.
little-brother
Junior Member
Junior Member
Posts: 43
Joined: 2021-09-27, 10:27 UTC

Re: jsontab: plugin to view Json files

Post by *little-brother »

for example allowing a comma before the closing bracket of an objects
It was mine request :)

Here is alpha version. I modified your code
parse_roots

Code: Select all

	JSON_Value* roots = json_value_init_array();
	
	while (**string != '\0') {
		JSON_Value *root = parse_value(string, 0);

		if (!root) {
			while (**string != '\0' && **string != '{' && **string != '[')
				SKIP_CHAR(string);
			
			continue;
		}
			
		json_array_add(json_value_get_array(roots), root);
	}

	int cnt = json_array_get_count(json_value_get_array(roots));
	if (cnt == 0) {
		json_value_free(roots);
		return NULL;
	} 
	
	if (cnt == 1) {
		JSON_Array* array = json_value_get_array(roots);
		JSON_Value* root = json_array_get_value(array, 0);
		// Replace by null and erase array
		JSON_Value* value = json_value_init_null();
		value->parent = json_array_get_wrapping_value(array);
		array->items[0] = value;
		json_value_free(roots);
		
		return root;
	}

	return roots;
It less accurate than yours but it allows to parse broken jsons like

Code: Select all

[1,2] abc [3, 4], [5,6] 
User avatar
white
Power Member
Power Member
Posts: 4593
Joined: 2003-11-19, 08:16 UTC
Location: Netherlands

Re: jsontab: plugin to view Json files

Post by *white »

little-brother wrote: 2022-03-21, 23:19 UTC
for example allowing a comma before the closing bracket of an objects
It was mine request :)
:D
little-brother wrote: 2022-03-21, 23:19 UTC Here is alpha version. I modified your code
parse_roots

Code: Select all

	JSON_Value* roots = json_value_init_array();
	
	while (**string != '\0') {
		JSON_Value *root = parse_value(string, 0);

		if (!root) {
			while (**string != '\0' && **string != '{' && **string != '[')
				SKIP_CHAR(string);
			
			continue;
		}
			
		json_array_add(json_value_get_array(roots), root);
	}

	int cnt = json_array_get_count(json_value_get_array(roots));
	if (cnt == 0) {
		json_value_free(roots);
		return NULL;
	} 
	
	if (cnt == 1) {
		JSON_Array* array = json_value_get_array(roots);
		JSON_Value* root = json_array_get_value(array, 0);
		// Replace by null and erase array
		JSON_Value* value = json_value_init_null();
		value->parent = json_array_get_wrapping_value(array);
		array->items[0] = value;
		json_value_free(roots);
		
		return root;
	}

	return roots;
It less accurate than yours but it allows to parse broken jsons like

Code: Select all

[1,2] abc [3, 4], [5,6] 
It's no good at all.
  • I complained earlier that json errors after the root were being ignored, causing parts of the contents of the file not being shown. Now, you simply ignore all errors. It's much worse.
  • I implemented handling of a specific error, namely when there are multiple roots. This is because there are programs that create such files. Mozilla Thunderbird for example, has stored my chats in json files with multiple roots. Why do you want to allow:

    Code: Select all

    [1,2] abc [3, 4], [5,6] 
    If your plan is to still show contents using your plugin for a wide range of errors, you should rewrite all the parser functions. And because your plugin is a viewer you should make sure no data is ever ignored, but always shown in some way.
  • And finally, you seem to assume that the root element can only be an object or an array, which is a common misconception. The parson library doesn't work that way. It allows all values. That's why it starts with the function parse_value.
little-brother
Junior Member
Junior Member
Posts: 43
Joined: 2021-09-27, 10:27 UTC

Re: jsontab: plugin to view Json files

Post by *little-brother »

Ok. How about a behaviour controlled by ini?
I added parse-mode: 0 (default) - strict, 1 - allows multi-root only, 2 - parse any malformed data
1 - is based on your code except an information node because I moved it to a root. Any malformed json will have <<Malformed>>-root instead of <<root>>.
2 - is my code.
And finally, you seem to assume that the root element can only be an object or an array, which is a common misconception. The parson library doesn't work that way. It allows all values. That's why it starts with the function parse_value.
I don't think that is a terrible simplification.

Code: Select all

[1, 2] 1000 [3,4] "aaa" [5, 6]
is a almost impossible json.

Here is updated version with sources.
I added ..._v2 functions to make future parson update are more simple.
Hurdet
Power Member
Power Member
Posts: 620
Joined: 2003-05-10, 18:02 UTC

Re: jsontab: plugin to view Json files

Post by *Hurdet »

Fix a crash for Win10x64/TCx32
Do you have fixed the bug that I have reported?
little-brother
Junior Member
Junior Member
Posts: 43
Joined: 2021-09-27, 10:27 UTC

Re: jsontab: plugin to view Json files

Post by *little-brother »

It was another potential issue.
At this moment I don't know how to fix your issue without loading Win7 msfsedit.dll :(
pplupo
Member
Member
Posts: 102
Joined: 2019-12-02, 16:26 UTC
Location: Canada
Contact:

Re: jsontab: plugin to view Json files

Post by *pplupo »

Hi little_brother!

I've just submitted a bug on your GitHub repo: https://github.com/little-brother/jsontab-wlx/issues/3

If I right click on the JSON tree panel, on any blank area (not on a tree node), TCM crashes and closes suddenly. Can you please take a look?

Thank you.
krasusczak
Senior Member
Senior Member
Posts: 282
Joined: 2011-09-23, 10:35 UTC

Re: jsontab: plugin to view Json files

Post by *krasusczak »

Hi little-brother,

I found some place for improvement or maybe it's bug?
The thing is that the column shows only if the value existing in the first "array" which sometimes is not a real world situation.

Here is the not working sample:

Code: Select all

[
  {
    "url": "dummy",
    "auths": [
      {
        "value": "dummy"
      }
    ],
    "contentType": "JSON",
    "title": "dummy"
  },
  {
    "url": "dummy",
    "lars": false,
    "auths": [
      {
        "value": "dummy"
      }
    ],
    "contentType": "JSON",
    "title": "dummy"
  }
]

This is working one:

Code: Select all

[
  {
    "url": "dummy",
    "lars": false,
    "auths": [
      {
        "value": "dummy"
      }
    ],
    "contentType": "JSON",
    "title": "dummy"
  },
  {
    "url": "dummy",
    "lars": false,
    "auths": [
      {
        "value": "dummy"
      }
    ],
    "contentType": "JSON",
    "title": "dummy"
  }
]

So the "lars" column is visible only if value exist in the first one, do you think is it possible to change this?
Tested 1.0.3, 32b, Win7

Best regards
User avatar
Horst.Epp
Power Member
Power Member
Posts: 6449
Joined: 2003-02-06, 17:36 UTC
Location: Germany

Re: jsontab: plugin to view Json files

Post by *Horst.Epp »

pplupo wrote: 2022-12-06, 04:36 UTC Hi little_brother!

I've just submitted a bug on your GitHub repo: https://github.com/little-brother/jsontab-wlx/issues/3

If I right click on the JSON tree panel, on any blank area (not on a tree node), TCM crashes and closes suddenly. Can you please take a look?

Thank you.
Confirmed
Windows 11 Home x64 Version 23H2 (OS Build 22631.3374)
TC 11.03 x64 / x86
Everything 1.5.0.1371a (x64), Everything Toolbar 1.3.2, Listary Pro 6.3.0.69
QAP 11.6.3.2 x64
krasusczak
Senior Member
Senior Member
Posts: 282
Joined: 2011-09-23, 10:35 UTC

Re: jsontab: plugin to view Json files

Post by *krasusczak »

Horst.Epp wrote: 2023-03-03, 15:39 UTC
pplupo wrote: 2022-12-06, 04:36 UTC Hi little_brother!

I've just submitted a bug on your GitHub repo: https://github.com/little-brother/jsontab-wlx/issues/3

If I right click on the JSON tree panel, on any blank area (not on a tree node), TCM crashes and closes suddenly. Can you please take a look?

Thank you.
Confirmed

It was fixed in 1.0.3
User avatar
AntonyD
Power Member
Power Member
Posts: 1231
Joined: 2006-11-04, 15:30 UTC
Location: Russian Federation

Re: jsontab: plugin to view Json files

Post by *AntonyD »

This is working one:
And I want to note that if you delete an element 'lars' at all in this working second example, the table view for this second element will just show an empty cell!!! And this is wrong! We need to show some kind of special substitute template. Like the word <not exist>/<not set>/<null>/????
Otherwise we might think it's just an empty string "" is specified as the value of this field.
"lars": "",
Which, of course, is wrong.
What do you think about it?
#146217 personal license
Post Reply