TheTVDB.com content plug-in

Discuss and announce Total Commander plugins, addons and other useful tools here, both their usage and their development.

Moderators: sheep, Hacker, Stefan2, white

j
Member
Member
Posts: 123
Joined: 2009-04-13, 19:49 UTC

TheTVDB.com content plug-in

Post by *j » 2009-05-29, 18:54 UTC

This is my very first Total Commander plug-in, so any feedback is appreciated.

TheTVDB.com content plug-in for Total Commander - Version 0.7

Description
This plug-in fetches information about TV shows from TheTVDB.com, an open online TV database.
The name of the TV show, as well as the season and episode is purely determined by the (input) filename.

When the name is not unique a dialog will be shown to ask for the correct show. Pressing "Cancel" will ignore the pattern for 5 minutes.

To speed up operations (and go easy on the servers), all retrieved data is stored in a SQLite 3 database, which is placed in the same folder as the [face=courier]wincmd.ini[/face] named [face=courier]thetvdb.sqlite[/face].
If anything goes wrong you can just delete the database file or use tools like SQLite Expert to manually modify the database directly.
The location of the database file will probably also be configurable in future versions, if there is any demand for it.

It is possible to change the language for querying TheTVDB.com by modifying [face=courier]contentplug.ini[/face] and adding:

Code: Select all

[thetvdb]
language=XX
Where [face=courier]XX[/face] is the language code (like "de"). Please be aware, that TheTVDB.com does not necessarily provide all show data in non-English languages.

Currently valid language codes:
all - All langauges
da - Dansk
fi - Suomeksi
nl - Nederlands
de - Deutsch
iz - Italiano
es - Español
fr - Français
pl - Polski
hu - Magyar
el - Ελληνικά
tr - Türkçe
ru - русский язык
he - עברית
ja - 日本語
pt - Português
zh - 中文
cs - čeština
sl - Slovenski
hr - Hrvatski
ko - 한국어
en - English
sv - Svenska
no - Norsk

The default is "en".

Detection patterns
To determine the name of the series as well as the number of the episode and season a regular expression based pattern is used which can be defined in the [face=courier][thetvdb][/face] section of the [face=courier]contetplug.ini[/face] file.
Per default, four entries are generated:

Code: Select all

[thetvdb]
1_regexp=^(.*?)[\s\.]*-?[\s\.]*[Ss](\d\d?)[Ee](\d\d?)
1_title=$1
1_season=$3
2_regexp=^(.*?)[\s\.]*-?[\s\.]*(\d\d?)x(\d\d?)
2_title=$1
2_season=$3
3_regexp=^(.*?)(\d\d\d\d)[^\d](\d\d)[^\d](\d\d)
3_title=$1
3_year=$2
3_month=$3
3_day=$4
4_regexp=^(.*?)(\d\d)[^\d](\d\d)[^\d](\d\d\d\d)
4_title=$1
4_year=$4
4_month=$2
4_day=$3
This will match files like [face=courier]Heroes - S01E01.avi[/face] or [face=courier]Heroes 1x1.avi[/face] or [face=courier]Heroes 2006-09-25.avi[/face]. You can modify this pattern or add custom ones.
For every pattern at least the regexp and the title have to be defined. There are four types of different pattern you can define and for every match of
the regular expression a placeholder [face=courier]$x[/face], starting from 1 is defined.

To match an series by its episode and season number, you have to define the episode field and optionally a season field. When no season is defined, season "1" is assumed.

Code: Select all

1_regexp=^(.*?)[\s\.]*-?[\s\.]*[Ss](\d\d?)[Ee](\d\d?)
1_title=$1
1_season=$3
Alternatively, you can define a pattern that matches the absolute number. Thus to match something like [face=courier]Bleach_029[/face] you could define:

Code: Select all

5_regexp=(.*?)_(\d*)
5_title=$1
5_absolute=$2
A third way is to define an episode by its name, so to match something like [face=courier]Familiy Guy\xx The Story So Far.avi[/face] you could use:

Code: Select all

6_regexp=(.*?)\\xx (.*)\.avi
6_title=$1
6_episodeName=$2
Finally you can match by the airdate. Here you have to define the year, month and day of the airdate as numerical values:

Code: Select all

3_regexp=^(.*?)(\d\d\d\d)[^\d](\d\d)[^\d](\d\d)
3_title=$1
3_year=$2
3_month=$3
3_day=$4
Please be aware, that parsing of the pattern is stopped when a wrong entry is found and all following entries will be ignored.


Notes
Support for Unicode is available when using Total Commander 7.5 (beta).


Known bugs and limitations
- Due to synchronous http connection, the GUI will be blocked when fetching data from thetvdb.com.


Change log
0.1 (2009-05-28): Initial release.
0.2 (2009-05-30): Added custom pattern support. Thanks to fenix_productions for the idea.
0.3 (2009-05-30): Added support for absolute episode numbers.
0.4 (2009-05-30): Added support to match the folder.
0.5 (2009-05-30): Enable logging by setting debug=N in the ini file where N is 0=ERROR, 1=WARNING, 2=INFO 3=DEBUG and 0 is the default.
0.6 (2009-06-01): Added support to match episode name.
0.7 (2009-06-14): Added support to match airdate. Improved support for multiple languages in queries ([face=courier]language=all[/face]). Note: All assigned lookups will be lost due to database changes.


Screenshot
Image: http://helicoid.de/tc/wdx_thetvdb.png


Download
http://helicoid.de/tc/wdx_thetvdb_0.7.zip (352 KB)

The source code will probably be made available later, if somebody is interested in it.
Last edited by j on 2009-06-14, 22:03 UTC, edited 9 times in total.

User avatar
fenix_productions
Power Member
Power Member
Posts: 1952
Joined: 2005-08-07, 13:23 UTC
Location: Poland
Contact:

Post by *fenix_productions » 2009-05-29, 20:00 UTC

2j
I've checked it with few files and must say: nice work.

Of course, I am waiting for more patterns, maybe with regular expressions for matching titles and numbers by your own.

It would be also nice to have some default "season" values (configurable). As an example: I have few files Bleach_XXX and I know this anime has only one season. It is ridiculous to add "1x" to each file for me.
"When we created the poke, we thought it would be cool to have a feature without any specific purpose." Facebook...

#128099

j
Member
Member
Posts: 123
Joined: 2009-04-13, 19:49 UTC

Post by *j » 2009-05-29, 20:15 UTC

fenix_productions wrote:Of course, I am waiting for more patterns, maybe with regular expressions for matching titles and numbers by your own.
Actually the patterns are already regular expressions. It would be easy to add more. I am not quite sure how to add them to a INI file and define the right order to associate name, season and episode.
Maybe something like:

Code: Select all

1_regexp=(\d*) (\d*) - (.*)
1_regexporder=season,episode,name
What do you think? This at least would be quite trivial to implement...
fenix_productions wrote:It would be also nice to have some default "season" values (configurable). As an example: I have few files Bleach_XXX and I know this anime has only one season. It is ridiculous to add "1x" to each file for me.
Indeed. Maybe the custom pattern could allow missing fields which in turn would be prompted for by the dialog (with the default value "1" and the ability to store the value for each show). Something like:

Code: Select all

2_regexp=(.*)_(\d*)
2_regexporder=name,episode
Thanks for the feedback.

User avatar
fenix_productions
Power Member
Power Member
Posts: 1952
Joined: 2005-08-07, 13:23 UTC
Location: Poland
Contact:

Post by *fenix_productions » 2009-05-29, 20:40 UTC

Nice thoughts you've got but I was thinking more about giving separate variables for values types (season, episode, name). That way there would be no "ordering problem".

Example 1. My pattern (Bleach_XX):

Code: Select all

1_regexp=(.*)_(\d*)
1_title=$1
1_season=1
1_episode=$2
Example 2. Your pattern (showname*S__E__*)

Code: Select all

2_regexp=(.*)S(\d*)E(\d*)(.*)
2_title=$1
2_season=$2
2_episode=$3
Of course, if x_season gives nothing then some global variable should be used. I think 1 is OK for most of the cases.
"When we created the poke, we thought it would be cool to have a feature without any specific purpose." Facebook...

#128099

j
Member
Member
Posts: 123
Joined: 2009-04-13, 19:49 UTC

Post by *j » 2009-05-30, 00:39 UTC

I liked your idea. Version 0.2 is available for download.

User avatar
fenix_productions
Power Member
Power Member
Posts: 1952
Joined: 2005-08-07, 13:23 UTC
Location: Poland
Contact:

Post by *fenix_productions » 2009-05-30, 01:56 UTC

2j
Your example from readme.txt works nicely for many files. Unfortunately: there are problems with mentioned Bleach :(

It is very strange:

When I have Bleach_YYY.avi where Y are numbers from 197 to 213, nothing is shown. If I rename the very same files to Bleach_1xYYY.avi pattern it works perfectly.

Similar rule applies for these:
Bleach_26.avi
Bleach_027.avi
Bleach_028.avi
Bleach_029.avi
Bleach_030.avi
Bleach_031.avi
Bleach_032.avi
Bleach_033.avi
Bleach_034.avi
Bleach_035.avi
Bleach_036.avi
Bleach_037.avi
Bleach_038.avi
Bleach_039.avi
Bleach_040.avi
BUT following files work right away w/o any changes:
Bleach_01.avi
Bleach_010.avi
Bleach_02.avi
Bleach_03.avi
Bleach_04.avi
Bleach_05.avi
Bleach_06.avi
Bleach_07.avi
Bleach_08.avi
Bleach_09.avi
Bleach_1.avi
Bleach_10.avi
Bleach_19.avi
Bleach_2.avi
Bleach_20.avi
Bleach_21.avi
Bleach_22.avi
Bleach_23.avi
Bleach_24.avi
Bleach_25.avi
Bleach_3.avi
Bleach_4.avi
Bleach_5.avi
Bleach_6.avi
Bleach_7.avi
Bleach_8.avi
Bleach_9.avi
Is it the problem of questioning TheTVDB.com site or something wrong with parser?

I assume that you might be sending episode number in the form you've got from file name (i.e. 23) but server holds it differently (023) and returns nothing. So my proposal is to try to assign leading zeros and ask again. Of course, this should be optional ;)


One more question: how many own x_regexp rules are allowed to be used?
"When we created the poke, we thought it would be cool to have a feature without any specific purpose." Facebook...

#128099

j
Member
Member
Posts: 123
Joined: 2009-04-13, 19:49 UTC

Post by *j » 2009-05-30, 02:09 UTC

First of all, the reason why [face=courier]1xYYY[/face] "works" is that the regular expression actually only sees [face=courier]1xYY[/face] (as of [face=courier]\d\d?[/face]).

According to http://thetvdb.com/?tab=series&id=74796&lid=7 Bleach has 10 Seasons. So there is not much you can do about it, but conform to the numbering scheme of TheTVDB.com if you want to get information from it.
So 026 actually is 2x1 in TheTVDB.com database.

The list of regexp are stored in a vector and are only limited by the integer used for counting ;-).

User avatar
fenix_productions
Power Member
Power Member
Posts: 1952
Joined: 2005-08-07, 13:23 UTC
Location: Poland
Contact:

Post by *fenix_productions » 2009-05-30, 02:29 UTC

2j
Looks like you are right. I didn't know they followed manga scheme. It will be bothersome to rename all of the files because of strange splitting (i.e. Season 4th contains episodes from 64 to 91) but it's not your problem anymore or maybe…

They have something like "Absolute number" which tells you that S2E1 is really 26th episode. Please think about x_isabsolute parameter if it's not very big problem to implement. With many possible filters I could even use full movie name in regexp.

After all: you did great job.

One more suggestion for the future: if you're willing to follow "plugins writer path" maybe you will create someday something similar to MP3Commander but with movies. It would be freaking awesome.
"When we created the poke, we thought it would be cool to have a feature without any specific purpose." Facebook...

#128099

j
Member
Member
Posts: 123
Joined: 2009-04-13, 19:49 UTC

Post by *j » 2009-05-30, 13:02 UTC

I just saw that the XML reply of TheTVDB.com contains an "[face=courier]absolute_number[/face]" field and I added support for it.

You should now also be able to automatically rename your Bleach episodes ;-)

User avatar
fenix_productions
Power Member
Power Member
Posts: 1952
Joined: 2005-08-07, 13:23 UTC
Location: Poland
Contact:

Post by *fenix_productions » 2009-05-30, 13:52 UTC

2j
Now it works nicely :)

Another suggestion: return ft_fieldempty instead of 0000-00-00 for Airdate. Looks better this way.
"When we created the poke, we thought it would be cool to have a feature without any specific purpose." Facebook...

#128099

j
Member
Member
Posts: 123
Joined: 2009-04-13, 19:49 UTC

Post by *j » 2009-05-30, 14:17 UTC

Glad to hear it works for you now.
I added the [face=courier]ft_fieldempty[/face] return value. It will be available in the next release.
#84251

WhACKO
Junior Member
Junior Member
Posts: 67
Joined: 2009-04-07, 11:17 UTC

Post by *WhACKO » 2009-05-30, 18:36 UTC

Any way of getting the add-on to recognize the show using this pattern (the folder in which the avi files are as Show Title):

The Big Bang Theory\s1e01 Pilot.avi
s1e02 The Big Bran Hypothesis.avi
s1e03 The Fuzzy Boots Corollary.avi
s1e04 The Luminous Fish Effect.avi
.
.
.
s2e01 The Bad Fish Paradigm.avi
s2e02 The Codpiece Topology.avi

or in place of the s# maybe s0#e##, or shows just by number like

Better Off Ted\01.avi
Better Off Ted\02 Heroes.avi
Better Off Ted\03 Through Rose Colored HAZMAT Suits.avi
Better Off Ted\04 Racial Sensitivity.avi



Keep up the good work!

j
Member
Member
Posts: 123
Joined: 2009-04-13, 19:49 UTC

Post by *j » 2009-05-30, 18:50 UTC

Sorry, it is not possible to get the name from the folder at the moment.

A spontaneous (and very easy to implement) idea would be to count the numbers of "\" in the pattern and cut the filename up to the Nth "\" from the back:
So: [face=courier](.*)\\s(\d*)e(\d*)[/face] would match [face=courier]Name\sXeX*avi[/face] (but [face=courier](.*).s(\d*)e(\d*)[/face] would not match)

Or [face=courier](.*?)\\.*(\d*)\\(\d*)[/face] would match [face=courier]Name\Season X\Y*avi[/face] (but [face=courier](.*).Season (\d*)[^\d](\d*)[/face] would not match)

Any better ideas?
#84251

User avatar
fenix_productions
Power Member
Power Member
Posts: 1952
Joined: 2005-08-07, 13:23 UTC
Location: Poland
Contact:

Post by *fenix_productions » 2009-05-30, 19:12 UTC

2j
Does it really matter if something does not match? In such case next regexp should be checked IMHO.

My idea is to have additional parameters x_parentlevel which would tell how many levels above should be combined with file name for regexp matching. 0 could be default value which would mean to take into consideration file name only (same like it is now).

With such scheme it would be up to user to decide that folders are important.

BTW What are the priorities for current parameters? Is x_absolute more important than x_episode?
"When we created the poke, we thought it would be cool to have a feature without any specific purpose." Facebook...

#128099

j
Member
Member
Posts: 123
Joined: 2009-04-13, 19:49 UTC

Post by *j » 2009-05-30, 19:26 UTC

I guess [face=courier]x_parentlevel[/face] is explicit where as counting the occurrence of the backslash is implicit. I can't see much advantage over it, as a pattern probably always contains the "\" to split up the string.
I just wanted to make clear that [face=courier](.*)\\(\d)[/face] is not a subset of [face=courier](.*).(\d)[/face] with my solution. Where as with [face=courier]x_parentlevel[/face] it would be, as the string given to the regular expression engine is different when [face=courier]x_parentlevel[/face] is not 1.

The patterns are matched according to their order in the INI file (or to be more precise the number in front), so first comes first serve. When parsing the pattern itself, [face=courier]x_episode[/face] has a higher priority (say when [face=courier]x_episode[/face] and [face=courier]x_absolute[/face] for the same [face=courier]x[/face] are defined only [face=courier]x_episode[/face] is used).
#84251

Post Reply