Let's make writing Java plugins fun!

[meisl]

Recently, I came across* Ken Händel's tc_java which enables writing plugins for TC in Java (all kinds are supported).
It does so by bridging TC's plugin interface via JNI (Java Native Interface) and also providing adapter classes**.

IMHO it's quite a good piece of work and there's plenty of documentation provided to learn from.
However, it's a bit outdated since, and seems not to be maintained any longer.

I gave it a try by starting to write a content plugin***. Guess I should mention that this is also my first plugin for TC

Anyways, from my experience so far I see a couple of problems:

• - the provided interface is very basic (although complete), that is to say, it just doesn't "feel" very much like Java. Rather like C (yes, NOT C++!)
  
  - you're returning integer error/success codes all the time (they're not even enums) and your real results go into "output" parameters, ie you modify things that you're being passed.
  
  - also, when you have a sequence of things to return it's always a sequence of calls with some "no-more-things" code you are to return at the end. That really should be abstracted, eg using an iterator pattern.
  
  - then, no generics.
  
  - on top, the README and javadocs contain quite a number of flaws/things-out-of-date. Not too big a deal each but surely frustrating for someone new to TC plugins and/or Java.
  
  - performance IS an issue
  
  - ...

All this, as it stands, makes it rather questionable why one should use this at all, given that you hardly get any of the benefits that Java has to offer.

On the other hand, I can absolutely imagine how to build upon tc_java and provide an interface that's much more "the Java way", and, possibly, improve performance.
Also, opening up to Java people (ie really opening up) should be an overall benefit for TC and its community, IMHO. I, personally, am addicted to TC since almost the beginning and have written myself quite a bunch of utilities, but each of them I invoke from the button bar, effectively...

So then, if anyone's interested - and ideally willing to contribute - I'd make a start by
a) trying to get into contact with the original author, Ken Händel, and
b) put up a github repo

---
* see this thread: "Help with the Java API"
** by "adapter class" I mean an abstract class inheriting the relevant Java interface and providing basic default implementations of all the methods that are not absolutely necessary
*** my plugin is about NTFS ADS (Alternate Data Streams), inspired by the thread "Integrate NTFS Streams handling into TC"

[jmwap]

I cannot say I will contribute (cause I don't have time at all), but if I ever needed to write a plugin, and I could use java, I would use it.


Also worth noting that having java as a possibility would actually mean being able to use other usual suspects such as groovy, clojure, jruby etc

[meisl]

[...]being able to use other suspects such as groovy, clojure, jruby

Definitely, scripting for TC in general deserves more attention! But 'til there it'll be a longer road...

So, even if you wouldn't be able to make code contributions - would you be interested in giving feedback every now and then?
Either here or in github issues, whatever suits.
And it would be about improving the Java interface only at first.

Oh & thanks a lot for your reply - wasn't expecting any so soon

[meisl]

Have put my little pet project, "NtfsStreamsJ" on github. But plz bear with me, it's in a very, VERY premature state...

Note: this is NOT really what I'm talking about here, only a precursor, at best.
Just to give ppl who are interested at least some kind of idea.

[jmwap]

[...]being able to use other suspects such as groovy, clojure, jruby

Definitely, scripting for TC in general deserves more attention! But 'til there it'll be a longer road...

So, even if you wouldn't be able to make code contributions - would you be interested in giving feedback every now and then?
Either here or in github issues, whatever suits.
And it would be about improving the Java interface only at first.

Oh & thanks a lot for your reply - wasn't expecting any so soon

the thing is, in order to give feedback I guess the only way is to try it by writing a plugin...

[meisl]

You could watch me writing it and ask questions about the why/how.
Also, code reviews, even if you're not into the code would be greatly appreciated. github offers quite some facilities.

I understand this still means effort from your side, but it's not necessarily only in-deep discussions that'd help, just someone else's perspective.

For example: how much time would be acceptable just to get NtfsStreamsJ running in your TC? Say, 20 minutes. Right now that's not realistic I think. Take 3 mins to skim through the repo and then 2 more to make an issue à la "write fool-proof HOWTO install in 15 min", with some hints on the things that made a "?" pop up in your head before.


If you need help with git or whatever else blocks you then say so and I'll try to provide it. But of course 20 min altogether never will be enough in that case.

[meisl]

Ok, here's a first improvement of the API (simple WDX only):

Instead of implementing the tedious

Code: Select all

    public int contentGetSupportedField(int fieldIndex,
                                        StringBuffer fieldName,
                                        StringBuffer units,
                                        int maxlen)

and

Code: Select all

    public final int contentGetValue(String fileName,
                                     int fieldIndex,
                                     int unitIndex,
                                     FieldValue fieldValue,
                                     int maxlen,
                                     int flags)

where you would repeat for the latter (similar for the first...):

Code: Select all

try {
    switch (fieldIndex) {
        case 0:
            fieldValue.setValue(FT_NUMERIC_32, 42);
            return FT_NUMERIC_32;
        case 1:
            fieldValue.setValue(FT_STRING, "foo");
            return FT_STRING;
        // more cases...
        default:
            return FT_NOSUCHFIELD;
    }
} catch IOException(e) {
    log.error(e);
    return FT_FILEERROR;
}

in each and every plugin

- you rather subclass

Code: Select all

ContentPlugin // extends WDXPluginAdapter

...and say in your constructor:

Code: Select all

define(new Field.INT("count") {
    public int getValue(String fileName) throws IOException {
       return 42;
    }
});
define(new Field.STRING("summary") {
    public String getValue(String fileName) throws IOException {
       return "foo";
    }
});
// more field definitions...

...and that's it!

Apart from obviously being more concise the advantages are:

• - type checking at compile time (+ you won't be able to create fields of some bogus type MyFooClass which can't be recognized by TC)
  
  - DRY ("Don't Repeat Yourself")
  
  - checks for validity of the field's name (its length, if it contains invalid characters) at runtime (= @ TC startup)
  
  - checks for validity of the returned value (apart from its type, see above) at runtime
  
  - the previous checks can easily be instrumented and covered by tests s.t. you get errors at build time rather than having to restart TC after every code change

So that's what came out my first refactorings of NtfsStreamsJ.
Alternatively we could have methods with a special annotation, yielding even more concise code but less type checking at compile time.

Also, I'm thinking of putting some basic test generation in the base class ContentPlugin, covering a) the field discovery process at TC startup and b) sample invocations on actual files for each field that the plugin provides.
These tests could be exercised at build time (only!) s.t. one gets early feedback on simple mistakes without having to set up eg jUnit - and most importantly: not to repeat those basic tests that apply to every plugin impl in jUnit test cases.

So, if anybody's got any thoughts on this - I'd greatly appreciate hearing them

[Lefteous]

Why is there more then one programming language and one framework on the planet? Well because there are very different kind of tasks to solve. For TC plugins very slim languages without large frameworks are the way to go.
Using java for this job is taking a sledgehammer to crack a nut - apart from the academical point of view.

[meisl]

Why, for example, is there more than one idea of the ideal society model on the planet?
(Since everybody, for him/herself, knows which is best...)

Why are people different?
(Since everybody, for him/herself, knows who's different - the others, only!)

Why doesn't anybody else seem to understand that my view of the world is exactly the right and true only one?
(see above)

...

EDIT: @Lefteous: I'd love to have a real discussion, about actual points.
For example:
What actually is the impact of having to load a JVM, for the user of TC + plugin?
What might be the benefits that Java has to offer over "slim languages"?
For developers, for the user/community?
Maybe someone thinks it's worth a deal?

You know, apart from making big noisy statements that speak more of the speaker than the subject at hand...

[Lefteous]

Well tell us about the benefits of java-based plugins for users!

[meisl]

1) Lowering the hurdle to write a plugin for quite a lot of people should provide for more ppl getting their specific task done, particularly if deemed too specific to justify being incorporated into TC

2) A well thought-out API, seconded by a capable type system helps to improve quality

These, after having thought about it a while, are the most basic points, IMHO. I'm aware that they both contribute to the user's benefit indirectly, but hey, if they do, then so what?

Also, I understand that 1) might in fact be considered a Bad Thing(TM) by some - possibly due to some weird sort of elitist attitude...

[Lefteous]

2meisl

1) Lowering the hurdle to write a plugin for quite a lot of people should provide for more ppl getting their specific task done, particularly if deemed too specific to justify being incorporated into TC

You are right - quite a few people know Java. Difficult to say if the fact that Java couldn't be used in the way you imagine actually ever has been a blocker for a plugin project.

A well thought-out API, seconded by a capable type system helps to improve quality

Yes I agree. I don't have to look to far to know that a type system would help to avoid some nasty bugs...
Anyway this is something that can also be achieved in C++ - maybe even better than in Java.

So working on an API makes sense - as long as performance doesn't suffer.

[meisl]

Thanks, Lefteous.

Difficult to say if the fact that Java couldn't be used in the way you imagine actually ever has been a blocker for a plugin project.

Part of this thread's intent is to find out. So far it's not many of which we know; jmwap above and then the other person who "asked" "Help with the Java API". Well, and myself.

[...]
Anyway this is something that can also be achieved in C++ - maybe even better than in Java.
So working on an API makes sense - as long as performance doesn't suffer.

Indeed. Even in Pascal, in which TC is written I suppose. Btw: is it the case?

So in this sense, even if I have made my choice here (and for now) maybe similar projects in named languages can profit from the ideas and discussions.
Comments from another language's point of view on general patterns/approaches are greatly appreciated - as long as they're constructive

One big pro of the JVM, as jmwap mentioned, is the nr of even more languages that'd be enabled automatically.
Btw, this would be true for .NET as well...

As for performance: I'd love to discuss this in a bit more detail but let's not make this post too long.

[meisl]

Here's what I came up with for editable fields (WDX, used in "Change attributes" dialog but without custom edit dialog so far).
Btw: found const FT_SETSUCCESS (0) to be missing in tc-apis-1.7 (part of tc_java).

Instead of contentGetSupportedFieldFlags and contentSetValue (not shown, due to ugliness):

Code: Select all

define(new EditableField.STRING("foo") {
    public String getValue(String fileName) throws IOException {
        return /* retrieve it */
    }
    public void setValue(String fileName, String value) throws IOException {  // << must implement here, must NOT implement in Field
        /* do set it */
    }
});

[jmwap]

even without having seen how a plugin is written before, those changes you show (in the first post mostly), make a lot of sense.

I am thinking of some plugin I could write to at least play around with things (I have some idea already, but it's so niche I would be the only user). If I ever get to do this...of course (time is limited).

@Lefteous, as meisl says, even if other languages are more suited for this, lots of people might not know them (I was very proficient in c/c++ years ago, not anymore), and know java, or want to use some java library that is not available in other languages for example.

Also, perf of java startup will only improve with time, and there are other more advanced features in some case, like http://kobo.github.io/groovyserv/ for the adventurous, that avoid spawning a new jvm each time.