[ome-devel] Writing ImageJ2 plugins

[Curtis Rueden]

Hi Graeme,

Based on the list that you & Ilan sent me a while back, I started a
spreadsheet detailing priority targets for support within OMERO:

https://docs.google.com/spreadsheet/ccc?key=0Ancmne1B9NOwdGxQOEpKSVVGbk1TQ1ZSMHEzWHFwbmc&usp=sharing

Anyone is free to check it out and comment if desired. Some of the items on
your list were quite general, and/or not totally clear to me, so edit away
as needed.

> my questions were motivated by the desire to write plugins for IJ1
> that will not be obsolete in IJ2, and preferably be able to be run
> headlessly, e.g. from OMERO.

OK, thanks for clarifying.

> It seems that ImageJ2 also uses a run method as the entry point for
> running a command? (correct me if I am wrong) In that case, at least
> the run method would have to be replaced.

ImageJ2's commands use Runnable's run(). ImageJ1 plugins use a custom
PlugIn interface's run(String). So they are technically compatible, though
I wouldn't necessarily recommend creating a hybrid. ;-)

> I was not so worried about ImageJ1's dialogs (which I expected would
> *not* be OK in a headless plugin)

Actually, confusingly, you *can* use GenericDialog when running headless in
a macro because of the javassist hacks I mentioned. When running from a
macro, the dialog is never actually shown, but instead parameters are
harvested from the macro options string.

> It sounds as though converting IJ1 plugins to use the IJ2 API is
> likely be more productive than trying to use the legacy IJ1 API from
> an IJ2 plugin (headless), but is the latter already known to be
> hopeless? If not, I might be tempted to try a little.

I wouldn't say hopeless. In many cases, you *can* run IJ1 plugins headless,
as explained on the Fiji wiki Headless page I linked before (
http://fiji.sc/Headless). My next step for the OMERO integration will be
create an ImageJ2 command "Run Script" which executes a given script or IJ1
macro. Then I can start testing calling IJ1 macro code headless from OMERO
and see how well it works in practice.

That said, I stand by my suggestion that *new* code would be better written
as an ImageJ2 command because otherwise it won't be easily accessible from
the growing number of contexts I mentioned such as CellProfiler and KNIME.

> Until now I have chosen to write for IJ1 since the IJ2 API is still a
> moving target and has a smaller user base, but I guess IJ2 / ImgLib2
> already makes a lot of  sense for headless processing tasks. Would you
> say that once imagej2-b9-rois is out, the IJ2 API will be fairly
> stable going forward?

My promise is that when ImageJ 2.0.0 final is released, *then* the API will
be stable, with old API being properly deprecated for long periods of time,
etc. Before that, I do not want to make any promises. The API is growing
more and more stable with each release, but it is tough to say what "fairly
stable" really means here.

One idea I had recently is to start bringing some components of ImageJ2 out
of beta in phases. The scijava-common component is already out of beta.
Some IJ2 components such as ij-core are already extremely stable. The
scifio library will probably be out of beta (and hence stable) within a
year. But the hard stuff remains: the ij-data component, how it ties in to
ImgLib2 (which also needs to come out of beta at some point), and the IJ2
display model. Those may need to bake a little longer.

Regards,
Curtis


On Wed, Oct 23, 2013 at 4:02 PM, Graeme Ball <graemeball at googlemail.com>wrote:

> Hi Curtis,
>
> thank you for your reply - that certainly clears up some of my questions.
> Yes, my questions were motivated by the desire to write plugins for IJ1
> that will not be obsolete in IJ2, and preferably be able to be run
> headlessly, e.g. from OMERO.
>
> If possible, when I write (ImageJ1) plugins I try to confine any GUI calls
> to the run method, and all of the real work to an exec method that does not
> require interactive input. It seems that ImageJ2 also uses a run method as
> the entry point for running a command? (correct me if I am wrong) In that
> case, at least the run method would have to be replaced.
>
> I was not so worried about ImageJ1's dialogs (which I expected would *not*
> be OK in a headless plugin), but was more concerned about ImagePlus, Roi
> etc. which make heavy use of awt classes. If these also cannot be used
> headlessly, then I guess a legacy mode with virtual UI, or re-writing to
> use the IJ2 API are the only real options.
>
> It sounds as though converting IJ1 plugins to use the IJ2 API is likely be
> more productive than trying to use the legacy IJ1 API from an IJ2 plugin
> (headless), but is the latter already known to be hopeless? If not, I might
> be tempted to try a little.
>
> Until now I have chosen to write for IJ1 since the IJ2 API is still a
> moving target and has a smaller user base, but I guess IJ2 / ImgLib2
> already makes a lot of  sense for headless processing tasks. Would you say
> that once imagej2-b9-rois is out, the IJ2 API will be fairly stable going
> forward?
>
> Thanks & Best Regards,
>
> Graeme
>
>
> On Wed, Oct 23, 2013 at 6:45 PM, Curtis Rueden <ctrueden at wisc.edu> wrote:
>
>>  Hi Graeme,
>>
>>  Sorry for the delay in reply. Writing to a public mailing list is
>> better. I am CCing imagej-devel and ome-devel since your questions are very
>> technical, and related to those two projects.
>>
>>  > I thought I would Email you directly because the ImageJ-devel
>> > mailing list seems to be intended for core project development.
>>
>>  From http://developer.imagej.net/mailing-lists:
>>  > ImageJ: For general user and developer discussion about ImageJ. This
>> > list is the best place to go for help with your specific use case.
>> >
>> > ImageJ-devel: For technical and developer discussion of the ImageJ2
>> > project. This list is for programmers contributing code to ImageJ2, or
>> > interested in technical discussions surrounding such development.
>>
>>  Both are public mailing lists intended for use by the community, not
>> just for core project development. The wording of ImageJ-devel is phrased
>> that way because ImageJ2 is still in beta and if you are using it for your
>> projects, you are an early adopter who is almost certainly "interested in
>> technical discussions" surrounding core development.
>>
>>  > 1. I was looking for example plugins (& example minimal plugin
>> > project), but get 404's when I try to look at the HelloWorld and
>> > GradientImage examples here:-
>> > http://developer.imagej.net/writing-imagej2-plugins
>> > (I found the github repo & Fiji wiki ImgLib2 examples)
>>
>>  All links on that page should be working again.
>>
>>  > 2. I see an example of calling IJ2 from a legacy plugin, but am also
>> > interested in doing the reverse, and unsure what the limitations are.
>>
>>  The limitations are evolving. Legacy support is very challenging to get
>> right. That said, it should work to call ImageJ1 data structures from an
>> ImageJ2 style command. There is a LegacyService for converting between IJ1
>> and IJ2 data structures, although the API still needs work.
>>
>>  Note that at the moment, ImageJ2 commands are only discovered and
>> exposed in the modern ImageJ2 UI. We are exploring ways of blending things
>> in additional ways, such as allowing the ImageJ1/Fiji legacy UI to discover
>> and allow execution of IJ2 commands.
>>
>>  What is your goal? You want to write a plugin that is available from
>> both IJ1 and IJ2? If so, structuring it as a legacy IJ1 plugin (which may
>> or may not leverage certain IJ2 API internally as demonstrated in
>> calling-modern-from-legacy) is the way to go for now.
>>
>>  > 3. What are the constraints / requirements for running plugins
>> > (including legacy wrapped in IJ2?) headless, with a view to running
>> > from OMERO in future?
>>
>>  In IJ2 terms, the ImgLib2 library is fully usable headless. If your IJ2
>> command does not depend on ij-ui you probably will be able to run it
>> headlessly. The requirements, in general, are the same as running any Java
>> code headless: do not use AWT or Swing or other UI, especially
>> Windows/Frames/Dialogs.
>>
>>  Unfortunately, you cannot just take a legacy IJ1 plugin and "wrap in
>> IJ2." The problem is that fundamentally, IJ1 is filled with java.awt
>> references including some core classes extending java.awt.Dialog and
>> java.awt.Frame, which are illegal to even instantiate headlessly.
>>
>>  We have some crazy hacks in Fiji to allow limited execution of IJ2 code
>> headless even when it would normally use those AWT window classes. It works
>> by using bytecode manipulation (javassist library) to rewrite parts of IJ1
>> at runtime to avoid references to those classes. However, in some cases
>> things will still blow up. It should go without saying that any time an IJ1
>> plugin asks the user for input (e.g., the WaitForUserInput dialog), it will
>> fail when trying to run headless.
>>
>>  All of that said, we may be able to support calling many IJ1 plugins
>> headlessly from OMERO in the same way we support it headlessly in general:
>> using by install Xvfb on the server side. This gets around the limitations
>> of java.awt.headless=true, in favor of actually giving Java a
>> virtual/offscreen UI. See: http://fiji.sc/Headless
>>
>>  As of this writing, I have not yet tried to get legacy IJ1 plugins
>> working from OMERO. The reason is that I know it will be a lot of effort,
>> for something that would only be transitional. We are writing IJ2 precisely
>> to avoid the sorts of problems incurred by IJ1's heavy use of AWT in core
>> classes. IJ2 maintains a strict separation of concerns, and its commands
>> have typed inputs and outputs, which are a natural match to other paradigms
>> such as OMERO scripts, CellProfiler modules, and KNIME nodes. Mapping IJ1
>> plugins to those paradigms is difficult not only due to headless issues but
>> also because in general there is no easy way to know the actual inputs and
>> outputs of the plugin, since IJ1 plugins do not declare that metadata.
>>
>>  So in conclusion, while I sympathize with the desire to execute ImageJ1
>> plugins from OMERO, ultimately a better direction is to rewrite plugins of
>> interest as ImageJ2 commands so we truly get a "write once, run anywhere"
>> effect with the whole software ecosystem interoperating with ImageJ2.
>>
>>  > Apologies if I missed some documentation, or a more appropriate
>> > community forum to ask these questions, and me know if I am jumping
>> > the gun and would be better waiting a few months.
>>
>>  Not at all; these are good questions and not well explained online. I
>> added a couple of short entries to the online FAQ:
>>
>>  http://developer.imagej.net/can-i-call-imagej2-api-imagej1-plugin
>>  http://developer.imagej.net/can-i-call-imagej1-api-imagej2-command
>>
>>  Please let me know if there are other topics you would like to see
>> covered on the website.
>>
>>  Regards,
>> Curtis
>>
>>
>> On Thu, Sep 26, 2013 at 6:39 PM, Graeme Ball <graemeball at googlemail.com>wrote:
>>
>>>  Hi Curtis,
>>>
>>>  I have a few questions about writing plugins for ImageJ2,
>>> and I thought I would Email you directly because the
>>> ImageJ-devel mailing list seems to be intended for core
>>> project development.
>>>
>>>  1. I was looking for example plugins (& example minimal
>>>    plugin project), but get 404's when I try to look at
>>>    the HelloWorld and GradientImage examples here:-
>>>      http://developer.imagej.net/writing-imagej2-plugins
>>>    (I found the github repo & Fiji wiki ImgLib2 examples)
>>>
>>>  2. I see an example of calling IJ2 from a legacy plugin,
>>>    but am also interested in doing the reverse, and
>>>    unsure what the limitations are.
>>>
>>>  3. What are the constraints / requirements for running
>>>    plugins (including legacy wrapped in IJ2?) headless,
>>>    with a view to running from OMERO in future?
>>>
>>>  Apologies if I missed some documentation, or a more
>>> appropriate community forum to ask these questions, and
>>> let me know if I am jumping the gun and would be better
>>> off waiting a few months.
>>>
>>>  Thanks & Best Regards,
>>>
>>>  Graeme
>>>
>>>  --
>>> ---------------------------
>>> Dr. Graeme Ball
>>> Department of Biochemistry
>>> University of Oxford
>>> South Parks Road
>>> Oxford OX1 3QU
>>> Phone +44-1865-613-359
>>> ---------------------------
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20131023/a045860a/attachment-0001.html>