[ome-devel] Existing client side code in C

[Ilya Goldberg]

On Aug 5, 2004, at 9:11 AM, Stefan Frank wrote:
> -----one----
> a) getting the pixels into ome-is
> b) getting the meta-data (and this includes the timestamp) into ome-ds
-----
> If I got it right, there are two options for one:
> a) using the perl-import-modules
> b) using the interfaces ome-is (basically http-put, as chris pointed 
> out) and ome-ds (basically xml-rpc) offers

That's exactly right.  To import XML, the steps are:
1.  Upload XML to omeis using POST and get a FileID (UploadFile is the 
omeis method)
2.  Call the back-end over XMLRPC to ImportFiles(), and pass the FileID.

Using the back-end to import any supported file format involves the 
same two steps.  The other option is to do a client-side import where 
you tell ome-ds about all the meta-data using DTOs and tell omeis about 
all the pixels using its methods.  Obviously, its more complicated.  
VisBio does this with several formats, and may even do it with Leica (I 
don't know).

> ----two----
> a) getting the pixels out of ome-is
> b) getting the meta-data out of ome-ds

This is done using the standard interface, for which there is no C 
library, but talk to Zach Pincus <zpincus at stanford.edu> because he's 
probably working on the same thing.  Sorry C is left out in the cold 
like this, but these things are pretty easy to put together with 
XMLRPC.  If there are good client-side SOAP libraries, it might make it 
even easier, but I'm not the expert here.

> Getting attributes of some Pixels from OME-DS into a C-Program?! How 
> do you do that in the programms inside the analysis-chains?! Don't 
> they need any MetaData?! Are they working solely on the raw pixels?

Of course not!  This is the beauty of AnalysisModules and 
SemanticTypes.  The idea is you declare an AnalysisModule by specifying 
its inputs and outputs using STs.  Pixels are STs just like any other 
meta-data.  You also specify which Handler you want to use, depending 
on how your algorithm is implemented.  The supported handlers are CLI 
(for command-line executables), Matlab (for Matlab modules) and Perl 
(for Perl classes).  The CLI handler has some execution instructions 
that you put in the AnalysisModule declaration.  This is basically 
connecting the inputs and outputs you declared to the executable - how 
to arrange them on the command-line, or put them on stdin, read them 
from stdout, etc.  The matlab handler execution instructions are a 
little simpler because STs are fed directly into Matlab using its 
'structs' (or whatever they're called).  The Perl handler is just a 
class you write in Perl (there are working examples in 
src/perl2/OME/Analysis/Modules/).  Presently the handlers are not in 
the best state in this release, but we're all working on fixing them up 
for the next major release (OME 2.4).

Once you have your AnalysisModules declared, you string them together 
into AnalysisChains.  Harry is working on a visual drag and drop UI 
that lets you connect boxes together with arrows to do this.  This is 
mostly finished, actually.
Once you have an AnalysisChain, you execute it against a Dataset.
The AnalysisEngine takes care of making sure instances of the 
SematicTypes you declared appear on your module's inputs (Pixels, 
meta-data, whatever - any ST in the system), and collects the results 
from your module's outputs (again, Pixels, meta-data, any ST in the 
system), and sends them on to the next module in the chain.  This 
seemingly magical thing is already working to a large extent in the 
system that you have, because every imported image gets processed by 
the Image Import Chain (defined in src/xml/imageImportChain.xml).  If 
this wasn't working, you wouldn't be able to see the thumbnails.  This 
is the most complicated way imaginable to display thumbnails in an HTML 
page, but it illustrates the power and flexibility of the system.

The system is designed around the idea that analysis modules are 
passive - they get executed by the AnalysisEngine whenever it decides 
they need executing.  Think of them as "plug-ins".  You can do it the 
other way where you write a big C program that gets things out of OME 
and sends them back, but this is a lot more complicated to pull off.  
The whole idea is that you want to write a specialized algorithm with 
as little work as possible, implement it in your favorite language 
(C->CLI, Perl, Matlab), drop it into OME and let it become part of the 
"system", talk to other modules, etc.  The CLI handler even lets you 
use opaque binaries where you don't have access to the source code and 
still lets you marry them to OME.

The 2.2 series of releases are really meant to flesh out the "Image 
repository" aspect of OME.  The 2.4 series will flesh out all of its 
analysis capabilities.  If you're interested in helping get the 2.4 
series working, we would *LOVE* the help.  Basically, this involves 
working with the CVS HEAD rather than the OME_2_2 branch.  The CLI 
handler is a bit of an orphan right now because we're all worshiping at 
the altar of Matlab.  It did work before, so it really only needs some 
massaging.  Josiah Johnston <siah at nih.gov> in my group wrote it and 
he's the man to see about sprucing it up.

Take care
Ilya

>
> thx for your patience!
> stefan
>
>> -Ilya
>>>
>>> -Josh.
>>> _______________________________________________
>>> ome-devel mailing list
>>> ome-devel at lists.openmicroscopy.org.uk
>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>>>
>> _______________________________________________
>> ome-devel mailing list
>> ome-devel at lists.openmicroscopy.org.uk
>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>
> -- 
> Stefan Frank
> iBioS - Intelligent BioInformatics Systems
> http://www.dkfz-heidelberg.de/ibios
> DKFZ - German Cancer Research Center
> Im Neuenheimer Feld 580
> 69120 Heidelberg
> Tel.: +49 (0) 6221 42-3612
> Mail: s.frank at dkfz-heidelberg.de
>