[ome-users] What did OME ever do for us?

[Ilya Goldberg]

On Jul 18, 2005, at 8:21 AM, Graham Klyne wrote:

> (with apologies to Monty Python and others for the subject line...)
>
> Now that I have OME up and running and able to import microscope 
> images, I am exploring the extent to which it can help us with our 
> image+annotation requirements.  I have some specific questions, but 
> first I'll try and outline what we're trying to do, in case we're 
> trying to do something that isn't reasonable to expect OME to handle.
>
> ...
>
> The research project I'm supporting aims to create a database of 
> images of in situ gene expression in drosophila testes.  To create 
> these images, the researchers go through a fairly involved process of 
> gene selection, gene expression analysis and in situ gene expression 
> image creation. Image acquisition is very much toward the end of the 
> overall experimental procedure, and the resulting images are not (in 
> the current procedure) subjected to subsequent computer analysis.  
> What we do want to do is to cross-refer the images to details of 
> genetic and other information from preparatory steps, and also to 
> capture researcher annotations at the time of image acquisition 
> (noting that researchers can often see more through the microscope 
> than they can see in the resulting computer images).
>
> At the present time, we're not trying to use OME for capturing the 
> preparatory information.  What I am trying to do is use OME to capture 
> researchers' observations and annotations in a structured form at the 
> point of image acquisition and importation to OME, with the 
> possibility of adding additional annotations later.  These will later 
> be combined with the genetic information and researchers conclusions 
> to create a database for final publication.
>
> From this, I hope it's clear that what is important to us is capturing 
> researchers' observations and annotations about images, rather than 
> automated image analysis, which appears to be OME's primary target.  
> The incorporation of annotation as an OME "module execution" seems to 
> support this kind of use, but I'm not sure how the annotation features 
> are intended to be used.
>
> ...
>
> So to my specific questions:
>
> 1. How are custom annotations intended to be used?

There are two types of ad-hoq annotations available in the release.  
One is called simply annotation, which allows you to add a text field 
(or other type of semantically-typed data) to an image.  The other is 
classification.  They differ in that a classification allows you to 
sort an image into one or more groups of mutually exclusive categories. 
  The weakness here is that the web user-interface is completely 
general, so its not targeted at what you're trying to do specifically.  
We are currently developing an HTML templates framework which will 
allow users to use different templates to annotate/search/view their 
data.  Currently, all of the web-ui display is also done with 
templates, but they are global and "fixed" (though still easily 
edited).

We are also involved in an in-situ hybridization project, and are 
addressing these issues by eating our own dog food (as it were).
We decided to use the ad-hoq classification mechanism to assign 
semantic tags to images at the time of acquisition.  The categorization 
mechanism consists of CategoryGroups and Categories, both of which are 
global structures.  An image is assigned to one or more categories 
using a Classification mapping.  In our current ad-hoq annotation 
efforts, we have CategoryGroups defined as Gene Name, Probe, Probe 
Type, and Embryo Stage (with the obvious categories defined within 
them).  We are in the process of making these more formal by declaring 
some of these structures as semantic types.  Also, we are developing 
the customized aanotation/disaply mechanism using this as one of the 
use-cases.  The result is a customized template that presents a large 
thumbnail, and a set of drop-downs and other targeted form controls to 
let the user quickly annotate a newly acquired image using selected 
fields.

>
> 2. How to create new annotation metadata structures?

Metadata structures are called Semantic Types.  These are basically 
collections of primitives (text, integer, etc) and references.  The 
structure itself has a name and so do all of its fields.  These are 
defined using XML (see examples in src/xml/OME).  The XML files can be 
imported at run-time using the normal file import mechanism without 
rebuilding or restarting the database.  The database is extended with 
new tables/columns to accommodate the new definitions.

>
> Details follow.  If these questions are answered in existing 
> documentation, please just provide pointers.
>
>
> 1. How are custom annotations intended to be used?  I have tried to 
> use custom annotations as follows, using the web browser interface:
> - create a dataset and import an image into that dataset
> - I have successfully created "DatasetAnnotation" and 
> "DatasetTestSignature" annotations for the dataset by selecting the 
> corresponding listbox entries and clicking on "Create a custom 
> annotation".  (I don't know if the values I entered make sense, but 
> they have been stored and are retrievable as part of the dataset 
> metadata.)
> - I then tried to use a similar procedure to create a custom 
> annotation for the image.Selecting "ImageExperiment" as the type of 
> custom annotation, I am asked to create an "Experiment" and an 
> "Experimenter", which I duly do, filling in all the proffered fields 
> and clicking the corresponding "Create ..." buttons.
> - The "Image experiment" dialog ends up with an experiment identified 
> by a number ("38" in my case) and an image (the one I am annotating), 
> but nothing that seems to reflect the information just entered.  
> Clicking on "Create image experiment" returns me to the original 
> screen, but it's not obvious that the annotation has been added.  (I 
> now notice that it appears in the "Module execution" list, but not at 
> the top or bottom of the list.  The first time I tried something like 
> this, the new information appears to have been lost.)
>
> My question is:  the above procedures seem non-obvious and error 
> prone; are they consonant with how the annotation feature is intended 
> to be used, or am I overlooking something?

That is one way of creating a more structured tag.  I think the 
difficulty you're having is that this is done in the release using a 
completely general mechanism, whereas a more specific one would 
probably be more useful.  The reason the "Image experiment" dialog is 
not as clear as it can be is because Experiments are global structures 
which can be associated with many images.  An image experiment is a 
mapping between this global experimental description and a specific 
image.  In the Image experiment view, the "38" should be a highlighted 
link to the global "Experiment" object you created.  Clicking on it 
should lead you to a display page with all of the details you entered 
for the experiment.  A more targeted web-based user interface would 
make this a lot more clear while preserving this data-model.  Actually, 
a more general user-interface should probably automatically 
de-reference any references, at least one level deep.


>
>
> 2. How to create new annotation metadata structures?  A simpler 
> question with, I suspect, a more complex answer, but hopefully one 
> that can be addressed by a pointer to relevant documentation.
>
> We aim to capture data in a semi-structured form that can be exported 
> to a semantic web database for further cross-referencing with other 
> information sources.  The built-in custom annotations seem to support 
> the right kind of general structures for this purpose.  Are the form 
> displays linked directly to the XML metadata schema, or is there an 
> additional presentation-description component?  How do I go about 
> adding additional metadata to OME?  Can the metadata format be 
> dynamically updated, or does this require a re-build of the database?

The web-ui mechanism you're using is completely general, so it works 
directly off of the Semantic Type definitions in the DB (which were 
declared originally in XML and imported into OME in the last phase of 
installation).  A mechanism allowing more targeted 
definition/search/display of these things is forthcoming.  This will 
obviously add a separate display component on top of the definition of 
the data model.  The majority of the custom-display mechanism (HTML 
templates) is already in the release (/OME/html/Templates), but it can 
only be customized per-site (i.e. you can change the image display for 
your entire site, but not for a specific user or group).

The best way to figure out how to add additional metadata to OME is to 
use the examples of how the core semantic model for OME is declared.  
We don't have a tutorial how to do this (a contributed tutorial would 
be most welcome!).  You've played with the image 
experiment/experiment/experimenter STs.  You can find their 
declarations here:
Image.ome (contains ImageExperiment among others)
Experiment.ome
Experimenter.ome (contains Experimenters and Groups)

The Semantic Type Definition (STD) Schema is documented here:
http://www.openmicroscopy.org/XMLschemas/STD/latest/STD_xsd/index.html

-Ilya



>
> ...
>
> Thanks for any insights or pointers that anyone can offer.
>
> #g
>
>
> ---
> Graham Klyne
> Image Bioinformatics Research Group (http://www.bioimage.org/)
> Department of Zoology, University of Oxford
> South Parks Road, Oxford OX1 3PS, UK
> E-mail: <Graham.Klyne at zoo.ox.ac.uk>
> Direct phone: +44-(0)1865-281991
> Departmental fax: +44-(0)1865-310447
>
> _______________________________________________
> ome-users mailing list
> ome-users at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>