[ome-devel] OMERO.features API development

[Lee Kamentsky]

Hi all,
Just chiming in, since we were mentioned...

On Wed, Apr 23, 2014 at 5:10 PM, Simon Li <s.p.li at dundee.ac.uk> wrote:

> Hi all
>
> Now that OMERO 5.0 is out of the way, and OMERO.searcher and WND-CHRM are
> either released or very close to release, I think it's time to restart our
> OMERO.features discussions.
>
> We got as far as the idea of a 2D table with any number of key-value pairs
> on each column and row, so for example each row could be as simple as
> (OmeroType: Image, OmeroId: 123), or in the case of features which are a
> function of multiple images or channels (OmeroType: Image, OmeroId: 123,
> Channel1: 0, Channel2: 3), etc. Columns could for example be
> (FeatureFamily: WndCharm, Feature: Zernike). Each table cell could either
> be a scalar or array. Retrieving features could be done by providing
> key-value pairs to be matched.
>
> All of this is still up for discussion, especially since the
> implementation of this interface could be challenging and there's some
> redundancy/ambiguity. Just to be clear, the above is a conceptual
> description of how the API would appear to users, the actual back-end could
> be completely different.
>
> Lee Kamentsky gave us a use case just before Christmas [1], Chris Coletta
> and Ivan Cao-berg are planning to summarise how they see WND-CHARM and
> OMERO.searcher fitting in. I know a few other people are interested in this
> discussion, so feel free to respond here or in the forums.
>

For us, it's important to link features to regions of interest,
specifically segmentations of whole cells and cellular compartments. The
other issues have to do with scalability and the efficiency of retrieving
large data sets either by selecting a few features for a large number of
images (e.g. up to on the order of 1,000,000 images and 1,000 entries per
feature per image) or by selecting many or all features associated with a
subset of the regions of interest.

We are also interested in recording tracking data. What's needed here is
the ability to record a link between the region of interest in one frame of
a time-series stack with a region of interest in a later frame and you need
the flexibility of a many-many relationship to represent cell division and
potentially merging. I'm fairly confident that you could encode that sort
of thing in a 2-D table which had columns referencing both ROIs.

Finally, we try to capture enough information about the analysis to make it
reproducible - things like the pipeline used for the analysis, the GIT hash
of the software used to run the analysis and of each image analyzed. I
think all of that is easily captured, though, in the tables and I doubt we
need any explicit functionality devoted to that. It might be nice to be
able to annotate the table itself with attributes in order to document the
linking of the analysis results to the experimental protocol, but the
linking could be documented using columns in an experiment-wide table.

>
> A few of us are planning to meet up at the OME Paris meeting- if you're
> interested drop me an email.
>
> Thanks
>
> Simon
>
> [1]
> http://lists.openmicroscopy.org.uk/pipermail/ome-devel/2013-November/002573.html
>
>
> On 7 Nov 2013, at 14:20, Simon Li <s.p.li at dundee.ac.uk> wrote:
>
> > Some notes from our meeting yesterday:
> >
> http://www.openmicroscopy.org/site/community/minutes/minigroup/omero-features-meetings/2013-11-06-omero-features-google-hangout
> >
> > Summary:
> > We're thinking of representing features as a 2D array, with metadata
> stored as key-value maps attached to the array, or individual columns or
> rows. These keys could describe things such as the feature name (column),
> sample metadata (row), algorithm parameters, calculation pipelines, etc.
> >
> > This should work as an OMERO API- in order to retrieve features you'd
> pass in a set of key-value pairs, for instance to specify which features
> you want and which images/ROIs etc, and OMERO would handle the logic and
> return the feature table(s) matching those parameters. Since everyone has
> different requirements the keys could be anything, however we're trying to
> define a small set of standard keys- any suggestions are very welcome.
> >
> > Outside of OMERO we still need a format for transporting features, so
> we're thinking some form of HDF5.
> >
> > Simon
> >
> >
> > The University of Dundee is a registered Scottish Charity, No: SC015096
> > _______________________________________________
> > ome-devel mailing list
> > ome-devel at lists.openmicroscopy.org.uk
> > http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>
>
> The University of Dundee is a registered Scottish Charity, No: SC015096
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20140424/71723ee8/attachment.html>