[ome-devel] ome-devel Digest, Vol 81, Issue 5

Wed Dec 8 16:51:06 GMT 2010

Hi everyone,

Rubén wrote:

> - The master/slave approach. All files will reference the one that contains
> the complete headers.
>  - "<Plate>", "<Image>" and "<Pixel>" elements could be grouped when
> similar (e.g. reg. expressions following a pattern)
>  - The "<Plate>", "<Image>" and "<Pixel>"  could be extracted to a separate
> file.
>
> The first alternative was supported by Andrew. I suggested the second, but
> the project philosophy is opposite to the third.
>

I like the regular expressions idea, but intuitively I am not sure how to
structure things as valid XML.

Personally I am in favor of allowing storage of the metadata in a separate
file. It also makes it easier for people to write simple scripts in various
environments to parse the metadata, since it will be in plaintext rather
than embedded in a TIFF.

Alessandro wrote:

> Does it make sense to add a level of redundancy like, for example, one in
> ten files has to carry the complete headers, in order to avoid the loss of
> metadata info if the master file got deleted/corrupted/abducted by aliens ?
>

For large numbers of files, I think any mandated level of redundancy will
still result in an undesirable increase in size.

-Curtis

On Wed, Dec 8, 2010 at 9:00 AM, Alessandro Dellavedova <
alessandro.dellavedova at ifom-ieo-campus.it> wrote:

> Hi Rubén and list,
>
> > Some options to simplify the format have ben discussed as follows:
> >
> >  - The master/slave approach. All files will reference the one that
> contains the complete headers.
>
> Does it make sense to add a level of redundancy like, for example, one in
> ten files has to carry the complete headers, in order to avoid the loss of
> metadata info if the master file got deleted/corrupted/abducted by aliens ?
>
> Best,
>
> Alessandro
>
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>

On Wed, Dec 8, 2010 at 6:58 AM, Rubén Muñoz <ruben.munoz at embl.de> wrote:

> Hi Andrew and list subscribers,
>
> I have some comments to add regarding the OME.TIFF and OME.XML requirements
> for changes. The current description of our issue is:
>
>
> * EMBL Screening (Ruben Muñoz, Jan Ellenberg)
>         * Not duplicating XML for each field, _plane_, etc.
>
> I would like to add that our use case, will apply to each user of the
> OME.TIF multi-file export option.
>
> We previously pointed out that the number of planes that are stored per
> OME-TIFF has a big impact in each file's size. For multi-file datasets, the
> conversion output will be exponentially bigger than the raw data.
>
> At EMBL-Heidelberg HCS Facility, we have used this as internal standard,
> with the pre-requisite of having one single plane per file.
> The reasons to do that can be summarized:  gives maximum compatibility with
> software for image processing, online control of the microscope and
> visualization, even after instrument/power failure. This software
> includes in-house developments: CellCognition, Micropilot, Cellbase and
> 3rd-party projects: CellProfiler, Image J/FIJI.
>
> Given this scenario we found OME.TIF convenient because it has the correct
> conversion tools and an evolving metadata structure, in addition the
> commercial adoption of the format is growing.
>
> In the practice, a lot of the metadata consist in "<Plate>", "<Image>" and
> "<Pixel>" elements (describing the SPW, dimensionally and the references to
> the files in the set).
>
> That can be prohibitive at the processing and the storage stage.
> Some options to simplify the format have ben discussed as follows:
>
>  - The master/slave approach. All files will reference the one that
> contains the complete headers.
>  - "<Plate>", "<Image>" and "<Pixel>" elements could be grouped when
> similar (e.g. reg. expressions following a pattern)
>  - The "<Plate>", "<Image>" and "<Pixel>"  could be extracted to a separate
> file.
>
> The first alternative was supported by Andrew. I suggested the second, but
> the project philosophy is opposite to the third.
>
> Are there other suggestions? I would like to keep this discussion open and
> to help to define more details if needed.
>
> Best,
> Rubén
>
> On Dec 7, 2010, at 4:03 PM, ome-devel-request at lists.openmicroscopy.org.ukwrote:
>
>
> Date: Tue, 7 Dec 2010 13:07:49 +0000
> From: Andrew Patterson <ajpatterson at lifesci.dundee.ac.uk>
> To: ome-devel at lists.openmicroscopy.org.uk,
> ome-users at lists.openmicroscopy.org.uk
> Subject: [ome-devel] OME-XML Updates
> Message-ID:
> <B5B2766B-2357-40C1-B1DD-06CCEC3A62C9 at lifesci.dundee.ac.uk>
> Content-Type: text/plain; charset=us-ascii
>
> Hello OME-XML & OME-TIFF users and potential users,
>
> We are in the process of compiling requirements for changes to the way our
> OME-XML and OME-TIFF formats work. This is in response to the new ways
> people are wanting to use our formats, and drawbacks they have come across
> when storing datasets in certain circumstances.
>
> Examples we have so far include:
> * storing large datasets, one plane per OME-TIFF: this is a valid way to
> want to store data, but one which at the moment causes metadata duplication
> on disk.
> * creating a 'lite' OME-TIFF for display or to pass to external
> applications.
>
> A full list of our current thoughts is on the requirement ticket:
> http://trac.openmicroscopy.org.uk/omero/ticket/3535
>
> Some of these changes may effect key features of our formats, e.g. our
> current insistence that all matadata is stored in the same file as the image
> data.
>
> We would really like to have your input on this feature, or any others.
>
> If you have a use case that you think would help guide out future work we
> would love to hear from you. If you can reply on either of the mailing lists
> (OME-USER or OME-DEVEL), it will let others see and join in!
>
> Thanks again for your help and support.
>
> Cheers,
>
> Andrew
>
> --
> Andrew Patterson
> ajpatterson at lifesci.dundee.ac.uk
> Software Developer, Open Microscopy Environment
> Wellcome Trust Centre for Gene Regulation & Expression, University of
> Dundee
>
>
>
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20101208/4f7af698/attachment-0001.html>