Hi everyone,<br><br>Rubén wrote:<br><blockquote style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;" class="gmail_quote">- The master/slave approach. All files will reference the one that contains the complete headers.<div>
-
"<Plate>", "<Image>" and "<Pixel>" elements could be
grouped when similar (e.g. reg. expressions following a pattern)</div><div> - The "<Plate>", "<Image>" and "<Pixel>" could be extracted to a separate file.</div><div><br></div>
<div>The first alternative was supported by Andrew. I suggested the second, but the project philosophy is opposite to the third. <br></div></blockquote><br>I like the regular expressions idea, but intuitively I am not sure how to structure things as valid XML.<br>
<br>Personally I am in favor of allowing storage of the metadata in a separate file. It also makes it easier for people to write simple scripts in various environments to parse the metadata, since it will be in plaintext rather than embedded in a TIFF.<br>
<br>Alessandro wrote:<br><blockquote style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;" class="gmail_quote">Does it make sense to add a level of redundancy like, for example, one
in ten files has to carry the complete headers, in order to avoid the
loss of metadata info if the master file got deleted/corrupted/abducted
by aliens ?<br></blockquote>
<br>For large numbers of files, I think any mandated level of redundancy will still result in an undesirable increase in size.<br><br>-Curtis<br><br><div class="gmail_quote">On Wed, Dec 8, 2010 at 9:00 AM, Alessandro Dellavedova <span dir="ltr"><<a href="mailto:alessandro.dellavedova@ifom-ieo-campus.it">alessandro.dellavedova@ifom-ieo-campus.it</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">Hi Rubén and list,<br>
<div class="im"><br>
> Some options to simplify the format have ben discussed as follows:<br>
><br>
> - The master/slave approach. All files will reference the one that contains the complete headers.<br>
<br>
</div>Does it make sense to add a level of redundancy like, for example, one in ten files has to carry the complete headers, in order to avoid the loss of metadata info if the master file got deleted/corrupted/abducted by aliens ?<br>
<br>
Best,<br>
<br>
Alessandro<br>
<br>
_______________________________________________<br>
ome-devel mailing list<br>
<div class="im"><a href="mailto:ome-devel@lists.openmicroscopy.org.uk">ome-devel@lists.openmicroscopy.org.uk</a><br>
</div><a href="http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel" target="_blank">http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel</a><br>
</blockquote></div><br><div class="gmail_quote">On Wed, Dec 8, 2010 at 6:58 AM, Rubén Muñoz <span dir="ltr"><<a href="mailto:ruben.munoz@embl.de">ruben.munoz@embl.de</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div style="word-wrap: break-word;">Hi Andrew and list subscribers, <div><br></div><div>I
have some comments to add regarding the OME.TIFF and OME.XML
requirements for changes. The current description of our issue is:</div><div><span style="font-family: 'Lucida Grande',Arial,sans-serif; font-size: 12px; line-height: 18px;"><pre style="margin: 10px 5px; padding: 6px; border: 1px solid rgb(221, 221, 221); background-color: rgb(248, 248, 248); font-family: Monaco,monospace;">
* EMBL Screening (Ruben Muñoz, Jan Ellenberg)<br> * Not duplicating XML for each field, _plane_, etc.</pre></span></div><div><span style="font-family: 'Lucida Grande',Arial,sans-serif; font-size: 12px; line-height: 18px;"></span></div>
<div>I would like to add that our use case, will apply to each user of the OME.TIF multi-file export option. </div><div><br></div><div>We
previously pointed out that the number of planes that are stored per
OME-TIFF has a big impact in each file's size. For multi-file datasets,
the conversion output will be exponentially bigger than the raw data.</div><div><br></div><div>At
EMBL-Heidelberg HCS Facility, we have used this as internal standard,
with the pre-requisite of having one single plane per file. </div><div>The
reasons to do that can be summarized: gives maximum compatibility with
software for image processing, online control of the microscope and
visualization, even after instrument/power failure. This software
includes in-house developments: CellCognition, Micropilot, Cellbase and
3rd-party projects: CellProfiler, Image J/FIJI.</div><div><br></div><div>Given
this scenario we found OME.TIF convenient because it has the correct
conversion tools and an evolving metadata structure, in addition the
commercial adoption of the format is growing.</div><div><br></div><div>In
the practice, a lot of the metadata consist in "<Plate>",
"<Image>" and "<Pixel>" elements (describing the SPW,
dimensionally and the references to the files in the set). </div><div><br></div><div>That can be prohibitive at the processing and the storage stage.</div><div>Some options to simplify the format have ben discussed as follows:</div>
<div><br></div><div> - The master/slave approach. All files will reference the one that contains the complete headers.</div><div> -
"<Plate>", "<Image>" and "<Pixel>" elements could be
grouped when similar (e.g. reg. expressions following a pattern)</div><div> - The "<Plate>", "<Image>" and "<Pixel>" could be extracted to a separate file.</div><div><br></div>
<div>The first alternative was supported by Andrew. I suggested the second, but the project philosophy is opposite to the third. </div><div><br></div><div>Are there other suggestions? I would like to keep this discussion open and to help to define more details if needed.</div>
<div><br></div><div>Best,</div><div>Rubén</div><div><br><div><div>On Dec 7, 2010, at 4:03 PM, <a href="mailto:ome-devel-request@lists.openmicroscopy.org.uk" target="_blank">ome-devel-request@lists.openmicroscopy.org.uk</a> wrote:</div>
<blockquote type="cite"><div><font color="#000000"><br></font>Date: Tue, 7 Dec 2010 13:07:49 +0000<br>From: Andrew Patterson <<a href="mailto:ajpatterson@lifesci.dundee.ac.uk" target="_blank">ajpatterson@lifesci.dundee.ac.uk</a>><br>
To: <a href="mailto:ome-devel@lists.openmicroscopy.org.uk" target="_blank">ome-devel@lists.openmicroscopy.org.uk</a>,<br><span style="white-space: pre-wrap;"> </span><a href="mailto:ome-users@lists.openmicroscopy.org.uk" target="_blank">ome-users@lists.openmicroscopy.org.uk</a><br>
Subject: [ome-devel] OME-XML Updates<br>Message-ID:<br><span style="white-space: pre-wrap;"> </span><<a href="mailto:B5B2766B-2357-40C1-B1DD-06CCEC3A62C9@lifesci.dundee.ac.uk" target="_blank">B5B2766B-2357-40C1-B1DD-06CCEC3A62C9@lifesci.dundee.ac.uk</a>><br>
Content-Type: text/plain; charset=us-ascii<br><br>Hello OME-XML & OME-TIFF users and potential users,<br><br>We
are in the process of compiling requirements for changes to the way our
OME-XML and OME-TIFF formats work. This is in response to the new ways
people are wanting to use our formats, and drawbacks they have come
across when storing datasets in certain circumstances.<br><br>Examples we have so far include:<br>
* storing large datasets, one plane per OME-TIFF: this is a valid way
to want to store data, but one which at the moment causes metadata
duplication on disk.<br> * creating a 'lite' OME-TIFF for display or to pass to external applications.<br><br>A full list of our current thoughts is on the requirement ticket:<br><a href="http://trac.openmicroscopy.org.uk/omero/ticket/3535" target="_blank">http://trac.openmicroscopy.org.uk/omero/ticket/3535</a><br>
<br>Some
of these changes may effect key features of our formats, e.g. our
current insistence that all matadata is stored in the same file as the
image data. <br><br>We would really like to have your input on this feature, or any others.<br><br>If
you have a use case that you think would help guide out future work we
would love to hear from you. If you can reply on either of the mailing
lists (OME-USER or OME-DEVEL), it will let others see and join in!<br><br>Thanks again for your help and support. <br><br>Cheers,<br><br>Andrew<br><br>--<br>Andrew Patterson<br><a href="mailto:ajpatterson@lifesci.dundee.ac.uk" target="_blank">ajpatterson@lifesci.dundee.ac.uk</a><br>
Software Developer, Open Microscopy Environment<br>Wellcome Trust Centre for Gene Regulation & Expression, University of Dundee<br></div></blockquote></div><br></div></div><br>_______________________________________________<br>
ome-devel mailing list<br>
<a href="mailto:ome-devel@lists.openmicroscopy.org.uk">ome-devel@lists.openmicroscopy.org.uk</a><br>
<a href="http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel" target="_blank">http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel</a><br>
<br></blockquote></div><br><br>