Hi Ghislain,<br><br><div class="gmail_quote"><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div link="blue" vlink="purple" lang="EN-US"><div>
<p><span style="font-size: 11pt; color: black;">I am glad that this is issue is raised as .ext1.ext2 is in no way
a standard extension.</span></p>
<p><span style="font-size: 11pt; color: black;"></span></p></div></div></blockquote><div><br>Not a common practice perhaps, but I am curious what you see as the practical pitfalls of this compound extension. It seems to me that using .ome.tif gives us the best of both worlds: 1) an unambiguous extension for the OME-TIFF format, and 2) compatibility with existing TIFF software.<br>
<br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div link="blue" vlink="purple" lang="EN-US"><div>
<p><span style="font-size: 11pt; color: black;">Perhaps, using simply a .tiff with a format specific IFD would
make more sense.</span></p></div></div></blockquote><div><br>We have discussed this idea in the past. Do you see any advantages to your approach other than those you mention below?<br><br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div link="blue" vlink="purple" lang="EN-US"><div><p><span style="font-size: 11pt; color: black;">In addition, this would allow for a mechanism to transform
Tiff based file formats more efficiently, and provide backward compatibility
with other readers.</span></p>
<p><span style="font-size: 11pt; color: black;"></span></p></div></div></blockquote><div><br>How would a custom IFD entry be more efficient than the current mechanism? Currently you could inject the OME-XML metadata extremely efficiently by surgically overwriting the ImageDescription tag. The only downside is that a TIFF cannot simultaneously be an OME-TIFF and some other flavor of TIFF that also uses the ImageDescription tag for its metadata. A custom IFD entry would alleviate that issue.<br>
<br>How is the current specification not backwardly compatible with other readers? The only way I can think of is if those other readers are also expecting custom metadata from the same ImageDescription tag.<br><br></div>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div link="blue" vlink="purple" lang="EN-US"><div>
<p><span style="font-size: 11pt; color: black;">For instance the Opera .flex file is roughly a Tiff file with some
specific header (and in some cases a specific compression of the pixel data).
One could think that adding the OME-XML header under an OME specific IFD would
make the most sense. While the original IFD would remain unchanged.</span></p>
<p><span style="font-size: 11pt; color: black;"></span></p></div></div></blockquote><div><br>Is your concern that you do not wish to overwrite the Flex file's existing ImageDescription tag? I checked a bunch of our sample Flex files, and none of them have an existing ImageDescription tag. Are you using this field?<br>
<br>The rationale thus far has been that any actual "image description" -- i.e., comment -- can go in the OME-XML block's Description tag. But I understand that there are cases where this solution is undesirable: like I said, if the same field holds metadata in some other structure, it could be a problem. Examples include TIFFs saved by ImageJ, and Leica TCS TIFFs.<br>
<br>Being able to combine these forms of metadata within a single TIFF is of potential benefit. Is there anyone out there who wants to do this, and wants to see the OME-TIFF specification changed?<br><br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div link="blue" vlink="purple" lang="EN-US"><div>
<p><span style="font-size: 11pt; color: black;">On another note and still about metadata. As the metadata can
become extremely large, would it make sense to provide a mechanism to compress
it using deflate for instance? Or is there a mechanism for this?</span></p></div></div></blockquote></div><br>For OME-TIFF, there is no mechanism to compress the XML at the moment. You can do it with the OME-XML file format, using extension .omez, with zlib. However, the metadata is a tiny fraction of the total data size, even when represented in a rather inefficient XML format.<br>
<br>The major goals of OME-TIFF are performance and compatibility. Uncompressed TIFF planes are essentially raw, and storing the metadata in the first ImageDescription tag allows quick access (not to say that a custom tag wouldn't be just as quick). Compressing the XML block would increase the amount of time necessary to parse the metadata, though that is no reason not to allow it as an option. If this option would really significantly reduce the size of your data, we can discuss how best to add such a facility.<br>
<br>
In conclusion, I am resistant to changing the OME-TIFF specification without a
compelling practical benefit. The spec has already seen some breaking
changes that have made it difficult to maintain compatibility within
Bio-Formats, and I would prefer to avoid any further complications in
the implementation. But it would be foolish not to discuss and evaluate any potential benefits to such changes.<br><br>In this case, unless someone in the community would directly benefit from the custom IFD entry approach, I think the advantages are mostly theoretical and would not warrant the disruption caused by changing the specification again. Please feel free to argue if you disagree. :-)<br>
<br>-Curtis<br><br><div class="gmail_quote">On Tue, Jan 13, 2009 at 3:09 PM, Ghislain Bonamy <span dir="ltr"><<a href="mailto:GBonamy@gnf.org">GBonamy@gnf.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div link="blue" vlink="purple" lang="EN-US">
<div>
<p><span style="font-size: 11pt; color: black;">Curtis, Frans,</span></p>
<p><span style="font-size: 11pt; color: black;"> </span></p>
<p><span style="font-size: 11pt; color: black;">I am glad that this is issue is raised as .ext1.ext2 is in no way
a standard extension.</span></p>
<p><span style="font-size: 11pt; color: black;"> </span></p>
<p><span style="font-size: 11pt; color: black;">Perhaps, using simply a .tiff with a format specific IFD would
make more sense. In addition, this would allow for a mechanism to transform
Tiff based file formats more efficiently, and provide backward compatibility
with other readers.</span></p>
<p><span style="font-size: 11pt; color: black;"> </span></p>
<p><span style="font-size: 11pt; color: black;">For instance the Opera .flex file is roughly a Tiff file with some
specific header (and in some cases a specific compression of the pixel data).
One could think that adding the OME-XML header under an OME specific IFD would
make the most sense. While the original IFD would remain unchanged.</span></p>
<p><span style="font-size: 11pt; color: black;"> </span></p>
<p><span style="font-size: 11pt; color: black;">On another note and still about metadata. As the metadata can
become extremely large, would it make sense to provide a mechanism to compress
it using deflate for instance? Or is there a mechanism for this?</span></p>
<p><span style="font-size: 11pt; color: black;"> </span></p>
<p><span style="font-size: 11pt; color: black;">Best,</span></p>
<p><span style="font-size: 11pt; color: black;"> </span></p>
<p style=""><span style="font-size: 11pt; color: black;">Ghislain Bonamy, PhD</span></p>
<p style=""><span style="font-size: 11pt; color: black;">__________________________________________</span></p>
<p style=""><span style="font-size: 11pt; color: black;">Research Investigator I</span></p>
<p style=""><span style="font-size: 11pt; color: black;"> </span></p>
<p style=""><span style="font-size: 11pt; color: black;">Genomic Institute of the</span></p>
<p style=""><span style="font-size: 11pt; color: black;">Novartis Research</span></p>
<p style=""><span style="font-size: 11pt; color: black;">Foundation</span></p>
<p style=""><span style="font-size: 11pt; color: black;">Department of Molecular &
Cell Biology, room G214</span></p>
<p style=""><span style="font-size: 11pt; color: black;">10675 John Jay Hopkins Drive</span></p>
<p style=""><span style="font-size: 11pt; color: black;">San Diego CA 92121</span></p>
<p style=""><span style="font-size: 11pt; color: black;">USA</span></p>
<p style=""><span style="font-size: 11pt; color: black;"> </span></p>
<p style=""><span style="font-size: 11pt; color: black;">+1 (858) 812-1534 (W & F)</span></p>
<p style=""><span style="font-size: 11pt; color: black;">+1 (757) 941-4194 (H)</span></p>
<p style=""><span style="font-size: 11pt; color: black;">+1 (858) 354-7388 (M)</span></p>
<p style=""><span style="font-size: 11pt; color: black;"><a href="http://www.gnf.org" target="_blank">www.gnf.org</a></span></p>
<p style=""><span style="font-size: 11pt; color: black;"> </span></p>
<p style=""><span style="font-size: 11pt; color: black;">Hudson-Alpha Institute for
Biotechnology</span></p>
<p style=""><span style="font-size: 11pt; color: black;"><a href="http://www.haib.org" target="_blank">www.hudsonalpha.org</a></span></p>
<p><span style="font-size: 11pt; color: black;"> </span></p>
<div style="border-style: solid none none; border-color: rgb(181, 196, 223) -moz-use-text-color -moz-use-text-color; border-width: 1pt medium medium; padding: 3pt 0cm 0cm;">
<p><b><span style="font-size: 10pt;">From:</span></b><span style="font-size: 10pt;">
<a href="mailto:ome-devel-bounces@lists.openmicroscopy.org.uk" target="_blank">ome-devel-bounces@lists.openmicroscopy.org.uk</a>
[mailto:<a href="mailto:ome-devel-bounces@lists.openmicroscopy.org.uk" target="_blank">ome-devel-bounces@lists.openmicroscopy.org.uk</a>] <b>On Behalf Of </b>Curtis
Rueden<br>
<b>Sent:</b> Tuesday, January 13, 2009 1:00 PM<br>
<b>To:</b> Cornelissen, Frans [PRDBE]<br>
<b>Cc:</b> <a href="mailto:ome-devel@lists.openmicroscopy.org.uk" target="_blank">ome-devel@lists.openmicroscopy.org.uk</a><br>
<b>Subject:</b> Re: [ome-devel] ome-tiff files: does it really needs to be
namedxx.ome.tif ??</span></p>
</div><div><div></div><div class="Wj3C7c">
<p> </p>
<p style="margin-bottom: 12pt;">Hi Frans,</p>
<div>
<blockquote style="border-style: none none none solid; border-color: -moz-use-text-color -moz-use-text-color -moz-use-text-color rgb(204, 204, 204); border-width: medium medium medium 1pt; padding: 0cm 0cm 0cm 6pt; margin-left: 4.8pt; margin-right: 0cm;">
<p>When using Tiff files, we would like to convert them to
OME-tiff so that<br>
they do contain the OME-XML metadata.<br>
<br>
Currently the new files have to contain the .ome.tiff as extension<br>
In our analysis processes, the altered name causes a disruption.</p>
</blockquote>
<div>
<p style="margin-bottom: 12pt;"><br>
Originally, the specification did not require the .ome.tif extension, but we
decided it would reduce ambiguity to prefer a more specific extension -- and
the .ome.tif extension allows non-OME-aware TIFF programs to continue seeing
the files as regular TIFFs.</p>
</div>
<blockquote style="border-style: none none none solid; border-color: -moz-use-text-color -moz-use-text-color -moz-use-text-color rgb(204, 204, 204); border-width: medium medium medium 1pt; padding: 0cm 0cm 0cm 6pt; margin-left: 4.8pt; margin-right: 0cm;">
<p>Question: is it really a hard requirement that the .ome.
part is in the<br>
filename?</p>
</blockquote>
<div>
<p style="margin-bottom: 12pt;"><br>
At the moment, for Bio-Formats and hence OMERO, yes it is a hard requirement.
We are not necessarily opposed to parsing OME-TIFF metadata out of files
without the .ome.tif extension, but at the moment there are some technical
barriers to doing so efficiently.</p>
</div>
<blockquote style="border-style: none none none solid; border-color: -moz-use-text-color -moz-use-text-color -moz-use-text-color rgb(204, 204, 204); border-width: medium medium medium 1pt; padding: 0cm 0cm 0cm 6pt; margin-left: 4.8pt; margin-right: 0cm;">
<p>This in itself is no proof of the fact that the file
*really* contains a<br>
valid OME-xml structure, so an application is probably going the check<br>
internally to decide whether it is an OME file anyway...</p>
</blockquote>
<div>
<p style="margin-bottom: 12pt;"><br>
True. The same is true for every file extension -- the only way to verify that
the file *really* contains correctly structured data of the indicated type is
to attempt to fully parse it. However, file extension is an extremely useful
hint that greatly improves performance. In some cases (e.g., certain raw data
formats) it might even be impossible to completely determine the file format
without the filename extension.</p>
</div>
<blockquote style="border-style: none none none solid; border-color: -moz-use-text-color -moz-use-text-color -moz-use-text-color rgb(204, 204, 204); border-width: medium medium medium 1pt; padding: 0cm 0cm 0cm 6pt; margin-left: 4.8pt; margin-right: 0cm;">
<p>Could the .ome. extension requirement be removed for
importing ome-tiff<br>
files into OMERO?</p>
</blockquote>
</div>
<p style="margin-bottom: 12pt;"><br>
Yes, we always parse a TIFF file's ImageDescription block. Ideally, we should
be properly parsing any OME-XML we find there. However, as I said, there are
some performance challenges we need to sort out. The fix shouldn't be too bad.
We'll file a ticket to keep you posted.<br>
<br>
-Curtis</p>
<div>
<p>On Mon, Jan 12, 2009 at 8:43 AM, Cornelissen, Frans [PRDBE]
<<a href="mailto:FCORNELI@its.jnj.com" target="_blank">FCORNELI@its.jnj.com</a>>
wrote:</p>
<p>Hi,<br>
<br>
When using Tiff files, we would like to convert them to OME-tiff so that<br>
they do contain the OME-XML metadata.<br>
<br>
Currently the new files have to contain the .ome.tiff as extension<br>
In our analysis processes, the altered name causes a disruption.<br>
<br>
Question: is it really a hard requirement that the .ome. part is in the<br>
filename?<br>
This in itself is no proof of the fact that the file *really* contains a<br>
valid OME-xml structure, so an application is probably going the check<br>
internally to decide whether it is an OME file anyway...<br>
<br>
Could the .ome. extension requirement be removed for importing ome-tiff<br>
files into OMERO?<br>
<br>
Best regards, frans cornelissen<br>
_______________________________________________<br>
ome-devel mailing list<br>
<a href="mailto:ome-devel@lists.openmicroscopy.org.uk" target="_blank">ome-devel@lists.openmicroscopy.org.uk</a><br>
<a href="http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel" target="_blank">http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel</a></p>
</div>
<p> </p>
</div></div></div>
</div>
</blockquote></div><br>