<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
Hi again
<div><br>
</div>
<div>I realise the discussion has moved on now but just for everyone’s information.</div>
<div>I’ve been writing synthetic test FLIM data .into ome-tiffs i.e. with the ModuloAlongT annotation set.</div>
<div><br>
</div>
<div>I have been doing this from Matlab using the bfsave routine in the bfmatlab package & then repeating with </div>
<div>a version of the same code modified to use the ifd as discussed in this thread.(clearing the ifd in the loop)</div>
<div><br>
</div>
<div>The good news is that both files now return the same data when read with bfopen.</div>
<div>They also pass both showinf & xmlvalid , import into an OMERO server (via insight ) and can then be read by FLIMfit </div>
<div>from the server. </div>
<div><br>
</div>
<div><br>
</div>
<div>On the performance side:</div>
<div>Adding the ifd code reduces the file size very slightly from 7401611 to 7344379.</div>
<div>Speed is harder to quantify as the exact time seems to vary but the ifd code does seem to be noticeably faster.</div>
<div><br>
</div>
<div>All of this is on OSX with Matlab 2013b.</div>
<div>It would be good if this could be repeated on Windows.</div>
<div><br>
</div>
<div>Best </div>
<div><br>
</div>
<div>Ian </div>
<div><br>
</div>
<div><br>
</div>
<div> </div>
<div><br>
<div>
<div>On 30 Oct 2014, at 03:25, Yellen, Gary <<a href="mailto:gary_yellen@hms.harvard.edu">gary_yellen@hms.harvard.edu</a>> wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<div lang="EN-US" link="blue" vlink="purple" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<div class="WordSection1" style="page: WordSection1;">
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">Hi All –<o:p></o:p></span></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">I took care of my <Plane> metadata problem and now my Bio-Formats/MATLAB hybrid is read fine by OMERO.<o:p></o:p></span></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">And the writing speed is way faster – about 18x - than the all Bio-Formats version (which still fails OMERO import on some sort of StatsInfo handling error).<o:p></o:p></span></div>
<div style="margin: 0in 0in 0.0001pt 0.5in; font-size: 12pt; font-family: 'Times New Roman', serif; text-indent: -0.25in;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"><span>-<span style="font-style: normal; font-variant: normal; font-weight: normal; font-size: 7pt; line-height: normal; font-family: 'Times New Roman';"> <span class="Apple-converted-space"> </span></span></span></span><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">gary<o:p></o:p></span></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span></div>
<div>
<div style="border-style: solid none none; border-top-color: rgb(181, 196, 223); border-top-width: 1pt; padding: 3pt 0in 0in;">
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<b><span style="font-size: 10pt; font-family: Tahoma, sans-serif;">From:</span></b><span style="font-size: 10pt; font-family: Tahoma, sans-serif;"><span class="Apple-converted-space"> </span>ome-devel [<a href="mailto:ome-devel-bounces@lists.openmicroscopy.org.uk">mailto:ome-devel-bounces@lists.openmicroscopy.org.uk</a>]<span class="Apple-converted-space"> </span><b>On
Behalf Of<span class="Apple-converted-space"> </span></b>Yellen, Gary<br>
<b>Sent:</b><span class="Apple-converted-space"> </span>Wednesday, October 29, 2014 6:12 PM<br>
<b>To:</b><span class="Apple-converted-space"> </span>Besson, Sebastien (forwarding)<br>
<b>Cc:</b><span class="Apple-converted-space"> </span>OME-devel OME<br>
<b>Subject:</b><span class="Apple-converted-space"> </span>Re: [ome-devel] OME-Tiff writing using LOCI calls from MATLAB, and ModuloAlongT annotation<o:p></o:p></span></div>
</div>
</div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<o:p> </o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">Hi Sebastien,<o:p></o:p></span></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">I tried this – the file is readable in ImageJ but not in OMERO.<o:p></o:p></span></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">It gives a different error (below), and the writing time is now 30 seconds.<o:p></o:p></span></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">Best,<o:p></o:p></span></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">Gary<o:p></o:p></span></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">ame: import-request-failure<o:p></o:p></span></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">Parameters: {message=omero.ValidationException<o:p></o:p></span></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> serverStackTrace = "ome.conditions.ValidationException: could not insert: [ome.model.stats.StatsInfo]; SQL [insert into statsinfo (creation_id, external_id, group_id,
owner_id, permissions, update_id, globalMax, globalMin, version, id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]; nested exception is org.hibernate.exception.DataException: could not insert: [ome.model.stats.StatsInfo]<o:p></o:p></span></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> at org.springframework.orm.hibernate3.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:642) …<o:p></o:p></span></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span></div>
<div>
<div style="border-style: solid none none; border-top-color: rgb(181, 196, 223); border-top-width: 1pt; padding: 3pt 0in 0in;">
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<b><span style="font-size: 10pt; font-family: Tahoma, sans-serif;">From:</span></b><span style="font-size: 10pt; font-family: Tahoma, sans-serif;"><span class="Apple-converted-space"> </span>Sebastien Besson [<a href="mailto:s.besson@dundee.ac.uk" style="color: purple; text-decoration: underline;">mailto:s.besson@dundee.ac.uk</a>]<span class="Apple-converted-space"> </span><br>
<b>Sent:</b><span class="Apple-converted-space"> </span>Wednesday, October 29, 2014 5:28 PM<br>
<b>To:</b><span class="Apple-converted-space"> </span>Yellen, Gary<br>
<b>Cc:</b><span class="Apple-converted-space"> </span>Munro Ian; OME-devel OME<br>
<b>Subject:</b><span class="Apple-converted-space"> </span>Re: [ome-devel] OME-Tiff writing using LOCI calls from MATLAB, and ModuloAlongT annotation<o:p></o:p></span></div>
</div>
</div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<o:p> </o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
Hi Gary,<o:p></o:p></div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<o:p> </o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
after a bit of debugging, the error should be solved by reinitializing the IFD state before <o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
each call to TiffWriter.saveBytes(). This should be solved by either creating a new<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
IFD in each iteration of the loop or resetting its state with for instance:<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<o:p> </o:p></div>
</div>
<div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 7.5pt; font-family: Courier;"> ifd.clear();<o:p></o:p></span></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 7.5pt; font-family: Courier;"> ifd.putIFDValue(ifd.ROWS_PER_STRIP, szY);<o:p></o:p></span></div>
</div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<o:p> </o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
Adding this two lines was enough to remove the Error thrown by running showinf on the<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
generated OME-TIFF files while preserving the size. I would expect this will also solve the<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
import into OMERO.<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<o:p> </o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
Could you try and let us know?<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
Sebastien<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<o:p> </o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<o:p> </o:p></div>
<div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
On 29 Oct 2014, at 20:57, Yellen, Gary <<a href="mailto:gary_yellen@hms.harvard.edu" style="color: purple; text-decoration: underline;">gary_yellen@hms.harvard.edu</a>> wrote:<o:p></o:p></div>
</div>
<p class="MsoNormal" style="margin: 0in 0in 12pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<o:p> </o:p></p>
<div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">I finally got to try uploading my files to an OMERO server, and I see the same problem as Ian. The metadata import ok, but the images are no good and the Import Log is
full of “Error untangling IFDs; the OME-TIFF file may be malformed” messages.</span><o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span><o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">OMERO can read the slow (70 sec to write) and oversized (5 MB instead of 1.6) file written with Bio-Formats just fine.</span><o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span><o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">Fiji can read the hybrid file (Bio-Formats for metadata, MATLAB for Tiff writing) just fine, but OMERO has a number of problems with it – the nominally compliant use of
a simplified <TiffData IFD=”0” PlaneCount=”90”> is not handled by OMERO, which seems to insist on have TheC etc. specified for every single plane.</span><o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span><o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">Gary</span><o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span><o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"> </span><o:p></o:p></div>
</div>
<div>
<div style="border-style: solid none none; border-top-color: rgb(181, 196, 223); border-top-width: 1pt; padding: 3pt 0in 0in;">
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<b><span style="font-size: 10pt; font-family: Tahoma, sans-serif;">From:</span></b><span class="apple-converted-space"><span style="font-size: 10pt; font-family: Tahoma, sans-serif;"> </span></span><span style="font-size: 10pt; font-family: Tahoma, sans-serif;">ome-devel
[<a href="mailto:ome-devel-bounces@lists.openmicroscopy.org.uk" style="color: purple; text-decoration: underline;">mailto:ome-devel-bounces@lists.openmicroscopy.org.uk</a>]<span class="apple-converted-space"> </span><b>On Behalf Of<span class="apple-converted-space"> </span></b>Munro,
Ian<br>
<b>Sent:</b><span class="apple-converted-space"> </span>Wednesday, October 29, 2014 10:26 AM<br>
<b>To:</b><span class="apple-converted-space"> </span>OME OME-devel<br>
<b>Subject:</b><span class="apple-converted-space"> </span>Re: [ome-devel] OME-Tiff writing using LOCI calls from MATLAB, and ModuloAlongT annotation</span><o:p></o:p></div>
</div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
Hi All<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
I’m just in the process of checking this to ensure that the data that is read back is the same both with and without this modification.<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
If I run the command line 'showinf tool 'on the smaller files produced with the suggested ifd mod then I get the following in the output:<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<o:p></o:p></div>
</div>
<div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
Reading pixel data (0-2)<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
Reading IFDs<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
Populating metadata<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
Error untangling IFDs; the OME-TIFF file may be malformed.<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
Error untangling IFDs; the OME-TIFF file may be malformed.<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
Read 3/3 planes (100%)<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
[done]<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
Is anyone else seeing this. I may have implemented the ifd mod incorrectly?<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
Regards<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<o:p></o:p></div>
</div>
<div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
Ian <o:p></o:p></div>
</div>
</div>
</div>
</div>
</div>
<div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: 'Times New Roman', serif;">
<o:p> </o:p></div>
</div>
_______________________________________________<br>
ome-devel mailing list<br>
<a href="mailto:ome-devel@lists.openmicroscopy.org.uk" style="color: purple; text-decoration: underline;">ome-devel@lists.openmicroscopy.org.uk</a><br>
<a href="http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel" style="color: purple; text-decoration: underline;">http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel</a></div>
</blockquote>
</div>
<br>
</div>
</body>
</html>