<div dir="ltr">Hi Jimmy & everyone,<div><br></div><div><br></div><div>== SamplesPerPixel vs. Integration ==</div><div><br></div><div>Jimmy wrote:</div><div><div>> we often average 3 raw samples together to form 1 pixel value, and</div><div>> only write the single pixel value to Tiff. We would like our Prairie</div><div>> XML file to still reflect SamplesPerPixel = 3 so users could go back</div><div>> and reference that value.</div></div><div><br></div><div>The term "SamplesPerPixel" has a very technical meaning in TIFF and OME-XML: it is the number of sample values actually recorded in the file. Since there is only one sample value recorded per pixel, SamplesPerPixel must equal 1 for this data.</div><div><br></div><div>Conversely, what you describe is, I think, what the OME schema refers to as "integration" [1]. It's not exactly "frame" averaging, but seems conceptually equivalent. Andrew, would you agree?</div><div><br></div><div><br></div><div>== Prairie metadata parsing bugs ==</div><div><br></div><div>Upon further investigation, there were more bugs in the Prairie metadata parsing relating to subindexed values. I think I have them all fixed now [2]. I also added regression tests to ensure things work as expected [3].</div><div><br></div><div><br></div><div>== Prairie vs. OME-TIFF format detection ==</div><div><br></div><div>I wrote:</div><div><div>> if you select the .xml or .env file, the dataset will be detected as</div><div>> "Prairie TIFF" format, whereas if you select one of the .ome.tif</div><div>> files, it will be detected as "OME-TIFF" format.</div></div><div><br></div><div>Actually, there was an additional wrinkle. Selecting the first OME-TIFF file (i.e., the one with a fully specified OME-XML block) detected as OME-TIFF. But selecting any other OME-TIFF file (i.e., one that uses the "BinaryOnly" mechanism to point to the first file) detected as a Prairie TIFF. I have now fixed that bug [4], so that selecting any OME-TIFF file results in the dataset being detected as OME-TIFF.</div><div><br></div><div><br></div><div>== SizeC inconsistency across multiple series ==</div><div><br></div><div>I wrote:</div><div></div><div>> I think it is indeed a bug in the Bio-Formats OME-TIFF reader that the<br></div><div><div>> first series is marked with "SizeC=1" but subsequent ones are "SizeC=5</div><div>> (effectively 1)". I will investigate if/how we can fix that on our</div><div>> end.</div></div><div><br></div><div>I did some digging, and found the issue [5]. It seems there is a hardcoded "i == 0" test, which I am reluctant to change, since doing so might have substantial ramifications. But I do still think we should somehow fix this behavior.</div><div><br></div><div>Melissa, what do you think? I uploaded a sample dataset to QA [6].</div><div><br></div><div><br></div><div>== Support for spectral datasets ==</div><div><br></div><div class="gmail_extra"><div>I wrote:</div><div></div><div class="gmail_extra">> Next on my list is to finish adding support for spectral datasets.</div><div class="gmail_extra">> Will keep everyone posted, and file a PR as soon as it is ready.</div><div><br></div><div>Done [7]. And PR filed [8].</div><div><br></div><div>Latest build available for testing from:</div><div><a href="http://curtis.imagej.net/prairie-view-5/">http://curtis.imagej.net/prairie-view-5/</a></div><div><br></div></div><div class="gmail_extra">Regards,</div><div class="gmail_extra">Curtis</div><div class="gmail_extra"><br></div><div class="gmail_extra">[1] <a href="https://www.openmicroscopy.org/Schemas/Documentation/Generated/OME-2013-06/ome_xsd.html#DetectorSettings_Integration">https://www.openmicroscopy.org/Schemas/Documentation/Generated/OME-2013-06/ome_xsd.html#DetectorSettings_Integration</a></div><div><br></div><div class="gmail_extra"><div>[2] <a href="https://github.com/ctrueden/bioformats/commit/073ddc27a69fa089880e26c9bd53d5f6bac64d1b">https://github.com/ctrueden/bioformats/commit/073ddc27a69fa089880e26c9bd53d5f6bac64d1b</a></div><div><br></div><div>[3] <a href="https://github.com/ctrueden/bioformats/commit/7124247e55484b2f77ba8766d51455efd3f1d1e4">https://github.com/ctrueden/bioformats/commit/7124247e55484b2f77ba8766d51455efd3f1d1e4</a><br></div><div><br></div><div>(Note to programmers: always write regression tests! I discovered and fixed two additional bugs in the Prairie metadata logic, thanks to the creation of these tests.)<br></div><div><br></div><div>[4] <a href="https://github.com/ctrueden/bioformats/commit/a00d1251cd7bbfa6eef8329777f788c7d242df83">https://github.com/ctrueden/bioformats/commit/a00d1251cd7bbfa6eef8329777f788c7d242df83</a></div></div><div><br></div><div>[5] <a href="https://github.com/openmicroscopy/bioformats/blob/v5.0.5/components/formats-bsd/src/loci/formats/in/OMETiffReader.java#L573-L590">https://github.com/openmicroscopy/bioformats/blob/v5.0.5/components/formats-bsd/src/loci/formats/in/OMETiffReader.java#L573-L590</a></div><div><br></div><div>[6] <a href="http://qa.openmicroscopy.org.uk/qa/feedback/10297/">http://qa.openmicroscopy.org.uk/qa/feedback/10297/</a></div><div><br></div>[7] <a href="https://github.com/ctrueden/bioformats/commit/f9b6ac38bb7f71b2dc9cfc709b7321d9884c99a9">https://github.com/ctrueden/bioformats/commit/f9b6ac38bb7f71b2dc9cfc709b7321d9884c99a9</a> <div class="gmail_extra"><br></div><div class="gmail_extra">[8] <a href="https://github.com/openmicroscopy/bioformats/pull/1365">https://github.com/openmicroscopy/bioformats/pull/1365</a></div><div class="gmail_extra"><br></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Oct 10, 2014 at 2:53 PM, Fong, Jimmy <span dir="ltr"><<a href="mailto:Jimmy.Fong@bruker-nano.com" target="_blank">Jimmy.Fong@bruker-nano.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div>
<div style="direction:ltr;font-family:Tahoma;color:rgb(0,0,0);font-size:10pt">Hi Curtis,<br>
<br>
We appreciate all your work. Just as some clarification, the SamplesPerPixel field in our metadata corresponds to the number of samples (off of our acquisition hardware) that were already binned together to form the pixel value. We do not provide the ability
to write out each individual sample in the Tiff data, so it would make sense that the Bio-Formats Prairie TIFF reader would always set the SamplesPerPixel field in the OME metadata to 1, regardless of the value of SamplesPerPixels field in the Prairie metadata.
<br>
<br>
For example, with our resonant scanner, we often average 3 raw samples together to form 1 pixel value, and only write the single pixel value to Tiff. We would like our Prairie XML file to still reflect SamplesPerPixel = 3 so users could go back and reference
that value.<br>
<br>
Thanks again and let me know if I'm just interpreting your email incorrectly.<br>
<br>
Regards,<span><br>
Jimmy<br>
<div><br>
<div style="font-size:13px;font-family:Tahoma">
<div>
<p class="MsoNormal"><b><span style="font-size:8pt;font-family:Arial,sans-serif">Jimmy Fong</span></b></p>
<p class="MsoNormal"><b><span style="font-size:8pt;font-family:Arial,sans-serif"></span></b><span style="font-size:8pt;font-family:Arial,sans-serif">Software Engineer</span></p>
<p class="MsoNormal"><span style="font-size:8pt;font-family:Arial,sans-serif"></span> </p>
<p class="MsoNormal"><span style="font-size:8pt;font-family:Arial,sans-serif">Bruker Nano Surfaces Division
</span><a href="mailto:Jimmy.Fong@bruker-nano.com" target="_blank"><span style="font-size:8pt;font-family:Arial,sans-serif"><font color="#0000ff">Jimmy.Fong@bruker-nano.com</font></span></a><span style="font-size:8pt;font-family:Arial,sans-serif"></span></p>
<p class="MsoNormal"><span style="font-size:8pt;font-family:Arial,sans-serif">3030 Laura Lane, Suite 140
</span><a href="https://remote.bruker-nano.com/itweb/www.bruker.com/nano" target="_blank"><span style="font-size:8pt;font-family:Arial,sans-serif"><font color="#0000ff">www.bruker.com/nano</font></span></a><span style="font-size:8pt;font-family:Arial,sans-serif"></span></p>
<p class="MsoNormal"><span style="font-size:8pt;font-family:Arial,sans-serif">Middleton, WI 53562-0677</span></p>
<p class="MsoNormal"><span style="font-size:8pt;font-family:Arial,sans-serif">Phone: <a href="tel:%2B1%20608-662-0022%20x163" value="+16086620022" target="_blank">+1 608-662-0022 x163</a></span></p>
<div class="MsoNormal"><span style="color:rgb(31,73,125)">
<hr style="width:6.25in" align="left" noshade size="8" width="600">
</span></div>
<p class="MsoNormal"><b><i><span style="font-family:Arial,sans-serif;color:rgb(31,73,125)">We are now Bruker!</span></i></b></p>
<p class="MsoNormal"><span><b><i><span style="font-family:Arial,sans-serif;color:rgb(31,73,125)">Same great products.</span></i></b></span><b><i><span style="font-family:Arial,sans-serif;color:rgb(31,73,125)">
<span>Same great technology.</span></span></i></b></p>
<p class="MsoNormal"><span><b><i><span style="font-family:Arial,sans-serif;color:rgb(31,73,125)">Same great people.</span></i></b></span><b><i><span style="font-family:Arial,sans-serif;color:rgb(31,73,125)">
<span>New great name.</span></span></i></b></p>
<p class="MsoNormal"> </p>
</div>
</div>
</div>
</span><div style="font-family:'Times New Roman';color:rgb(0,0,0);font-size:16px">
<hr>
<div style="direction:ltr"><font color="#000000" face="Tahoma"><b>From:</b> <a href="mailto:ctrueden.wisc@gmail.com" target="_blank">ctrueden.wisc@gmail.com</a> [<a href="mailto:ctrueden.wisc@gmail.com" target="_blank">ctrueden.wisc@gmail.com</a>] on behalf of Curtis Rueden [<a href="mailto:ctrueden@wisc.edu" target="_blank">ctrueden@wisc.edu</a>]<br>
<b>Sent:</b> Thursday, October 09, 2014 5:21 PM<br>
<b>To:</b> Wussow, Mike; Fong, Jimmy<br>
<b>Cc:</b> <a href="mailto:j.r.swedlow@dundee.ac.uk" target="_blank">j.r.swedlow@dundee.ac.uk</a>; <a href="mailto:ome-devel@lists.openmicroscopy.org.uk" target="_blank">ome-devel@lists.openmicroscopy.org.uk</a>; Staniszewski, Kevin<div><div><br>
<b>Subject:</b> Re: [ome-devel] Prairie View XML changes for upcoming v5.2 Release<br>
</div></div></font><br>
</div><div><div>
<div></div>
<div>
<div dir="ltr">Hi Mike, Jimmy, Kevin and everyone,
<div><br>
</div>
<div>
<div>> We'll test the reader when it's released and keep you posted about</div>
<div>> what we find.</div>
</div>
<div><br>
</div>
<div>Thanks for getting back to me (via private mail) with your findings regarding the new PrairieView 5.2 file format reader. I am replying to the public thread to keep others in the loop regarding this work.</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div>== Reading the files as a PrairieView dataset ==</div>
<div><br>
</div>
<div>It is important to note that if you select the .xml or .env file, the dataset will be detected as "Prairie TIFF" format, whereas if you select one of the .ome.tif files, it will be detected as "OME-TIFF" format. Completely separate reader code is used
in each case. The recent changes I made to the PrairieReader [3] only affect the former case.</div>
<div><br>
</div>
<div>In short: please select the .xml or .env file if you wish to use the Bio-Formats Prairie TIFF reader, rather than the OME-TIFF reader.<br>
</div>
<div><br>
</div>
</div>
<div><br>
</div>
<div>== SamplesPerPixel issue when reading as OME-TIFF ==</div>
<div><br>
</div>
<div>Kevin Staniszewski wrote:</div>
<div>
<div>> 1. The "SizeC" attribute is incorrect for all but the first frame.</div>
<div>> This acquisition only contained one channel, but every frame after the</div>
<div>> first reports 5 channels. It appears that the number 5 is coming from</div>
<div>> the "SamplesPerPixel" attribute, as in a separate data set, both</div>
<div>> values reported 6 (using a slightly longer dwell time).</div>
<div>></div>
<div>
<div>
<div>> 2. The "SamplesPerPixel" attribute is incorrect for only the first</div>
<div>> frame. The correct value for this acquisition is 5, but the first</div>
<div>> frame reports a value of 1. The value of 1 may be coming from the</div>
<div>> number of channels in the acquisition.</div>
</div>
</div>
<div><br>
</div>
<div>Actually, SamplesPerPixel needs to be set to 1 for the dataset you sent. From the OME-XML schema documentation [1]:</div>
<div><br>
</div>
<div>
<div> The number of samples the detector takes to form each pixel value.</div>
<div> [units:none] Note: This is not the same as "Frame Averaging" - see</div>
<div> Integration in DetectorSettings</div>
</div>
<div><br>
</div>
<div>SamplesPerPixel is the actual number of recorded sample values per pixel. If you use the "showinf" Bio-Formats command line tool on the dataset, the following warning is shown:</div>
<div><br>
</div>
<div> SamplesPerPixel mismatch: OME=5, TIFF=1</div>
<div><br>
</div>
<div>This indicates that the OME-XML indicates SamplesPerPixel=5, but the TIFF file only contains 1 sample value per pixel (as indicated by the IFD's SamplesPerPixel directory entry—which in this case is absent and hence defaults to 1 according to page 39 of
the TIFF specification [2]).</div>
</div>
<div><br>
</div>
<div>All of that said: I think it is indeed a bug in the Bio-Formats OME-TIFF reader that the first series is marked with "SizeC=1" but subsequent ones are "SizeC=5 (effectively 1)". I will investigate if/how we can fix that on our end.</div>
<div><br>
</div>
<div><br>
</div>
<div>== Failure to extract (X, Y, Z) stage positions ==<br>
</div>
<div><br>
</div>
<div>
<div>Kevin Staniszewski wrote:</div>
<div></div>
</div>
<div>
<div>> The experiment that generated this data was a T series that took one</div>
<div>> image at all XY locations. If I use the bio-formats importer to look</div>
<div>> at the OME data, it shows up in a way such that each frame looks like</div>
<div>> a different time point, instead of a different XY position.</div>
</div>
<div><br>
</div>
<div>Yes, there was a bug! Fixed now:</div>
<div><br>
</div>
<div><a href="https://github.com/ctrueden/bioformats/commit/ffa5e3071b2083cc0385032b6fdbcf40b3157271" target="_blank">https://github.com/ctrueden/bioformats/commit/ffa5e3071b2083cc0385032b6fdbcf40b3157271</a><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>== Spectral datasets ==</div>
<div><br>
</div>
<div>Next on my list is to finish adding support for spectral datasets. Will keep everyone posted, and file a PR as soon as it is ready.</div>
<div><br>
</div>
<div>Regards,<br>
</div>
<div>Curtis</div>
<div class="gmail_extra"><br>
</div>
<div class="gmail_extra">[1] <a href="https://www.openmicroscopy.org/Schemas/Documentation/Generated/OME-2013-06/ome_xsd.html#Channel_SamplesPerPixel" target="_blank">https://www.openmicroscopy.org/Schemas/Documentation/Generated/OME-2013-06/ome_xsd.html#Channel_SamplesPerPixel</a></div>
<div class="gmail_extra"><br>
</div>
<div class="gmail_extra">[2] <a href="http://partners.adobe.com/public/developer/en/tiff/TIFF6.pdf" target="_blank">http://partners.adobe.com/public/developer/en/tiff/TIFF6.pdf</a></div>
<div class="gmail_extra"><br>
</div>
<div class="gmail_extra">[3] <a href="https://github.com/openmicroscopy/bioformats/pull/1306" target="_blank">https://github.com/openmicroscopy/bioformats/pull/1306</a></div>
<div class="gmail_extra"></div></div></div></div></div></div></div></div></blockquote></div></div><div class="gmail_extra"><br></div></div>