<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>A bit of background first.</div>
<div><br>
</div>
As you perhaps know the main driver for our involvement in OMERO is the need to share data & make some data
<div>publicly available.</div>
<div><br>
</div>
<div>To achieve this we have been modifying our acquisition software to generate ome-tiffs which can then be imported into OMERO.</div>
<div>We have also modified FLIMfit to allow it to read ome-tiffs.</div>
<div>This is in order to be able to use the “download original files” functionality in OMERO.web which wasn’t compatible with our earlier approach </div>
<div>of writing data directly to OMERO via the OMERO api.</div>
<div><br>
</div>
<div>We now have software in place to write all our data types as ome-tiff with one exception.</div>
<div>We now need to decide how best to handle FLIM plate data.</div>
<div><br>
</div>
<div>As a first step<span style="font-size: 11px;"> have now succeeded in writing Java code to create a dummy SPW plate and write it into a Fileset.</span></div>
<div><span style="font-size: 11px;"><br>
</span></div>
<div><a href="https://github.com/imunro/bioformats/commit/aeea8cea014a9d546e22c3e5cb577787d2e60352">https://github.com/imunro/bioformats/commit/aeea8cea014a9d546e22c3e5cb577787d2e60352</a></div>
<div><br>
</div>
<div>Our original plan was to have one ome-tiff per FOV.</div>
<div><span style="font-size: 11px;"><br>
</span></div>
<div>However If I’ve written the above code correctly (?) then the issue is, OME-XML’ bloat.’ </div>
<div>The metadata for the entire plate is required to be in each file of the Fileset therefore the total amount of metadata rises with the number of files.</div>
<div style="margin: 0px;">Guesstimating the size of the metadata by linearly scaling from my dummy plate gives us 1.2Mb per file of Metadata for a ‘typical’ plate.</div>
<div style="margin: 0px;">Where typical’ is assumed to be 80 wells x 5 FOVs/well x 5 time-points/FOV i.e. 400 FLIMages.</div>
<div style="margin: 0px;">For comparison the data stored as a stack of tiffs would occupy 140Mb.</div>
<div style="margin: 0px;"><br>
</div>
<div style="margin: 0px; min-height: 13px;">We therefore have a range of options & we need your input to select one that allows the data to be downloaded ( when this capability becomes available).</div>
<div style="margin: 0px; min-height: 13px;">see <a href="https://trello.com/c/nYSjzdnM/260-rfe-extend-batch-download-of-original-files-to-include-spw-in-web-client">https://trello.com/c/nYSjzdnM/260-rfe-extend-batch-download-of-original-files-to-include-spw-in-web-client</a></div>
<div style="margin: 0px; min-height: 13px;"><br>
</div>
<div style="margin: 0px; min-height: 13px;">We note that a solution is in place for JCB</div>
<div style="margin: 0px; min-height: 13px;"><br>
</div>
<div style="margin: 0px;">Some of our options are:</div>
<div style="margin: 0px; min-height: 13px;"><br>
</div>
<div style="margin: 0px;">1) Don’t bother with OME SPW data at all.Simply store the plate data in a dataset</div>
<div style="margin: 0px;"><span class="Apple-tab-span" style="white-space:pre"></span>Estimated ‘bloat’ 0bytes.</div>
<div style="margin: 0px;"><span class="Apple-tab-span" style="white-space:pre"></span>Pros: Could simplify the FLIMfit code by removing all Screen/plate menu items.</div>
<div style="margin: 0px;"> FLIMfit can already read this data by parsing the filename.</div>
<div style="margin: 0px;"> A hypothetical public downloader (MOP) would get a single file per FOV (easy for them to interpret).</div>
<div style="margin: 0px;"> Code already in place to generate ome-tiffs.</div>
<div style="margin: 0px;"> “Original file download" already works!</div>
<div style="margin: 0px;"> <span class="Apple-tab-span" style="white-space:pre"> </span>
Cons: Data doesn’t display as a plate in OMERO clients.</div>
<div style="margin: 0px;"> Any future plate tools won’t work.</div>
<div style="margin: 0px; min-height: 13px;"><br>
</div>
<div style="margin: 0px;">2) Put the whole plate in a small number (say 1-4) files.</div>
<div style="margin: 0px;"> Estimated ‘bloat’ 1.2 - 4.8 Mb ( <2% of the data size stored as ‘stack of tiffs’)</div>
<div style="margin: 0px;"><span class="Apple-tab-span" style="white-space:pre"></span>Pros: Data will import into OMERO without problem as a plate Minimal bloat.</div>
<div style="margin: 0px;"> <span class="Apple-tab-span" style="white-space:pre"> </span>
Cons: FLIMfit needs modification to use SPW metadata rather than filename. </div>
<div style="margin: 0px;"><span class="Apple-tab-span" style="white-space:pre"></span>Big! files.</div>
<div style="margin: 0px;"><span class="Apple-tab-span" style="white-space:pre"></span> A MOP will get huge ome-tiffs</div>
<div style="margin: 0px;"> They will also need to sort out the metadata to work out what goes where.</div>
<div style="margin: 0px; min-height: 13px;"><br>
</div>
<div style="margin: 0px;">3) Have one file per FOV. </div>
<div style="margin: 0px;"> Estimated ‘bloat’ 480Mb (350% of the data size stored as ‘stack of tiffs’)</div>
<div style="margin: 0px;"> Pros: No modification required to FLIMfit ( each file is a FOV as now). </div>
<div style="margin: 0px;"> Will import to OMERO as a plate.</div>
<div style="margin: 0px;"> Cons: bloat!</div>
<div style="margin: 0px; min-height: 13px;"><br>
</div>
<div style="margin: 0px;">4) Have one file per well.</div>
<div style="margin: 0px;"> Estimated ‘bloat’ 96Mb (70% of the data size stored as ‘stack of tiffs)</div>
<div style="margin: 0px;"> Pros: Data will import into OMERO without problem as a plate.</div>
<div style="margin: 0px;"> Cons: FLIMfit needs modification.</div>
<div style="margin: 0px;"> ‘Bloat’ still large<span class="Apple-tab-span" style="white-space:pre">
</span> </div>
<div style="margin: 0px;"> Non-trivial for hypothetical MOP.</div>
<div style="margin: 0px;"> Although easier than 2) as they can determine the well from the Filename </div>
<p style="margin: 0px; min-height: 13px;"> <br class="webkit-block-placeholder">
</p>
<div style="margin: 0px;">5) Use BinaryOnly ome-tiff format. </div>
<div style="margin: 0px;"> [ In this format there is only one copy of the metadata. all the other files just contain a reference to this. “master’ file. ]</div>
<div style="margin: 0px;"> Estimated ‘bloat’ 1.2Mb ( < 1 % of data size stored as ‘stack of tiffs’)</div>
<div style="margin: 0px;"> Pros: No mods required to FLIMfit (one file per FOV) although using the SPW metadata is a desirable feature.. Minimal ‘bloat’</div>
<div style="margin: 0px;"> Cons: Not yet available. </div>
<div style="margin: 0px;"> </div>
<div style="margin: 0px;">6) Store data as ome-tiffs with no SPW xml but add this info in OMERO after import (either via dataset-to-plate script or the API)</div>
<div style="margin: 0px;"> Estimated bloat. Unclear -ask OME team</div>
<div style="margin: 0px;"> Pros. Might allow one file per FOV & work with future plate tools.</div>
<div style="margin: 0px;"> Cons: Might not work - ask OME team</div>
<div style="margin: 0px;"> Big software effort and end of project is approaching rapidly.</div>
<div style="margin: 0px;"> </div>
<div style="margin: 0px;"><br>
</div>
<div style="margin: 0px;">Thanks in advance</div>
<div style="margin: 0px;"><br>
</div>
<div style="margin: 0px;">Ian</div>
<p style="margin: 0px; min-height: 14px;"> <br class="webkit-block-placeholder">
</p>
<div style="margin: 0px; min-height: 14px;"><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
</body>
</html>