<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif;">
<div style="font-size: 14px; font-family: Calibri, sans-serif;">Hi Jake-</div>
<div style="font-size: 14px; font-family: Calibri, sans-serif;"><br>
</div>
<div style="font-size: 14px; font-family: Calibri, sans-serif;">As always, thanks for your comments. Responses in line.</div>
<div style="font-size: 14px; font-family: Calibri, sans-serif;"><br>
</div>
<div style="font-size: 14px; font-family: Calibri, sans-serif;">On 21/12/2016 12:28, "ome-users on behalf of Jake Carroll" <<a href="mailto:ome-users-bounces@lists.openmicroscopy.org.uk">ome-users-bounces@lists.openmicroscopy.org.uk</a> on behalf of
<a href="mailto:jake.carroll@uq.edu.au">jake.carroll@uq.edu.au</a>> wrote:</div>
<div style="font-size: 14px; font-family: Calibri, sans-serif;"><br>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="font-size: 14px; font-family: Calibri, sans-serif; border-left-color: rgb(181, 196, 223); border-left-width: 5px; border-left-style: solid; padding: 0px 0px 0px 5px; margin: 0px 0px 0px 5px;">
<div>Hi,</div>
<div><br>
</div>
<div>Having just read the blog post here:</div>
<div><br>
</div>
<div><a href="http://blog.openmicroscopy.org/future-plans/community/2016/12/21/omero-5-3/">http://blog.openmicroscopy.org/future-plans/community/2016/12/21/omero-5-3/</a></div>
<div><br>
</div>
<div>Things are looking really wonderful and I look forward to deploying 5.3 into production.
</div>
</blockquote>
<div style="font-size: 14px; font-family: Calibri, sans-serif;"><br>
</div>
<div style="font-size: 14px; font-family: Calibri, sans-serif;">Thanks, 5.3.0 will be a big step, and we’ll have more coming in the following point releases.</div>
<div style="font-size: 14px; font-family: Calibri, sans-serif;"><br>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="font-size: 14px; font-family: Calibri, sans-serif; border-left-color: rgb(181, 196, 223); border-left-width: 5px; border-left-style: solid; padding: 0px 0px 0px 5px; margin: 0px 0px 0px 5px;">
<div><br>
</div>
<div>I have a couple of questions.</div>
<div><br>
</div>
<div>1. Given the data model is changing, what becomes of the existing structures, infrastructure, underlying semantics of the way 5.2.x platforms work? Is there a smart conversion mechanism or table mapper of some description when we go from A to B?</div>
</blockquote>
<div style="font-size: 14px; font-family: Calibri, sans-serif;"><br>
</div>
<div>
<p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;" id="docs-internal-guid-ae915575-25ee-447f-0500-79cd067dff0b">
<span style="vertical-align: baseline;">The data model changes should be described in the Model documentation [1]. We published these so that anyone interested could follow our work and prepare for any changes relevant to them. A couple of these model changes
will affect OMERO directly, notably the dropped ROI properties and the reduced Marker enumerations. Additionally, the shape transform will be stored as OMERO objects instead of strings - see our blog post [2] for more information.</span></p>
<p dir="ltr" style="font-size: 14px; font-family: Calibri, sans-serif; line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;" id="docs-internal-guid-ae915575-25ee-447f-0500-79cd067dff0b">
<span style="font-size: 14.666666666666666px; font-family: Arial; vertical-align: baseline;"><br>
</span></p>
<p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;" id="docs-internal-guid-ae915575-25ee-447f-0500-79cd067dff0b">
<span style="vertical-align: baseline;">As always, more information will be available in the relevant section of the upcoming developer documentation [3] as we progress towards OMERO 5.3.0.</span></p>
<p dir="ltr" style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;" id="docs-internal-guid-ae915575-25ee-447f-0500-79cd067dff0b">
<span style="vertical-align: baseline;"><br>
</span></p>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="font-size: 14px; font-family: Calibri, sans-serif; border-left-color: rgb(181, 196, 223); border-left-width: 5px; border-left-style: solid; padding: 0px 0px 0px 5px; margin: 0px 0px 0px 5px;">
<div>2. Key to OMERO’s acceptance (for us at least) when dealing with really (really) big data is throughput, performance of ingest and ability to then download those very large objects again at very high speed. Can you talk to any of the efforts made in this
regard or progress? I’d love to liberate the data and see the full potential of my hardware platforms utilised by OMERO. Currently due to certain eccentricities in ICE/Grid, it kind of feels like a giant funnel (a very functional and well designed funnel,
I might add (!)– but a funnel, none the less).</div>
</blockquote>
<div><br>
</div>
<div>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;" id="docs-internal-guid-ae915575-25ef-b9eb-24a0-6ff0efaca72d">
<span style="vertical-align: baseline;">Thanks again for raising-- Luke Hammond, Douglas Russell and others brought this up at our 2016 Annual Users Meeting [4], and it’s a very important issue. We run into similar problems with our work on IDR [5]. As always
getting work funded is a challenge, however we owe huge thanks to Douglas Russell (Harvard Med School) and Damir Sudar (Knight Center, OHSU) who have led an effort to secure supplements to a BD2K award to the NIH COMMONS LINCS project [6] to fund 3 separate
projects directed towards the “funnel problem”. These involve a true read-only OMERO server, portable session storage, and service routing. This work is underway and is being performed by the team at Glencoe Software. The project is public and all generated
code will be released under the GPL. Apologies for not writing to the community sooner about this-- the last two months have been a whirlwind. We’ll try to get an update on all this in the New Year.
<b><i>**A huge thanks to Douglas Russell & Peter Sorger (HMS) and Damir Sudar & Joe Gray (OHSU) and the teams at LINCS HMS and LINCS OHSU for securing this funding from the BD2K programme**</i></b>.</span></p>
<br>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="vertical-align: baseline;">A note in this regard-- requests for work to be done are always welcome, and we try to track and respond to all requests. During 2016, we have had
several entities-- Carl Zeiss, 3i, PerkinElmer, EMBL-EBI, HMS, OHSU-- who have needed work done and either performed the work themselves and contributed it to the community, or actively contributed to the costs of performing the work and delivering it to the
community. Of course, we continue to receive contributions from several individuals and other entities [7]. We hope to grow this type of activity in 2017. This is particularly important as we have more work to do-- for example, scaling imports is a major
unfunded priority. Obviously, anyone wanting to make contributions should get in touch.</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size: 14.666666666666666px; font-family: Arial; vertical-align: baseline;"><br>
</span></p>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="font-size: 14px; font-family: Calibri, sans-serif; border-left-color: rgb(181, 196, 223); border-left-width: 5px; border-left-style: solid; padding: 0px 0px 0px 5px; margin: 0px 0px 0px 5px;">
<div>3. What of formats such as HDF5? From a computational model representation and programmatic manipulation perspective - it seems to be something persistently on everybody’s minds. Might we see progress down this path? If not – I am sure there is probably
a good reason which I don’t understand. Would love to understand more about it…</div>
</blockquote>
<div><br>
</div>
<div>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;" id="docs-internal-guid-ae915575-25f0-7d5c-4ab5-5955207d2456">
<span style="vertical-align: baseline;">Indeed, we have discussed this at length, for some time. We have been reluctant to take on the challenge of standardising binary containers-- we have enough to do dealing with metadata. However, as imaging experiments
become more complex, tools that properly store binary data and metadata become more important for our community. They are also becoming critical for making our software work-- we are running into several examples where data storage structures are challenging,
e.g., proprietary HCS systems are writing data from plate-based imaging as single TIFFs. This was (barely) manageable when users were collecting 2-4 fields and 2-3 channels per well. Now, with Cell Painting [8], 50 fields/well and timelapse imaging, a single
plate generates 10^5 - 10^6 files on disk. Simply reading these data is very challenging, only because of the file format implementation (the proprietary vendors who use these antiquated storage mechanisms know who they are!!!).</span></p>
<br>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="vertical-align: baseline;">To this end, we have submitted a proposal (BSRC TRDF [9]) to fund work that combines OME metadata with several established binary containers-- a
few different variants of HDF5 (e.g., BigDataViewer [10], Cellh5 [11], etc.), the KLB format [12], and a few others. We haven’t found a single binary container that can satisfy all the needs of the broad community we serve, so we hope to support a few-- if
we get the resources. For example, some members of the community might need lossless compression (i.e., KLB), others might need BDV for Fiji-based visualisation, and so on. We note, with great foreboding, that HDF5 write-locking remains an issue and we are
concerned about using HDF5-based formats in large, distributed environments. If anyone in the community has a good solution for this, we’d love to hear it (and use it!).</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="vertical-align: baseline;"><br>
</span></p>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="font-size: 14px; font-family: Calibri, sans-serif; border-left-color: rgb(181, 196, 223); border-left-width: 5px; border-left-style: solid; padding: 0px 0px 0px 5px; margin: 0px 0px 0px 5px;">
<div><br>
</div>
<div>Thank you and keep up the amazing work. We really appreciate it and are looking forward very much to what is coming.</div>
</blockquote>
<div><br>
</div>
<div>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;" id="docs-internal-guid-ae915575-25f0-f462-b153-5709e95c5b00">
<span style="vertical-align: baseline;">Thank you for all the usage, feedback and support. </span></p>
<br>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="vertical-align: baseline;">Jason, Josh, Seb, Jean-Marie, Chris and the whole OME Team.</span></p>
<br>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="vertical-align: baseline;">[1]
</span><a href="https://www.openmicroscopy.org/site/support/ome-model/schemas/june-2016.html" style="text-decoration:none;"><span style="color: rgb(17, 85, 204); text-decoration: underline; vertical-align: baseline;">https://www.openmicroscopy.org/site/support/ome-model/schemas/june-2016.html</span></a></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="vertical-align: baseline;">[2]
</span><a href="http://blog.openmicroscopy.org/data-model/future-plans/2016/06/20/shape-transforms/" style="text-decoration:none;"><span style="color: rgb(17, 85, 204); text-decoration: underline; vertical-align: baseline;">http://blog.openmicroscopy.org/data-model/future-plans/2016/06/20/shape-transforms/</span></a></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="vertical-align: baseline;">[3]
</span><a href="http://www.openmicroscopy.org/site/support/omero5.3/developers/whatsnew.html" style="text-decoration:none;"><span style="color: rgb(17, 85, 204); text-decoration: underline; vertical-align: baseline;">http://www.openmicroscopy.org/site/support/omero5.3/developers/whatsnew.html</span></a></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="vertical-align: baseline;">[4]
</span><a href="http://www.openmicroscopy.org/site/community/minutes/meetings/11th-annual-users-meeting-2016" style="text-decoration:none;"><span style="color: rgb(17, 85, 204); text-decoration: underline; vertical-align: baseline;">http://www.openmicroscopy.org/site/community/minutes/meetings/11th-annual-users-meeting-2016</span></a><span style="vertical-align: baseline;">
</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="vertical-align: baseline;">[5]
</span><a href="http://idr-demo.openmicroscopy.org/" style="text-decoration:none;"><span style="color: rgb(17, 85, 204); text-decoration: underline; vertical-align: baseline;">http://idr-demo.openmicroscopy.org/</span></a></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="vertical-align: baseline;">[6]
</span><a href="http://www.lincsproject.org/" style="text-decoration:none;"><span style="color: rgb(17, 85, 204); text-decoration: underline; vertical-align: baseline;">http://www.lincsproject.org/</span></a><span style="vertical-align: baseline;">
</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="vertical-align: baseline;">[7]
</span><a href="http://www.openmicroscopy.org/site/about/ome-contributors" style="text-decoration:none;"><span style="color: rgb(17, 85, 204); text-decoration: underline; vertical-align: baseline;">http://www.openmicroscopy.org/site/about/ome-contributors</span></a><span style="vertical-align: baseline;">
</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="vertical-align: baseline;">[8]
</span><a href="http://www.nature.com/nprot/journal/v11/n9/full/nprot.2016.105.html" style="text-decoration:none;"><span style="color: rgb(17, 85, 204); text-decoration: underline; vertical-align: baseline;">http://www.nature.com/nprot/journal/v11/n9/full/nprot.2016.105.html</span></a><span style="vertical-align: baseline;">
</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="vertical-align: baseline;">[9]
</span><a href="http://www.bbsrc.ac.uk/funding/filter/2016-tools-resources-development-fund-bioimaging/" style="text-decoration:none;"><span style="color: rgb(17, 85, 204); text-decoration: underline; vertical-align: baseline;">http://www.bbsrc.ac.uk/funding/filter/2016-tools-resources-development-fund-bioimaging/</span></a><span style="vertical-align: baseline;">
</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="vertical-align: baseline;">[10]
</span><a href="http://www.nature.com/nmeth/journal/v12/n6/full/nmeth.3392.html" style="text-decoration:none;"><span style="color: rgb(17, 85, 204); text-decoration: underline; vertical-align: baseline;">http://www.nature.com/nmeth/journal/v12/n6/full/nmeth.3392.html</span></a></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="vertical-align: baseline;">[11]
</span><a href="http://www.cellh5.org/" style="text-decoration:none;"><span style="color: rgb(17, 85, 204); text-decoration: underline; vertical-align: baseline;">http://www.cellh5.org/</span></a></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="vertical-align: baseline;">[12]
</span><span style="text-decoration: underline; color: rgb(17, 85, 204); vertical-align: baseline;"><a href="http://www.nature.com/nprot/journal/v10/n11/abs/nprot.2015.111.html" style="text-decoration:none;">http://www.nature.com/nprot/journal/v10/n11/abs/nprot.2015.111.html</a></span><span style="vertical-align: baseline;"></span></p>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="border-left-color: rgb(181, 196, 223); border-left-width: 5px; border-left-style: solid; padding: 0px 0px 0px 5px; margin: 0px 0px 0px 5px;">
<div><br>
</div>
</blockquote>
<br>
<span style="font-size:10pt;">The University of Dundee is a registered Scottish Charity, No: SC015096</span>
</body>
</html>