<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<div dir="ltr">On 15 August 2016 at 20:47, Damir Sudar <span dir="ltr"><<a href="mailto:dsudar@lbl.gov" target="_blank">dsudar@lbl.gov</a>></span> wrote:<br>
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">Hi Simon,<br>
<br>
Thanks for the quick response.<span class=""><br>
<br>
<div>On 8/15/2016 2:31 AM, Simon Li wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr"><br>
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">Just a few questions and possibly there are some suggestions for improvement hidden in there:<br>
1) when I upload a .csv file with "bin/omero metadata populate --file filename.csv --context csv Screen:<screen_id>"<br>
the resulting Table is called "bulk_annotations". Is there a way to upload with a different name?<br>
</div>
</blockquote>
<div><br>
</div>
<div>Not at the moment, though it should be possible to change the name since OMERO.web uses the file-annotation's namespace. You can change it manually if you want:<br>
<br>
omero obj update OriginalFile:123 Name=experimental-data.h5<br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</span>Ah yes, good thought. I'll try to do that.<span class=""><br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">2) one reason I ask is that I would like to upload 2 different annotations to a Screen: #1 is the experimental metadata that describes the experiment and #2 is a feature measurements file after analysis which is likely
to be done at a later time, thus the desire to separate that from the experimental metadata. Currently both attached Tables care named bulk_annotations.<br>
</div>
</blockquote>
<div><br>
</div>
<div>That sounds similar to something we're doing at the moment in the IDR, though we're not displaying the feature values at the moment. One thing we're considering is having a separate namespace for the bulk-annotations and the feature analysis.<br>
<br>
</div>
<div>Do you have some ideas for how this should be presented, or is the current tabular display OK for you?<br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</span>For now, the tabular display is fine since our current feature tables are per-image summary statistics, not per-cell feature measurement data. Once one wants to store per-cell features as a table per image (and we will want to do so in the near future)
the simple tabular display will no longer suffice since it will be at least a multi-column table. In order to display anything like, you probably will need a separate window/table, maybe similar to:
<a href="http://www.openmicroscopy.org/site/support/omero5.2/developers/analysis.html#third-party-analysis-and-omero-tables" target="_blank">
http://www.openmicroscopy.org/<wbr>site/support/omero5.2/<wbr>developers/analysis.html#<wbr>third-party-analysis-and-<wbr>omero-tables</a><br>
But that's probably something for a serious design review and I presume will rely on all the new ROI work going on, right?<span class=""><br>
<br>
</span></div>
</blockquote>
<div><br>
</div>
<div>You're right. We do have cell-level analysis data for some of the IDR screens, but we decided against displaying it in the UI due to the significant work required. It's certainly important, so if it's OK we might ask for your input in future with regards
to it's design.<br>
</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><span class="">
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">3) so while it is possible to upload 2 different annotation files and both show up as Attachments to the Screen object (with the same "bulk_annotations" name), when I look at the Image objects within that Screen, they only
show in the Tables tab the contents from the most recently uploaded annotation file.<br>
</div>
</blockquote>
<div><br>
</div>
<div>That was a workaround we implemented a few years ago<br>
<a href="https://github.com/openmicroscopy/openmicroscopy/pull/512#issuecomment-11372968" target="_blank">https://github.com/openmicrosc<wbr>opy/openmicroscopy/pull/512#<wbr>issuecomment-11372968</a><br>
</div>
<div>Now that the metadata work is seeing more use it's something we should improve.<br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</span>Ah, I see. For now there are a few work-arounds that I can use: 1) merge all my annotation files into one and upload that (and redo that when the due to a new analysis algorithm there are different measured features), and 2) (and this is what I am currently
doing) upload the static experimental metadata, create the key-value pairs using the metadata populate functionality for everything I want people to see, then upload the feature annotation file and that then shows up in the Tables tab. A little clunky but
workable.<span class=""><br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div></div>
<div></div>
</div>
</div>
</div>
</blockquote>
</span></div>
</blockquote>
<div>The original intention in the IDR was to use Map-annotations for display, and to not display the content of the tables at all, though they'd still be accessible as file-annotations. I'll make a note that it should be configurable!<br>
<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><span class="">
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">4) the metadata annotation works great for "normal" plates that have numbered columns and "lettered" rows. However, when one has lettered columns and numbered rows, everything gets jumbled and when there are more than 26
rows the routine throws an exception (see my earlier message). I worked around this issue (mostly with Will Moore's help, see:
<a href="https://github.com/openmicroscopy/openmicroscopy/pull/4743" target="_blank">
https://github.com/openmicrosc<wbr>opy/openmicroscopy/pull/4743</a>) but it would be great if that issue was really fixed before releasing the metadata functionality into the wild.<br>
</div>
</blockquote>
<div><br>
</div>
<div>Again this is something that's only come to light as more people use it, I've added it to our bug-list.<br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</span>Thanks. That is actually a bug I'd love to see disappear.<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div>Thanks for your feedback!<br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
And thank you guys for building all this functionality I need.<br>
<br>
One more question: in my current workflow I do not create a column in my experimental metadata annotation file called "Image Name", mostly because I wouldn't know what for to use that, and thus the populator generates lots of messages: "INFO:omero.util.populate_<wbr>metadata:Missing
image name column, skipping."<br>
Not a big problem but annoying. Since it appears to be just an INFO message, is there a way to make it appear only once? Or am I missing something and I really should have a Image Name column?<br>
</div>
</blockquote>
<div><br>
</div>
<div>After some discussion in the team we're not aware of a reason, I'll add a ticket for the log messages to be turned down.<br>
<br>
</div>
<div>Simon<br>
<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><br>
Cheers,<br>
- Damir<span class=""><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div>Simon<br>
</div>
<div><br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">Thanks,<br>
- Damir<br>
<br>
<div>On 6/27/2016 2:03 PM, Eleanor Williams wrote:<br>
</div>
<blockquote type="cite">
<p>Hi Damir</p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">To
convert the omero tables to key-value pairs annotations we are again using the CLI metadata populate script along with a yaml-format configuration file to specify which columns we want to take from the omero tables and put into Key-Value Annotations.
<br>
</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline"><br>
</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">This
component of the metadata functionality is still in the development stage and so to use it you need to:</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline"><br>
</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">1.
use the latest version of omero (5.2.4) and</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">2.
then run as</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">OMERO_DEV_PLUGINS=1
bin/omero metadata populate --context bulkmap --cfg bulkmap-config.yml Screen:n</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">where
n is the screen number in omero.</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline"><br>
</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">An
example config file for one of our screens is here</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><a href="https://github.com/IDR/idr-metadata/blob/master/idr0002-heriche-condensation/screenA/idr0002-screenA-bulkmap-config.yml" style="text-decoration:none" target="_blank"><span style="font-size:14.6667px;font-family:Arial;color:rgb(17,85,204);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:underline;vertical-align:baseline"></span></a><a href="https://github.com/IDR/idr-metadata/blob/master/idr0002-heriche-condensation/screenA/idr0002-screenA-bulkmap-config.yml" target="_blank">https://github.com/IDR/idr-met<wbr>adata/blob/master/idr0002-heri<wbr>che-condensation/screenA/idr00<wbr>02-screenA-bulkmap-config.yml</a></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline"><br>
</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">You
can see I have just picked out some of the columns from the</span><a href="https://github.com/IDR/idr-metadata/blob/master/idr0002-heriche-condensation/screenA/idr0002-screenA-annotation.csv" style="text-decoration:none" target="_blank"><span style="font-size:14.6667px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline"></span><span style="font-size:14.6667px;font-family:Arial;color:rgb(17,85,204);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:underline;vertical-align:baseline">annotation.csv</span></a><span style="font-size:14.6667px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">
file, in this case ones that I thought IDR users might use as search terms, and I have specified URLs for some. You can also change the 'display name' of the column using the ' clientname' tag and create URLs using the 'clientvalue' tag. </span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline"><br>
</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:14.6667px;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline">Let
us know if you have any problems running this, or suggestions for improvement.</span></p>
Best regards<br>
Eleanor<br>
<br>
</blockquote>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
<br>
</span><span class="">
<pre cols="72">--
Damir Sudar - Affiliate Scientist
Lawrence Berkeley Natl Laboratory / MBIB
One Cyclotron Road, MS 977, Berkeley, CA 94720, USA
T: <a href="tel:510%2F486-5346" value="+15104865346" target="_blank">510/486-5346</a> - F: <a href="tel:510%2F486-5586" value="+15104865586" target="_blank">510/486-5586</a> - E: <a href="mailto:DSudar@lbl.gov" target="_blank">DSudar@lbl.gov</a>
<a href="http://biosciences.lbl.gov/profiles/damir-sudar-2/" target="_blank">http://biosciences.lbl.gov/<wbr>profiles/damir-sudar-2/</a>
Visiting Scientist, Oregon Health & Science University</pre>
<br>
<span style="font-size:10pt">The University of Dundee is a registered Scottish Charity, No: SC015096</span>
</span></div>
<br>
______________________________<wbr>_________________<br>
ome-devel mailing list<br>
<a href="mailto:ome-devel@lists.openmicroscopy.org.uk">ome-devel@lists.<wbr>openmicroscopy.org.uk</a><br>
<a href="http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel" rel="noreferrer" target="_blank">http://lists.openmicroscopy.<wbr>org.uk/mailman/listinfo/ome-<wbr>devel</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
<br>
<span style="font-size:10pt;">The University of Dundee is a registered Scottish Charity, No: SC015096</span>
</body>
</html>