<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>Hi Damir</p>
<p></p>
<p></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;" id="docs-internal-guid-b74132da-93ab-92d1-0213-48859af781e5">
<span style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">To convert the omero tables to key-value pairs annotations
we are again using the CLI metadata populate script along with a yaml-format configuration file to specify which columns we want to take from the omero tables and put into Key-Value Annotations.
<br>
</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"><br>
</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">This
component of the metadata functionality is still in the development stage and so to use it you need to:</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"><br>
</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">1.
use the latest version of omero (5.2.4) and</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">2.
then run as</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">OMERO_DEV_PLUGINS=1
bin/omero metadata populate --context bulkmap --cfg bulkmap-config.yml Screen:n</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">where
n is the screen number in omero.</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"><br>
</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">An
example config file for one of our screens is here</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><a href="https://github.com/IDR/idr-metadata/blob/master/idr0002-heriche-condensation/screenA/idr0002-screenA-bulkmap-config.yml" style="text-decoration:none;"><span style="font-size:14.666666666666666px;font-family:Arial;color:#1155cc;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:underline;vertical-align:baseline;"></a><a class="moz-txt-link-freetext" href="https://github.com/IDR/idr-metadata/blob/master/idr0002-heriche-condensation/screenA/idr0002-screenA-bulkmap-config.yml">https://github.com/IDR/idr-metadata/blob/master/idr0002-heriche-condensation/screenA/idr0002-screenA-bulkmap-config.yml</a></span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"><br>
</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">You
can see I have just picked out some of the columns from the</span><a href="https://github.com/IDR/idr-metadata/blob/master/idr0002-heriche-condensation/screenA/idr0002-screenA-annotation.csv" style="text-decoration:none;"><span style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
</span><span style="font-size:14.666666666666666px;font-family:Arial;color:#1155cc;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:underline;vertical-align:baseline;">annotation.csv</span></a><span style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">
file, in this case ones that I thought IDR users might use as search terms, and I have specified URLs for some. You can also change the 'display name' of the column using the ' clientname' tag and create URLs using the 'clientvalue' tag. </span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"><br>
</span></p>
<p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:14.666666666666666px;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">Let
us know if you have any problems running this, or suggestions for improvement.</span></p>
<p></p>
Best regards<br>
Eleanor<br>
<br>
<div class="moz-cite-prefix">On 27/06/2016 03:04, Damir Sudar wrote:<br>
</div>
<blockquote cite="mid:e00055fc-3c7b-acf4-30fa-3d6061190286@lbl.gov" type="cite">Hi Eleanor and Josh,<br>
<br>
I'm learning a lot from your work on putting the IDR together and I'm applying it for our image repository for our LINCS project (<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://www.lincsproject.org/centers/data-and-signature-generating-centers/oregon-u/">http://www.lincsproject.org/centers/data-and-signature-generating-centers/oregon-u/</a>)
that hopefully will go live in a few months. We're also looking at Zegami as a possible way to provide visualizations of that data (Great demo, Roger!! and thanks for the code for that).<br>
<br>
I've gotten as far as putting my metadata into a a format that can be used to generate the omero tables using the Populate Metadata script. But I couldn't find how you guys then populate the Key-Value Annotations. It appears to be a subset of what's in the
Tables. Do you have some magic scripts for that and are those available?<br>
<br>
Cheers,<br>
- Damir<br>
<br>
<div class="moz-cite-prefix">On 6/16/2016 2:21 AM, Eleanor Williams wrote:<br>
</div>
<blockquote cite="mid:bacc151b-5e5c-4af6-f31d-26a7559d457f@dundee.ac.uk" type="cite">
<p>Hi Steve and Roger<br>
</p>
<p>I think idr0002-heriche-condensation would be a good screen to start with. It is relatively small at 12 plates and has good annotation with control, quality control and phenotype values. It can be found here
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://idr-demo.openmicroscopy.org/webclient/?show=screen-102">
http://idr-demo.openmicroscopy.org/webclient/?show=screen-102</a>.<br>
</p>
<p>I am doing a bit of a stock-take of the annotation files I've produced so far and will be changing a few aspects over the next few weeks, mainly to include more of the information that is currently held in the study files. For example I'd like to get more
of the sample attributes such as species and cell line into the annotation.csv file, and then into the omero tables and Map Annotations. Sample attributes for screens were originally recorded in the study file because it didn't vary per screen, but now that
we are also looking at non-screen data where this might vary it makes sense to always have it visible with the images. Also we haven't got as far as exposing much of the study information in omero yet.
<br>
</p>
<p>I would really like to work with you to understand how you are going to use the experimental and analytic annotations and how we can help in providing the metadata you need and in the most usable format. For example would it be useful if I indicate what
is sample attribute information in the annotation.csv file by using something like a constistent header format e.g. Characterisics [Organism], Characteristics [Cell Line], Characteristics [Organism Part] etc, or is this not really necessary for your purposes
and 'Organism', 'Cell Line' column headings would be enough? <br>
</p>
<p>Finally, it was <a moz-do-not-send="true" href="https://www.openmicroscopy.org/site/about/development-teams/jason/balaji.png/view">
Balaji Ramalingam</a> that you were talking to about streaming the thumbnails. I'm sure he would be happy to discuss this further with you.
<br>
</p>
<p>Best regards</p>
<p>Eleanor</p>
<p><br>
</p>
<p><br>
</p>
<br>
<div class="moz-cite-prefix">On 15/06/2016 20:59, Josh Moore wrote:<br>
</div>
<blockquote cite="mid:CAD3Ea=X2o+AewGgw0aCJGkaDfJbMDqsNZAHeC3QuHKSknHpiGQ@mail.gmail.com" type="cite">
<pre wrap="">Forwarding this to the mailing list so that anyone who had expressed
interested in <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://zegami.com/">http://zegami.com/</a> at this year's users' meeting can
follow along.
Cheers,
~Josh
P.S. If you weren't able to attend #OME2016, a recap is now up on the
blog <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://blog.openmicroscopy.org/community/2016/06/15/user-meeting/">http://blog.openmicroscopy.org/community/2016/06/15/user-meeting/</a>
---------- Forwarded message ----------
From: Stephen Taylor <a moz-do-not-send="true" class="moz-txt-link-rfc2396E" href="mailto:stephen.taylor@imm.ox.ac.uk"><stephen.taylor@imm.ox.ac.uk></a>
Date: Tue, Jun 14, 2016 at 8:17 AM
Subject: Re: [IDR Project] Zegami / IDR
Cc: Roger Noble <a moz-do-not-send="true" class="moz-txt-link-rfc2396E" href="mailto:roger@zegami.com"><roger@zegami.com></a>
Hi,
I’ve been look at a few examples from the link Josh sent to me and
there are quite a lot! What would be a good data set to get import
into Zegami from IDR? I am thinking a set that has decent metadata
and a study that people are more familiar with that would make an
interesting use case.
Kind regards,
Steve
From: Stephen Taylor
Sent: 10 June 2016 14:25
Cc: Roger Noble <a moz-do-not-send="true" class="moz-txt-link-rfc2396E" href="mailto:roger@zegami.com"><roger@zegami.com></a>; '<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:e.x.williams@dundee.ac.uk">e.x.williams@dundee.ac.uk</a>'
<a moz-do-not-send="true" class="moz-txt-link-rfc2396E" href="mailto:e.x.williams@dundee.ac.uk"><e.x.williams@dundee.ac.uk></a>
Subject: RE: [IDR Project] Zegami / IDR
Hi all,
I am interested generally in getting data in and out of OMERO from
both internal and external. Over a WAN the images are less of an issue
since they are streamed on demand but large metadata on the other hand
can get out of hand since it all has to be loaded completely into
Zegami’s database.
Currently I do “OMERO project” to “Zegami collection” using a Zegami
export script (modified version of the OMERO batch export script).
This takes a project or selection of images from OMERO and produces
thumbnails and data in tab delimited Zegami format and then I manually
import that into Zegami for processing. The OMERO id is preserved so I
can link to the original image in OMERO using the OMERO web client. I
only export certain metadata currently (filenames and OMERO tags) and
my plan is to make this more comprehensive and allow MapAnnotations
and Tables to be exported using this mechanism as well. I need to find
out more how to query these data types. Since we have internal
projects at Oxford that need this I’d like to finish this and make
that plugin freely available for other OMERO users to publish their
data in Zegami.
Longer term for IDR/Zegami
One of your team (apologies I didn’t get his name but he was doing
OMERO/ImageJ in the Unconference session) suggested we just stream in
the thumbnail and just have 1 level (i.e. no zoom) and when you want
to see the image just open it up in OMERO. This would not be as nice
to use but may be sufficient for HCS and would allow you to use
Zegami’s facet based query engine and would be probably little (no?)
work for you guys. We need to test this to see what it looks like and
it assumes OMERO can supply a unique thumbnail URL for each image and
that the metadata will provide a primary key so that we can get to
this image via a thumbnail id. To get at the metadata Josh and Eleanor
said they were thinking of making the CSV downloadable for a
collection, which could be loaded into Zegami on the fly (we have
instances where we dynamically load metadata via URL already).
However, major caveat is on how well this would perform over a WAN
with a large amount of data.
BTW we have developer docs for Zegami for those interested at:
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="https://support.zegami.com/hc/en-us/categories/200350657-Developers">https://support.zegami.com/hc/en-us/categories/200350657-Developers</a>
and would welcome feedback.
Kind regards and thanks,
Steve
From: Jason Swedlow (Staff) [<a moz-do-not-send="true" class="moz-txt-link-freetext" href="mailto:j.r.swedlow@dundee.ac.uk">mailto:j.r.swedlow@dundee.ac.uk</a>]
Sent: 08 June 2016 13:09
Cc: Eleanor Williams (Staff) <a moz-do-not-send="true" class="moz-txt-link-rfc2396E" href="mailto:e.x.williams@dundee.ac.uk"><e.x.williams@dundee.ac.uk></a>
Subject: Re: [IDR Project] Zegami / IDR
Hi Josh et al—
My two cents— can we consider this use case from an external users’
perspective, mindful that distribution of such a solution would
definitely cause us to consider how we ship thumbnails, handle
straws, etc.
Cheers,
Jason
On Wed, Jun 8, 2016 at 1:31 PM, Josh Moore <a moz-do-not-send="true" class="moz-txt-link-rfc2396E" href="mailto:josh@glencoesoftware.com"><josh@glencoesoftware.com></a> wrote:
Stephen,
On Wed, Jun 8, 2016 at 9:52 AM, Stephen Taylor
<a moz-do-not-send="true" class="moz-txt-link-rfc2396E" href="mailto:stephen.taylor@imm.ox.ac.uk"><stephen.taylor@imm.ox.ac.uk></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Hi Josh,
Yes! What would be the best way of bulk downloading the thumbnails for a screen or plate
</pre>
</blockquote>
<pre wrap="">Interesting question. Guess the reverse question is: do you want to do
this as an external user or as if you had access to the OMERO?
</pre>
<blockquote type="cite">
<pre wrap="">( I notice the "Run Script" button isn't active ).
</pre>
</blockquote>
<pre wrap="">Correct. This server is stripped down in several ways as a public resource.
~J.
</pre>
<blockquote type="cite">
<pre wrap="">Kind regards,
Steve
</pre>
</blockquote>
<pre wrap="">-----Original Message-----
From: Josh Moore [<a moz-do-not-send="true" class="moz-txt-link-freetext" href="mailto:josh@glencoesoftware.com">mailto:josh@glencoesoftware.com</a>]
Sent: 08 June 2016 08:07
To: Stephen Taylor <a moz-do-not-send="true" class="moz-txt-link-rfc2396E" href="mailto:stephen.taylor@imm.ox.ac.uk"><stephen.taylor@imm.ox.ac.uk></a>
Cc: Eleanor Williams <a moz-do-not-send="true" class="moz-txt-link-rfc2396E" href="mailto:exwilliams@dundee.ac.uk"><exwilliams@dundee.ac.uk></a>;
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:e.x.williams@dundee.ac.uk">e.x.williams@dundee.ac.uk</a>; <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:j.a.moore@dundee.ac.uk">j.a.moore@dundee.ac.uk</a>
Subject: Re: Zegami / IDR
Hi Stephen,
On Wed, Jun 8, 2016 at 8:40 AM, Stephen Taylor
<a moz-do-not-send="true" class="moz-txt-link-rfc2396E" href="mailto:stephen.taylor@imm.ox.ac.uk"><stephen.taylor@imm.ox.ac.uk></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Hi Eleanor,
Thanks very useful. Is the web site publically available?
</pre>
</blockquote>
<pre wrap="">You mean <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://idr-demo.openmicroscopy.org/webclient">http://idr-demo.openmicroscopy.org/webclient</a> ?
Cheers,
~Josh
</pre>
<blockquote type="cite">
<pre wrap="">Kind regards and thanks,
Steve
</pre>
</blockquote>
<blockquote type="cite">
<pre wrap="">From: Eleanor Williams [<a moz-do-not-send="true" class="moz-txt-link-freetext" href="mailto:exwilliams@dundee.ac.uk">mailto:exwilliams@dundee.ac.uk</a>]
Sent: 07 June 2016 11:09
To: Stephen Taylor <a moz-do-not-send="true" class="moz-txt-link-rfc2396E" href="mailto:stephen.taylor@imm.ox.ac.uk"><stephen.taylor@imm.ox.ac.uk></a>;
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:e.x.williams@dundee.ac.uk">e.x.williams@dundee.ac.uk</a>; <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:j.a.moore@dundee.ac.uk">j.a.moore@dundee.ac.uk</a>
Subject: Re: Zegami / IDR
Hi Stephen
Here is a link to our workshop presentation but it describes how to
get the data into omero rather than out.
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://downloads.openmicroscopy.org/presentations/2016/Users-Meeting/Workshops/Metadata-at-Scale.pdf">http://downloads.openmicroscopy.org/presentations/2016/Users-Meeting/Workshops/Metadata-at-Scale.pdf</a>
We will be adding the annotation.csv files as attachments to each
screen over the next couple of weeks.
Let us know if you have any more questions.
Best regards
Eleanor
</pre>
</blockquote>
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap="">On 07/06/2016 10:41, Stephen Taylor wrote:
Hi Eleanor and Josh,
Great to meet you both at the OME meeting last week.
I'd be interested in trying get some data into Zegami from IDR. I
missed your Unmeeting (ironically I was demoing Zegami) where I think
you went into some of the technical details. Can you send me the link
to your presentation please?
Kind regards and thanks,
Steve
</pre>
</blockquote>
<pre wrap="">_______________________________________________
ome-devel mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:ome-devel@lists.openmicroscopy.org.uk">ome-devel@lists.openmicroscopy.org.uk</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel">http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Eleanor Williams PhD
Data Annotator/Software Engineer
Centre for Gene Regulation and Expression
University of Dundee, UK</pre>
<br>
<span style="font-size:10pt;">The University of Dundee is a registered Scottish Charity, No: SC015096</span>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset> <br>
<pre wrap="">_______________________________________________
ome-devel mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:ome-devel@lists.openmicroscopy.org.uk">ome-devel@lists.openmicroscopy.org.uk</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel">http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Damir Sudar - Affiliate Scientist
Lawrence Berkeley Natl Laboratory / MBIB
One Cyclotron Road, MS 977, Berkeley, CA 94720, USA
T: 510/486-5346 - F: 510/486-5586 - E: <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:DSudar@lbl.gov">DSudar@lbl.gov</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://biosciences.lbl.gov/profiles/damir-sudar-2/">http://biosciences.lbl.gov/profiles/damir-sudar-2/</a>
Visiting Scientist, Oregon Health & Science University</pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset> <br>
<pre wrap="">_______________________________________________
ome-devel mailing list
<a class="moz-txt-link-abbreviated" href="mailto:ome-devel@lists.openmicroscopy.org.uk">ome-devel@lists.openmicroscopy.org.uk</a>
<a class="moz-txt-link-freetext" href="http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel">http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Eleanor Williams PhD
Data Annotator/Software Engineer
Centre for Gene Regulation and Expression
University of Dundee, UK</pre>
<br>
<span style="font-size:10pt;">The University of Dundee is a registered Scottish Charity, No: SC015096</span>
</body>
</html>