[ome-devel] Fwd: [IDR Project] Zegami / IDR

Mon Jun 27 03:04:11 BST 2016

Hi Eleanor and Josh,

I'm learning a lot from your work on putting the IDR together and I'm 
applying it for our image repository for our LINCS project 
(http://www.lincsproject.org/centers/data-and-signature-generating-centers/oregon-u/) 
that hopefully will go live in a few months. We're also looking at 
Zegami as a possible way to provide visualizations of that data (Great 
demo, Roger!! and thanks for the code for that).

I've gotten as far as putting my metadata into a a format that can be 
used to generate the omero tables using the Populate Metadata script. 
But I couldn't find how you guys then populate the Key-Value 
Annotations. It appears to be a subset of what's in the Tables. Do you 
have some magic scripts for that and are those available?

Cheers,
- Damir

On 6/16/2016 2:21 AM, Eleanor Williams wrote:
>
> Hi Steve and Roger
>
> I think idr0002-heriche-condensation would be a good screen to start 
> with.  It is relatively small at 12 plates and has good annotation 
> with control, quality control and phenotype values. It can be found 
> here http://idr-demo.openmicroscopy.org/webclient/?show=screen-102.
>
> I am doing a bit of a stock-take of the annotation files I've produced 
> so far and will be changing a few aspects over the next few weeks, 
> mainly to include more of the information that is currently held in 
> the study files.  For example I'd like to get more of the sample 
> attributes such as species and cell line into the annotation.csv file, 
> and then into the omero tables and Map Annotations.  Sample attributes 
> for screens were originally recorded in the study file because it 
> didn't vary per screen, but now that we are also looking at non-screen 
> data where this might vary it makes sense to always have it visible 
> with the images.  Also we haven't got as far as exposing much of the 
> study information in omero yet.
>
> I would really like to work with you to understand how you are going 
> to use the experimental and analytic annotations and how we can help 
> in providing the metadata you need and in the most usable format.  For 
> example would it be useful if I indicate what is sample attribute 
> information in the annotation.csv file by using something like a 
> constistent header format e.g. Characterisics [Organism], 
> Characteristics [Cell Line], Characteristics [Organism Part] etc, or 
> is this not really necessary for your purposes and 'Organism', 'Cell 
> Line' column headings would be enough?
>
> Finally, it was Balaji Ramalingam 
> <https://www.openmicroscopy.org/site/about/development-teams/jason/balaji.png/view> 
> that you were talking to about streaming the thumbnails.  I'm sure he 
> would be happy to discuss this further with you.
>
> Best regards
>
> Eleanor
>
>
>
>
> On 15/06/2016 20:59, Josh Moore wrote:
>> Forwarding this to the mailing list so that anyone who had expressed
>> interested inhttp://zegami.com/  at this year's users' meeting can
>> follow along.
>>
>> Cheers,
>> ~Josh
>>
>> P.S. If you weren't able to attend #OME2016, a recap is now up on the
>> bloghttp://blog.openmicroscopy.org/community/2016/06/15/user-meeting/
>>
>>
>> ---------- Forwarded message ----------
>> From: Stephen Taylor<stephen.taylor at imm.ox.ac.uk>
>> Date: Tue, Jun 14, 2016 at 8:17 AM
>> Subject: Re: [IDR Project] Zegami / IDR
>> Cc: Roger Noble<roger at zegami.com>
>>
>> Hi,
>>
>> I’ve been look at a few examples from the link Josh sent to me and
>> there are quite a lot! What would be a good data set to get import
>> into Zegami from IDR?  I am thinking a set that has decent metadata
>> and a study that people are more familiar with that would make an
>> interesting use case.
>>
>> Kind regards,
>> Steve
>>
>>
>> From: Stephen Taylor
>> Sent: 10 June 2016 14:25
>> Cc: Roger Noble<roger at zegami.com>; 'e.x.williams at dundee.ac.uk'
>> <e.x.williams at dundee.ac.uk>
>> Subject: RE: [IDR Project] Zegami / IDR
>>
>> Hi all,
>>
>> I am interested generally in getting data in and out of OMERO from
>> both internal and external. Over a WAN the images are less of an issue
>> since they are streamed on demand but large metadata on the other hand
>> can get out of hand since it all has to be loaded completely into
>> Zegami’s database.
>>
>> Currently I do “OMERO project” to “Zegami collection” using a Zegami
>> export script (modified version of the OMERO batch export script).
>> This takes a project or selection of images from OMERO and produces
>> thumbnails and data in tab delimited Zegami format and then I manually
>> import that into Zegami for processing. The OMERO id is preserved so I
>> can link to the original image in OMERO using the OMERO web client. I
>> only export certain metadata currently (filenames and OMERO tags) and
>> my plan is to make this more comprehensive and allow MapAnnotations
>> and Tables to be exported using this mechanism as well. I need to find
>> out more how to query these data types.  Since we have internal
>> projects at Oxford that need this I’d like to finish this and make
>> that plugin freely available for other OMERO users to publish their
>> data in Zegami.
>>
>>
>> Longer term for IDR/Zegami
>>
>>
>> One of your team (apologies I didn’t get his name but he was doing
>> OMERO/ImageJ in the Unconference session) suggested we just stream in
>> the thumbnail and just have 1 level (i.e. no zoom) and when you want
>> to see the image just open it up in OMERO. This would not be as nice
>> to use but may be sufficient for HCS and would allow you to use
>> Zegami’s facet based query engine and would be probably little (no?)
>> work for you guys. We need to test this to see what it looks like and
>> it assumes OMERO can supply a unique thumbnail URL for each image and
>> that the metadata will provide a primary key so that we can get to
>> this image via a thumbnail id. To get at the metadata Josh and Eleanor
>> said they were thinking of making the CSV downloadable for a
>> collection, which could be loaded into Zegami on the fly (we have
>> instances where we dynamically load metadata via URL already).
>> However, major caveat is on how well this would perform over a WAN
>> with a large amount of data.
>>
>> BTW we have developer docs for Zegami for those interested at:
>>
>> https://support.zegami.com/hc/en-us/categories/200350657-Developers
>>
>> and would welcome feedback.
>>
>> Kind regards and thanks,
>>
>> Steve
>>
>>
>> From: Jason Swedlow (Staff) [mailto:j.r.swedlow at dundee.ac.uk]
>> Sent: 08 June 2016 13:09
>> Cc: Eleanor Williams (Staff)<e.x.williams at dundee.ac.uk>
>> Subject: Re: [IDR Project] Zegami / IDR
>>
>>
>> Hi Josh et al—
>>
>> My two cents— can we consider this use case from an external users’
>> perspective, mindful that distribution of such a solution would
>> definitely cause us to  consider how we ship thumbnails, handle
>> straws, etc.
>>
>> Cheers,
>> Jason
>>
>>
>>
>> On Wed, Jun 8, 2016 at 1:31 PM, Josh Moore<josh at glencoesoftware.com>  wrote:
>>
>> Stephen,
>>
>>
>> On Wed, Jun 8, 2016 at 9:52 AM, Stephen Taylor
>> <stephen.taylor at imm.ox.ac.uk>  wrote:
>>
>>> Hi Josh,
>>>
>>>
>>> Yes! What would be the best way of bulk downloading the thumbnails for a screen or plate
>> Interesting question. Guess the reverse question is: do you want to do
>> this as an external user or as if you had access to the OMERO?
>>
>>
>>> ( I notice the "Run Script" button isn't active ).
>> Correct. This server is stripped down in several ways as a public resource.
>> ~J.
>>
>>
>>> Kind regards,
>>> Steve
>> -----Original Message-----
>> From: Josh Moore [mailto:josh at glencoesoftware.com]
>> Sent: 08 June 2016 08:07
>> To: Stephen Taylor<stephen.taylor at imm.ox.ac.uk>
>> Cc: Eleanor Williams<exwilliams at dundee.ac.uk>;
>> e.x.williams at dundee.ac.uk;j.a.moore at dundee.ac.uk
>>
>> Subject: Re: Zegami / IDR
>>
>>
>> Hi Stephen,
>>
>>
>> On Wed, Jun 8, 2016 at 8:40 AM, Stephen Taylor
>> <stephen.taylor at imm.ox.ac.uk>  wrote:
>>
>>> Hi Eleanor,
>>>
>>> Thanks very useful. Is the web site publically available?
>> You meanhttp://idr-demo.openmicroscopy.org/webclient  ?
>>
>>
>> Cheers,
>> ~Josh
>>
>>
>>> Kind regards and thanks,
>>>
>>>
>>> Steve
>>> From: Eleanor Williams [mailto:exwilliams at dundee.ac.uk]
>>> Sent: 07 June 2016 11:09
>>> To: Stephen Taylor<stephen.taylor at imm.ox.ac.uk>;
>>> e.x.williams at dundee.ac.uk;j.a.moore at dundee.ac.uk
>>> Subject: Re: Zegami / IDR
>>>
>>>
>>>
>>> Hi Stephen
>>>
>>> Here is a link to our workshop presentation but it describes how to
>>> get the data into omero rather than out.
>>>
>>> http://downloads.openmicroscopy.org/presentations/2016/Users-Meeting/Workshops/Metadata-at-Scale.pdf
>>>
>>> We will be adding the annotation.csv files as attachments to each
>>> screen over the next couple of weeks.
>>>
>>> Let us know if you have any more questions.
>>>
>>> Best regards
>>>
>>> Eleanor
>>
>>> On 07/06/2016 10:41, Stephen Taylor wrote:
>>>
>>> Hi Eleanor and Josh,
>>>
>>> Great to meet you both at the OME meeting last week.
>>>
>>>
>>> I'd be interested in trying get some data into Zegami from IDR. I
>>> missed your Unmeeting (ironically I was demoing Zegami) where I think
>>> you went into some of the technical details. Can you send me the link
>>> to your presentation please?
>>>
>>>
>>> Kind regards and thanks,
>>> Steve
>> _______________________________________________
>> ome-devel mailing list
>> ome-devel at lists.openmicroscopy.org.uk
>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>
> -- 
> Eleanor Williams PhD
>
> Data Annotator/Software Engineer
> Centre for Gene Regulation and Expression
> University of Dundee, UK
>
> The University of Dundee is a registered Scottish Charity, No: SC015096
>
>
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel

-- 
Damir Sudar - Affiliate Scientist
Lawrence Berkeley Natl Laboratory / MBIB
One Cyclotron Road, MS 977, Berkeley, CA 94720, USA
T: 510/486-5346 - F: 510/486-5586 - E: DSudar at lbl.gov
http://biosciences.lbl.gov/profiles/damir-sudar-2/

Visiting Scientist, Oregon Health & Science University

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20160626/9440afaa/attachment.html>