<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
Hi Harri
<div class=""><br class="">
</div>
<div class="">
<div class="">Just to clarify few points.</div>
</div>
<div class=""><br class="">
</div>
<div class="">1. Which CellProfiler and OMERO server version are you using? Did you manage to connect to OMERO?</div>
<div class="">
<div class=""><br class="">
</div>
</div>
<div class="">2. Based on naming convention LMU-CELLINSIGHT_170111150001_B03f00d0.C01 we understood that channels are stored per file (are these Cellomics files)?</div>
<div class=""><br class="">
</div>
<div class="">
<div class="">From the documentation (<a href="https://github.com/CellProfiler/CellProfiler/wiki/OMERO:-Accessing-images-from-CellProfiler" class="">https://github.com/CellProfiler/CellProfiler/wiki/OMERO:-Accessing-images-from-CellProfiler</a>) looks like
CellProfiler decides which communication protocol to use, depending on the schema. For OMERO, it uses the OMERO client library (java jar) to fetch and load the data rather then HTTP. In your case we suspect that URL
<i class="">omero:iid=imageID </i>would need to be accompanied with more parameters as raw data files are per channel. You could try using CellProfiler metadata module to see if it could help you to build cvs and import to Image module. </div>
</div>
<div class=""><br class="">
</div>
<div class="">3. Alternatively you could try writing your own script to use RawFileStore
<a href="http://downloads.openmicroscopy.org/omero/5.2.7/api/ome/api/RawFileStore.html" class="">
http://downloads.openmicroscopy.org/omero/5.2.7/api/ome/api/RawFileStore.html</a> directly to download single files (channels) for analysis, instead. That may be more efficient, then downloading entire plate via web api.</div>
<div class="">Here <a href="https://github.com/openmicroscopy/openmicroscopy/blob/develop/examples/Training/python/Filesets.py" class="">https://github.com/openmicroscopy/openmicroscopy/blob/develop/examples/Training/python/Filesets.py</a> is an example (<a href="https://github.com/openmicroscopy/openmicroscopy/blob/develop/components/tools/OmeroPy/src/omero/gateway/__init__.py#L9335" class="">https://github.com/openmicroscopy/openmicroscopy/blob/develop/components/tools/OmeroPy/src/omero/gateway/__init__.py#L9335</a>
and hql query if you need that written in Java).</div>
<div class=""><br class="">
</div>
<div class="">Balaji prototyped analysis script for ImageJ showing how to run it headless directly from OMERO.server <a href="https://github.com/bramalingam/scripts/blob/ImageJ_server_side/omero/ImageJ_scripts/Batch_Analysis_IJMacro.py#L384" class="">https://github.com/bramalingam/scripts/blob/ImageJ_server_side/omero/ImageJ_scripts/Batch_Analysis_IJMacro.py#L384</a>.
Would you like to check it as well?</div>
<div class=""><br class="">
</div>
<div class="">How would you like to store results? Are you going to save them back to Omero?</div>
<div class=""><br class="">
</div>
<div class="">I hope that helps</div>
<div class="">Ola</div>
<div class=""><br class="">
</div>
<div>
<blockquote type="cite" class="">
<div class="">On 13 Feb 2017, at 10:36, William Moore (Staff) <<a href="mailto:W.Moore@dundee.ac.uk" class="">W.Moore@dundee.ac.uk</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
Hi Harri,
<div class=""><br class="">
</div>
<div class=""> In general I would imagine that downloading a whole plate of data and processing it locally is not the best solution.</div>
<div class=""><br class="">
</div>
<div class=""> - By default, the OMERO web and Insight clients don’t allow you to download a whole plate of data because of the burden this puts on the server. In the case of web, the web code tries to create a single zip file (since the web can’t access your
file system to download many separate files). However, for SPW data of any reasonable size, creating the zip will typically take longer than your web server (nginx/apache) timeout (1 minute) so you’d need to configure this to be a longer time.</div>
<div class=""><br class="">
</div>
<div class=""> - If you download the SPW data and write a script to parse it, you have to treat every different SPW format differently. Whereas using the OMERO API, all different SPW formats would be accessed the same way.</div>
<div class=""><br class="">
</div>
<div class=""> - I’m not so familiar with the way that CellProfiler accesses data from OMERO, but I imagine the strategy works for most users and that "several parallel downloads of individual images from the OMERO server” is not a problem.</div>
<div class=""><br class="">
</div>
<div class=""> - When accessing single channels and images of a Plate, you will not be unnecessarily downloading any extra images or channels that you don’t need.</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">To enable download of Plates, see the configuration of omero.policy.binary_access:</div>
<div class=""> <a href="https://www.openmicroscopy.org/site/support/omero5.2/sysadmins/config.html#omero-policy-binary-access" class="">https://www.openmicroscopy.org/site/support/omero5.2/sysadmins/config.html#omero-policy-binary-access</a> </div>
<div class="">E.g: to allow plate download:</div>
<div class="">$ bin/omero config set omero.policy.binary_access "+read,+write,+image,+plate”</div>
<div class="">(you will probably need to restart OMERO after this).</div>
<div class=""><br class="">
</div>
<div class="">Then you should be able to download the plate from any image ID in the plate (every image is linked to all the SPW files).</div>
<div class="">This url will build and download the plate zip for the specified image:</div>
<div class="">/webgateway/archived_files/download/?image=62845&zipname=myData.zip</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">In your original strategy of building a list of image IDs from a plate,</div>
<div class="">I would suggest a python OMERO.script, which can be called from the web or Insight client to create a .csv file to download.</div>
<div class=""><br class="">
</div>
<div class="">Using the python Blitz Gateway it’s quite easy to list images for a plate:</div>
<div class=""><br class="">
</div>
<div class="">
<div style="margin: 0px; font-family: Courier; color: rgb(76, 47, 45); background-color: rgb(223, 219, 196);" class="">
plate = conn.getObject("Plate", 5017)</div>
<div style="margin: 0px; font-family: Courier; color: rgb(76, 47, 45); background-color: rgb(223, 219, 196); min-height: 14px;" class="">
<br class="">
</div>
<div style="margin: 0px; font-family: Courier; color: rgb(76, 47, 45); background-color: rgb(223, 219, 196);" class="">
for well in plate.listChildren():</div>
<div style="margin: 0px; font-family: Courier; color: rgb(76, 47, 45); background-color: rgb(223, 219, 196);" class="">
<span style="font-variant-ligatures: no-common-ligatures; color: #4329d4" class=""> </span> print "Well: ", well.id</div>
<div style="margin: 0px; font-family: Courier; color: rgb(76, 47, 45); background-color: rgb(223, 219, 196);" class="">
<span style="font-variant-ligatures: no-common-ligatures; color: #4329d4" class=""> </span> for well_sample in well.listChildren():</div>
<div style="margin: 0px; font-family: Courier; color: rgb(76, 47, 45); background-color: rgb(223, 219, 196);" class="">
<span style="font-variant-ligatures: no-common-ligatures; color: #4329d4" class=""> </span> print " ws:", well_sample.id</div>
<div style="margin: 0px; font-family: Courier; color: rgb(76, 47, 45); background-color: rgb(223, 219, 196);" class="">
<span style="font-variant-ligatures: no-common-ligatures; color: #4329d4" class=""> </span> image = well_sample.getImage()</div>
<div style="margin: 0px; font-family: Courier; color: rgb(76, 47, 45); background-color: rgb(223, 219, 196);" class="">
<span style="font-variant-ligatures: no-common-ligatures; color: #4329d4" class=""> </span> print " image:", image.id, image.name</div>
</div>
<div class=""><br class="">
</div>
<div class="">A .csv file could be attached to the Plate object and returned by the script.</div>
<div class="">See “How to create a file annotation…” at <a href="https://www.openmicroscopy.org/site/support/omero5.2/developers/Python.html" class="">https://www.openmicroscopy.org/site/support/omero5.2/developers/Python.html</a> </div>
<div class=""><br class="">
</div>
<div class="">See also “Introduction to OMERO scripts” at <a href="https://www.openmicroscopy.org/site/support/omero5.2/developers/scripts/index.html" class="">https://www.openmicroscopy.org/site/support/omero5.2/developers/scripts/index.html</a></div>
<div class="">If you need any help turning this into an OMERO.script let us know.</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">Hope this helps, whichever strategy you find works best,</div>
<div class=""><br class="">
</div>
<div class=""> Regards,</div>
<div class=""><br class="">
</div>
<div class=""> Will.</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
<div class="">
<blockquote type="cite" class="">
<div class="">On 12 Feb 2017, at 06:28, Jäälinoja, Harri Tapio <<a href="mailto:harri.jaalinoja@helsinki.fi" class="">harri.jaalinoja@helsinki.fi</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">Hi Ola,<br class="">
<br class="">
thanks a lot for your reply! With merged images I mean the ones you see in the plate viewer, not individual C01 per channel, but a multichannel image.<br class="">
<br class="">
Anyway, after sleeping over this a couple of nights, I started to wonder if what I was trying to do was a good idea to begin with. Let's say I have CellProfiler on a cluster, and I start to process an image list with "omero::iid=nnn" entries. That would mean
several parallel downloads of individual images from the OMERO server, right? I noticed there's a link in the OMERO plate viewer to download the entire plate in a an archive. Can I find the archive URL programmatically? Then I could just download that once,
unzip and sort out the paths in a script. And exclude images with an error tag found with your second example.<br class="">
<br class="">
I'll try this tomorrow. Thanks again!<br class="">
<br class="">
Best,<br class="">
Harri<br class="">
<br class="">
<br class="">
<br class="">
<br class="">
<br class="">
________________________________________<br class="">
From: ome-devel <<a href="mailto:ome-devel-bounces@lists.openmicroscopy.org.uk" class="">ome-devel-bounces@lists.openmicroscopy.org.uk</a>> on behalf of Aleksandra Tarkowska (Staff) <<a href="mailto:A.Tarkowska@dundee.ac.uk" class="">A.Tarkowska@dundee.ac.uk</a>><br class="">
Sent: 10 February 2017 17:31:57<br class="">
To: OME External Developer List<br class="">
Subject: Re: [ome-devel] OMERO URLs of original images from a plate<br class="">
<br class="">
Hi<br class="">
<br class="">
What do you mean by merged images? Did you have a chance to check HCS structures (Screen, Plate and Well) of the OME Model in
<a href="http://www.openmicroscopy.org/site/support/ome-model/developers/model-overview.html?" class="">
http://www.openmicroscopy.org/site/support/ome-model/developers/model-overview.html?</a><br class="">
<br class="">
If you query DB directly to see how that is structured you could try something like<br class="">
<br class="">
SQL:<br class="">
select w.id as well_id, ws.well_index as field, i.id as image_id<br class="">
from image as i<br class="">
join wellsample as ws on (ws.image=i.id)<br class="">
join well as w on (ws.well=w.id)<br class="">
join plate as p on (w.plate=p.id)<br class="">
join screenplatelink as spl on (spl.child=p.id)<br class="">
where spl.parent = :screen_id<br class="">
<br class="">
<br class="">
Using omero api you could try HQL query and filter by tag value:<br class="">
<br class="">
<br class="">
ParametersI param = new ParametersI();<br class="">
param.map.put("sid", omero.rtypes.rlong(102));<br class="">
<br class="">
StringBuffer sb = new StringBuffer();<br class="">
sb.append("select new map( w.id as wellid, index(ws) as field, i.id as imgid) ");<br class="">
sb.append("from Well w ");<br class="">
sb.append("join w.wellSamples ws ");<br class="">
sb.append("join ws.image i ");<br class="">
sb.append("join i.annotationLinks ial ");<br class="">
sb.append("join ial.child tag ");<br class="">
sb.append("join w.plate pl ");<br class="">
sb.append("join pl.screenLinks sl ");<br class="">
sb.append("join sl.parent s ");<br class="">
sb.append("where s.id = :sid “);<br class="">
sb.append("and tag.class = TagAnnotation and tag.textValue = 'error' “);<br class="">
<br class="">
for (final List<RType> result : queryService.projection(sb.toString(), param)) {<br class="">
System.out.println(omero.rtypes.unwrap(result.get(0)));<br class="">
}<br class="">
<br class="">
depends on which version of OMERO.jar you are using annotation discriminator may be /basic/text/tag/ (this was fixed in 5.3-develop to map the class TagAnnotation)<br class="">
<br class="">
<br class="">
Hope that helps<br class="">
<br class="">
Kind regards<br class="">
<br class="">
Ola<br class="">
Software Engineer<br class="">
Open Microscopy Environment<br class="">
University of Dundee<br class="">
<br class="">
On 9 Feb 2017, at 15:36, Jäälinoja, Harri Tapio <<a href="mailto:harri.jaalinoja@helsinki.fi" class="">harri.jaalinoja@helsinki.fi</a><<a href="mailto:harri.jaalinoja@helsinki.fi" class="">mailto:harri.jaalinoja@helsinki.fi</a>>> wrote:<br class="">
<br class="">
Hi,<br class="">
<br class="">
this is continuation of my previous question about browsing a plate with the Java API. I've uploaded a couple of plates, in the database it looks like this:<br class="">
<br class="">
select id,name from image order by id;<br class="">
<br class="">
id | name<br class="">
------+-----------------------------------------------------------------<br class="">
1 | LMU-CELLINSIGHT_170111150001_B03f00d0.C01 [Well B03, Field #00]<br class="">
...<br class="">
1920 | LMU-CELLINSIGHT_150428120001_B05f00d0.C01 [Well B05, Field #15]<br class="">
(1920 rows)<br class="">
<br class="">
So the database only has the merged images of each field in well. The original files are on the OMERO server, but it seems it's not possible to find them with a URL like "omero::iid=123"?<br class="">
<br class="">
What I have in mind is to pick a plateID and build a corresponding CSV formatted image list for CellProfiler LoadData module (see
<a href="https://github.com/CellProfiler/CellProfiler/wiki/OMERO:-Accessing-images-from-CellProfiler" class="">
https://github.com/CellProfiler/CellProfiler/wiki/OMERO:-Accessing-images-from-CellProfiler</a>). Something like:<br class="">
<br class="">
URL_DAPI,URL_GFP,URL_CY5<br class="">
omero::iid=123,omero::iid=124,omero::iid=125<br class="">
<br class="">
(Ideally the image list would exclude fields tagged by the user as imaging errors.)<br class="">
Is this possible at the moment? If not, would it a reasonable request?<br class="">
<br class="">
Best,<br class="">
Harri<br class="">
<br class="">
---<br class="">
Harri Jäälinoja<br class="">
Light Microscopy Unit<br class="">
Institute of Biotechnology<br class="">
University of Helsinki<br class="">
_______________________________________________<br class="">
ome-devel mailing list<br class="">
<a href="mailto:ome-devel@lists.openmicroscopy.org.uk" class="">ome-devel@lists.openmicroscopy.org.uk</a><<a href="mailto:ome-devel@lists.openmicroscopy.org.uk" class="">mailto:ome-devel@lists.openmicroscopy.org.uk</a>><br class="">
<a href="http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel" class="">http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel</a><br class="">
<br class="">
<br class="">
The University of Dundee is a registered Scottish Charity, No: SC015096<br class="">
_______________________________________________<br class="">
ome-devel mailing list<br class="">
<a href="mailto:ome-devel@lists.openmicroscopy.org.uk" class="">ome-devel@lists.openmicroscopy.org.uk</a><br class="">
<a href="http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel" class="">http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel</a><br class="">
</div>
</blockquote>
</div>
<br class="">
</div>
<br class="">
<span style="font-size:10pt;" class="">The University of Dundee is a registered Scottish Charity, No: SC015096</span>
</div>
_______________________________________________<br class="">
ome-devel mailing list<br class="">
<a href="mailto:ome-devel@lists.openmicroscopy.org.uk" class="">ome-devel@lists.openmicroscopy.org.uk</a><br class="">
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel<br class="">
</div>
</blockquote>
</div>
<br class="">
<br>
<span style="font-size:10pt;">The University of Dundee is a registered Scottish Charity, No: SC015096</span>
</body>
</html>