[ome-devel] OMERO URLs of original images from a plate

Mon Feb 13 15:45:15 GMT 2017

Hi Harri

Just to clarify few points.

1. Which CellProfiler and OMERO server version are you using? Did you manage to connect to OMERO?

2. Based on naming convention LMU-CELLINSIGHT_170111150001_B03f00d0.C01 we understood that channels are stored per file (are these Cellomics files)?

>From the documentation (https://github.com/CellProfiler/CellProfiler/wiki/OMERO:-Accessing-images-from-CellProfiler) looks like CellProfiler decides which communication protocol to use, depending on the schema. For OMERO, it uses the OMERO client library (java jar) to fetch and load the data rather then HTTP. In your case we suspect that URL omero:iid=imageID would need to be accompanied with more parameters as raw data files are per channel. You could try using CellProfiler metadata module to see if it could help you to build cvs and import to Image module.

3. Alternatively you could try writing your own script to use RawFileStore http://downloads.openmicroscopy.org/omero/5.2.7/api/ome/api/RawFileStore.html directly to download single files (channels) for analysis, instead. That may be more efficient, then downloading entire plate via web api.
Here https://github.com/openmicroscopy/openmicroscopy/blob/develop/examples/Training/python/Filesets.py is an example (https://github.com/openmicroscopy/openmicroscopy/blob/develop/components/tools/OmeroPy/src/omero/gateway/__init__.py#L9335 and hql query if you need that written in Java).

Balaji prototyped analysis script for ImageJ showing how to run it headless directly from OMERO.server  https://github.com/bramalingam/scripts/blob/ImageJ_server_side/omero/ImageJ_scripts/Batch_Analysis_IJMacro.py#L384. Would you like to check it as well?

How would you like to store results? Are you going to save them back to Omero?

I hope that helps
Ola

On 13 Feb 2017, at 10:36, William Moore (Staff) <W.Moore at dundee.ac.uk<mailto:W.Moore at dundee.ac.uk>> wrote:

Hi Harri,

 In general I would imagine that downloading a whole plate of data and processing it locally is not the best solution.

 - By default, the OMERO web and Insight clients don’t allow you to download a whole plate of data because of the burden this puts on the server. In the case of web, the web code tries to create a single zip file (since the web can’t access your file system to download many separate files). However, for SPW data of any reasonable size, creating the zip will typically take longer than your web server (nginx/apache) timeout (1 minute) so you’d need to configure this to be a longer time.

 - If you download the SPW data and write a script to parse it, you have to treat every different SPW format differently. Whereas using the OMERO API, all different SPW formats would be accessed the same way.

 - I’m not so familiar with the way that CellProfiler accesses data from OMERO, but I imagine the strategy works for most users and that "several parallel downloads of individual images from the OMERO server” is not a problem.

 - When accessing single channels and images of a Plate, you will not be unnecessarily downloading any extra images or channels that you don’t need.

To enable download of Plates, see the configuration of omero.policy.binary_access:
 https://www.openmicroscopy.org/site/support/omero5.2/sysadmins/config.html#omero-policy-binary-access
E.g: to allow plate download:
$ bin/omero config set omero.policy.binary_access "+read,+write,+image,+plate”
(you will probably need to restart OMERO after this).

Then you should be able to download the plate from any image ID in the plate (every image is linked to all the SPW files).
This url will build and download the plate zip for the specified image:
/webgateway/archived_files/download/?image=62845&zipname=myData.zip

In your original strategy of building a list of image IDs from a plate,
I would suggest a python OMERO.script, which can be called from the web or Insight client to create a .csv file to download.

Using the python Blitz Gateway it’s quite easy to list images for a plate:

plate = conn.getObject("Plate", 5017)

for well in plate.listChildren():
     print "Well: ", well.id
     for well_sample in well.listChildren():
         print "  ws:", well_sample.id
         image = well_sample.getImage()
         print "  image:", image.id, image.name

A .csv file could be attached to the Plate object and returned by the script.
See “How to create a file annotation…” at https://www.openmicroscopy.org/site/support/omero5.2/developers/Python.html

See also “Introduction to OMERO scripts” at https://www.openmicroscopy.org/site/support/omero5.2/developers/scripts/index.html
If you need any help turning this into an OMERO.script let us know.

Hope this helps, whichever strategy you find works best,

 Regards,

   Will.

On 12 Feb 2017, at 06:28, Jäälinoja, Harri Tapio <harri.jaalinoja at helsinki.fi<mailto:harri.jaalinoja at helsinki.fi>> wrote:

Hi Ola,

thanks a lot for your reply! With merged images I mean the ones you see in the plate viewer, not individual C01 per channel, but a multichannel image.

Anyway, after sleeping over this a couple of nights, I started to wonder if what I was trying to do was a good idea to begin with. Let's say I have CellProfiler on a cluster, and I start to process an image list with "omero::iid=nnn" entries. That would mean several parallel downloads of individual images from the OMERO server, right? I noticed there's a link in the OMERO plate viewer to download the entire plate in a an archive. Can I find the archive URL programmatically? Then I could just download that once, unzip and sort out the paths in a script. And exclude images with an error tag found with your second example.

I'll try this tomorrow. Thanks again!

Best,
Harri

________________________________________
From: ome-devel <ome-devel-bounces at lists.openmicroscopy.org.uk<mailto:ome-devel-bounces at lists.openmicroscopy.org.uk>> on behalf of Aleksandra Tarkowska (Staff) <A.Tarkowska at dundee.ac.uk<mailto:A.Tarkowska at dundee.ac.uk>>
Sent: 10 February 2017 17:31:57
To: OME External Developer List
Subject: Re: [ome-devel] OMERO URLs of original images from a plate

Hi

What do you mean by merged images? Did you have a chance to check HCS structures (Screen, Plate and Well) of the OME Model in http://www.openmicroscopy.org/site/support/ome-model/developers/model-overview.html?

If you query DB directly to see how that is structured you could try something like

SQL:
select w.id as well_id, ws.well_index as field, i.id as image_id
from image as i
   join wellsample as ws on (ws.image=i.id)
   join well as w on (ws.well=w.id)
   join plate as p on (w.plate=p.id)
   join screenplatelink as spl on (spl.child=p.id)
where spl.parent = :screen_id

Using omero api you could try HQL query and filter by tag value:

ParametersI param = new ParametersI();
param.map.put("sid", omero.rtypes.rlong(102));

StringBuffer sb = new StringBuffer();
sb.append("select new map( w.id as wellid, index(ws) as field, i.id as imgid) ");
sb.append("from Well w ");
sb.append("join w.wellSamples ws ");
sb.append("join ws.image i ");
sb.append("join i.annotationLinks ial ");
sb.append("join ial.child tag ");
sb.append("join w.plate pl ");
sb.append("join pl.screenLinks sl ");
sb.append("join sl.parent s ");
sb.append("where s.id = :sid “);
sb.append("and tag.class = TagAnnotation and tag.textValue = 'error' “);

for (final List<RType> result : queryService.projection(sb.toString(), param)) {
   System.out.println(omero.rtypes.unwrap(result.get(0)));
}

depends on which version of OMERO.jar you are using annotation discriminator may be /basic/text/tag/ (this was fixed in 5.3-develop to map the class TagAnnotation)

Hope that helps

Kind regards

Ola
Software Engineer
Open Microscopy Environment
University of Dundee

On 9 Feb 2017, at 15:36, Jäälinoja, Harri Tapio <harri.jaalinoja at helsinki.fi<mailto:harri.jaalinoja at helsinki.fi><mailto:harri.jaalinoja at helsinki.fi>> wrote:

Hi,

this is continuation of my previous question about browsing a plate with the Java API. I've uploaded a couple of plates, in the database it looks like this:

select id,name from image order by id;

id  |                              name
------+-----------------------------------------------------------------
  1 | LMU-CELLINSIGHT_170111150001_B03f00d0.C01 [Well B03, Field #00]
...
1920 | LMU-CELLINSIGHT_150428120001_B05f00d0.C01 [Well B05, Field #15]
(1920 rows)

So the database only has the merged images of each field in well. The original files are on the OMERO server, but it seems it's not possible to find them with a URL like "omero::iid=123"?

What I have in mind is to pick a plateID and build a corresponding CSV formatted image list for CellProfiler LoadData module (see https://github.com/CellProfiler/CellProfiler/wiki/OMERO:-Accessing-images-from-CellProfiler). Something like:

URL_DAPI,URL_GFP,URL_CY5
omero::iid=123,omero::iid=124,omero::iid=125

(Ideally the image list would exclude fields tagged by the user as imaging errors.)
Is this possible at the moment? If not, would it a reasonable request?

Best,
Harri

---
Harri Jäälinoja
Light Microscopy Unit
Institute of Biotechnology
University of Helsinki
_______________________________________________
ome-devel mailing list
ome-devel at lists.openmicroscopy.org.uk<mailto:ome-devel at lists.openmicroscopy.org.uk><mailto:ome-devel at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel

The University of Dundee is a registered Scottish Charity, No: SC015096
_______________________________________________
ome-devel mailing list
ome-devel at lists.openmicroscopy.org.uk<mailto:ome-devel at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel

The University of Dundee is a registered Scottish Charity, No: SC015096
_______________________________________________
ome-devel mailing list
ome-devel at lists.openmicroscopy.org.uk<mailto:ome-devel at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel

The University of Dundee is a registered Scottish Charity, No: SC015096
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20170213/1e9596af/attachment.html>