[ome-devel] Can get raw images (vs. compressed) for several datasets?

[Josh Moore]

Hi Jon,

On Mon, May 1, 2017 at 11:50 PM, Jon Hazard <jhazard at google.com> wrote:
> Hi Josh - replies to your questions inline below
>
> On Mon, May 1, 2017 at 1:19 PM, Josh Moore <josh at glencoesoftware.com> wrote:
>>
>> (List members: sorry for the re-send)
>>
>> On Fri, Apr 28, 2017 at 8:07 PM, Jon Hazard <jhazard at google.com> wrote:
>> > Hello - first, a hearty thanks to OME for making this data available!
>>
>> Gladly! And glad to you're interesting in using it.
>>
>>
>> > My team is interested in the images from two datasets:
>> >
>> > idr0033-rohban-pathways/screenA
>> >
>> > and
>> >
>> > idr0028-pascualvargas-rhogtpases
>> >
>> > It looks like the images for at least the first data set are compressed.
>>
>> How do you mean compressed? (Can you send a link?) The original data
>> has been uploaded to the IDR but certainly for viewing, rendering is
>> performed server-side before sending to the client.
>
>
> To clarify, I meant that the images I can obtain are lower-resolution JPGs
> rather than the original-quality TIFF files. This is true whether I export
> through either the individual image download (by clicking the Down Arrow as
> shown in the first section of the export help page
> http://help.openmicroscopy.org/export.html) or by using the public web API
> (e.g. by issuing the following command: curl
> "http://idr-demo.openmicroscopy.org/webclient/render_image/3194301/?c=1&q=1"
>> image.jpg).

I understand: there's not yet a webclient/ or webgateway/ yet that
would return raw image planes. Our other APIs (e.g. Python, Java, C++)
do. See below.


>> > It would help us a lot to be able to work with the raw, uncompressed
>> > images.
>> > Are those available?
>>
>> Can you explain more about what you are trying to achieve? We're
>> working on having full download enabled from EBI but until then we
>> should be able to get you access to something more significant.
>
>
> I would like to have access to the original-quality images for all images in
> the two datasets I mentioned previously:
>
>> idr0028-pascualvargas-rhogtpases and idr0033-rohban-pathways/screenA
>
> That is great that the full downloads will eventually be supported through
> EBI. Is there an expected timeframe for that capability to come online?

I've checked on the status, but there's not yet a concrete date. My
guess would be that it'll be on the order of weeks rather than months.


> Assuming that there is some non-trivial infrastructure required there, is
> there any other way that we could have access to those two datasets in a
> more timely manner?

The OME project regularly makes sample data available for download [1]
. To at least unblock you initially, I'd propose putting one plate for
each study up on the downloads site (10GB and 40GB respectively), but
we'd certainly like to avoid mirroring the dozens of TBs from IDR via
that mechanism. One short-term option for accessing all the plate data
would be for you to make use of our jupyter instance[2]. See
https://github.com/IDR/idr-notebooks for examples.

All the best,
~Josh

[1] https://downloads.openmicroscopy.org/images/
[2] https://idr.openmicroscopy.org/jupyter