[ome-users] 12k x 12k limits for ROI extraction/manipulation - are there fixes in the pipeline?

[William Moore (Staff)]

Hi Jake,

 Just to clarify a little:

 The “12k x 12k” pixel download limit is only for “Export as TIFF/JPEG/PNG” and can be configured
https://www.openmicroscopy.org/site/support/omero5.2/sysadmins/config.html#omero-client-download-as-max-size

This limit is dependent on the maximum plane size that can be rendered by the rendering engine,
since we get an OOM error in omeis.providers.re.RGBIntBuffer.
You can try to see if your infrastructure might support a higher limit.

Export as OME-TIFF doesn’t support tiled images (default limit for this is about 3k x 3k but is also configurable).

However, there is no fixed limit for downloads of the original image data.
This is purely dependent on your bandwidth and other infrastructure.


The changes to the “Images from ROIs” script that adds support for big images are in this PR:
https://github.com/ome/scripts/pull/115
If you are happy to download the latest version of that script and update it on your server
then you wouldn’t have to wait for back-porting and point release.


Regards,

  Will.



On 25 Oct 2016, at 20:09, Jake Carroll <jake.carroll at uq.edu.au<mailto:jake.carroll at uq.edu.au>> wrote:

Hi Jean-Marie.

Thanks for getting back to me. Most appreciated.

>> We currently have a limit on the size of the image that can downloaded.

Yes, and this worries people locally a great deal. They feel like their data “becomes trapped”. They can absolutely get the data IN to OMERO, but from a preservation perspective it becomes questionable to them if they can’t get *the same* data back out. From a data-governance and provenance perspective, I’d really like to see this fixed. It would allow us to adhere to several strict federal regulations and laws by doing so – and it’d make the scientists feel a lot more secure in their knowledge that OMERO is an authoritative data sink. If data can only go “one way” it represents a risk to scientists in some respects, from a digital preservation and data governance perspective, as you can appreciate.

>> By ROI extraction, are you thinking of creating ROI on some of those large images and “cropping” to generate a new image then download that image?

Absolutely correct, yes.

>> A new script is available in OMERO 5.3, https://github.com/ome/scripts/pull/115 to generate tiles images from ROIs

That could be exceedingly useful to us and I’d welcome it.

>> We are considering a 5.2.7 release to add support for ice 3.6.3
>> So we could certainly back port the script  mentioned above if it solved some of the issues.

You’d absolutely make a lot of progress for us by doing this and we’d be exceedingly appreciative. These things have been long-term niggling issues with complete OMERO adoption and total trust for a very long time. Fixing these things would boost confidence in the researchers and make OMERO a more ubiquitous platform for all.

Regards, and thanks again for your response. It is heartening.

-jc

From: "Jean-Marie Burel (Staff)" <j.burel at dundee.ac.uk<mailto:j.burel at dundee.ac.uk>>
Date: Wednesday, 26 October 2016 at 07:02
To: Jake Carroll <jake.carroll at uq.edu.au<mailto:jake.carroll at uq.edu.au>>, "ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>" <ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>>
Cc: "Sebastien Besson (Staff)" <s.besson at dundee.ac.uk<mailto:s.besson at dundee.ac.uk>>
Subject: Re: 12k x 12k limits for ROI extraction/manipulation - are there fixes in the pipeline?

Dear Jake

We currently have a limit on the size of the image that can downloaded.
By ROI extraction, are you thinking of creating ROI on some of those large images and “cropping” to generate a new image then download that image?

A new script is available in OMERO 5.3, https://github.com/ome/scripts/pull/115 to generate tiles images from ROIs

We are considering a 5.2.7 release to add support for ice 3.6.3
So we could certainly back port the script  mentioned above if it solved some of the issues.


Cheers

Jmarie

From: Jake Carroll <jake.carroll at uq.edu.au<mailto:jake.carroll at uq.edu.au>>
Date: Tuesday, 25 October 2016 03:50
To: "ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>" <ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>>
Cc: jmarie burel <j.burel at dundee.ac.uk<mailto:j.burel at dundee.ac.uk>>, "Sebastien Besson (Staff)" <s.besson at dundee.ac.uk<mailto:s.besson at dundee.ac.uk>>
Subject: 12k x 12k limits for ROI extraction/manipulation - are there fixes in the pipeline?

Hi all.

I was just wondering if there had been any consideration given to the 12k x 12k pixel limit on certain ROI Extraction/manipulation semantics in OMERO 5.3 for your road map? From what I can tell, in OMERO 5.2.x series code, there still exists limitations on how large an extraction of an ROI can be, and the ability to manipulate data beyond 12k x 12k pixel wide dimensions.

We’re now on a daily basis, getting significant requests/complaints from people ingesting very big things (100k x 100k pixel style stuff) who’d dearly love to work with their data – but can’t ROI extract, nor work with it anymore, due to the fact that there seems to be a 12k x 12k limit on file format transforms (I assume it is a BIOFormats thing/BIgTIFF thing/OMETIFF limitation).

Would like to know if any progress had been made with this limitation – it’d kind of liberate OMERO for us in an amazing way/really make a lot of scientists smile…

Thank you.

-jc

The University of Dundee is a registered Scottish Charity, No: SC015096
_______________________________________________
ome-users mailing list
ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users


The University of Dundee is a registered Scottish Charity, No: SC015096
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20161026/767b5b10/attachment.html>