[ome-users] Import of larger 2D OME-TIFF fails

[Matthias Baldauf]

Hello all,

I am having problems with the import of larger 2D OME-TIFFs. The OME-TIFF is a multipage TIFF (three pages, one for each color-channel of the RGB-image), striped with 1 row per strip, 16384x9600 pixels, 450 MB, and 8bit, uncompressed.

For the import I use OMERO.importer 4.4.4 and the OMERO.server Virtual Appliance 4.4.4 with 4 GB RAM assigned. Importer and server are running on the same machine. The Host OS is Windows 7 64bit, 8 GB RAM and i7-2720QM CPU.
It takes about three minutes to analyze the OME-TIFF. The first two steps ("1/3" & "2/3") seem to work but import fails after 13 minutes during the last step with the following exception (the complete exception is attached as a text file to this email):

omero.InternalException
    serverStackTrace = "ome.conditions.InternalException:  Wrapped Exception: (java.lang.OutOfMemoryError):
                        Java heap space

So it seems the server is running out of memory, but I can see in the Task Manager that the 4 GB RAM of the server are never used completely during the import. The OMERO.importer has only a slight change in RAM usage for about 30 MB. The image is not a really large one, therefore I am wondering why the import fails?
I thought that this OME-TIFF format would be ideal for import because when I export small images in OMERO.insight the resulting OME-TIFF has the same structure. I also tried to import a tiled OME-TIFF (one RGB-image, 16384x9600 pixels, 456 MB, and 8bit, uncompressed) but import results in the same exception at step "3/3". The import of a smaller OME-TIFF (10.000 x 5.000 pixels, same settings as the multipage tiff) works and takes about six minutes.


I also have some additional questions about the import process:

Why does the analyze-step and the further steps take about 13 minutes in total until the import fails? The OME-TIFF is uncompressed and already split into single colors (one page for each color-channel) so I thought the import would be faster.

Am I right, that OMERO.importer reads the OME-TIFF section after section, so the whole OME-TIFF is not loaded into the RAM?

Does the OMERO.server need much RAM for the construction of an image pyramid and does OMERO.server need much RAM after the import (e.g. when viewing large images with 60.000 x 40.000 pixels)? I would like to import a few thousand large 2D images and view them.

Which criterion is decisive for not processing the image on client-side but uploading the raw files directly to the server?

What are the three import steps? Is the step "1/3" the reading and splitting into the three color-channels, step "2/3" the data-import on the server and "3/3" the construction of the image pyramid? Which steps are done locally and which steps are done on the server?

How is the concept of reading images during the import? Is it like: some image formats need to be loaded completely into the RAM in order to read, decompress and import them and other formats can be read section per section resulting in a low RAM usage?


Thank you in advance for your answers!

Regards,
Matthias

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: omero_exception.txt
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20120927/5b97a99a/attachment.txt>