[ome-users] Import of larger 2D OME-TIFF fails

[Matthias Baldauf]

Hello Chris,

thank you for your fast reply!


> Would it be possible to make these files available so that we might be able to fun some tests locally?

Of course, I uploaded the two OME-TIFFs for which the upload failed:
1) OME-TIFF, 16384 x 9600 pixels, three pages (one for each color-channel of the RGB-image), striped with 1 row per strip, 450 MB, and 8bit, uncompressed
Link: http://dl.dropbox.com/u/4868278/test_16384x9600_3pages.ome.tif
2) OME-TIFF, 16384 x 9600 pixels, one RGB-image, tiled (512 x 512), 456 MB, and 8bit, uncompressed
Link: http://dl.dropbox.com/u/4868278/test_16384x9600_singleRGBPage.ome.tif
The OME-TIFFs are filled with random data and were created with help of the Bio-Formats library.


>> Does the OMERO.server need much RAM for the construction of an image pyramid and does OMERO.server need much RAM after the import (e.g. when viewing large images with 60.000 x 40.000 pixels)? I would like to import a few thousand large 2D images and view them.
>It needs enough ram to read the smallest tile into memory and compress it. Obviously this is highly dependent on the source image.

Sorry for asking, but does your answer refers to the creation of the image pyramid? I would like to know more about how the import of large 2D images and tiling works, because it is important for my Master Thesis. As far as I know (please correct me if I am wrong), the image data gets split in each color-channel during the import process and the image data gets stored on the server in a binary file (one file for each color-channel)? Is the creation of the image pyramid done before the data gets stored on the server? Is the local import like that way: based on the source image a tile size is dynamically chosen, client reads tile and sends uncompressed tile to server, server creates six different resolution levels for this tile and stores it, client reads and sends next uncompressed tile, server creates six different resolution levels for this tile and stores it, etc. until finished? At the end there are many binary files on the server but these files are not in a tiled format, is that correct? In the server configuration the tile size can be defined but is this is only relevant for reading the images from the binary files and could be changed anytime? It would be very nice from you if you could provide me more insight in the import and tiling concept.

> The size criteria. By default this is (sizeX * sizeY) > (3192 * 3192)

I understand - this is the Big Image criterion. However, is it right that OME-TIFFs are always processed locally regardless of this criterion? My OME-TIFFs were all processed locally.


>> What are the three import steps? Is the step "1/3" the reading and splitting into the three color-channels
> No. It is just reading the metadata.
>> , step "2/3" the data-import on the server
> This is saving the metadata to the server.
>> and "3/3" the construction of the image pyramid?
> This is the process of copying data to the server directly in to the image pyramid.
>> Which steps are done locally and which steps are done on the server?
> All steps are done locally.

Thank you for the explanation. When I import the mentioned OME-TIFFs it takes about five minutes for reading and saving the metadata, which is quite long. It seems that OMERO.server is involved, because of the CPU usage during these two steps.


Thank you in advance for looking at the OME-TIFFs and for your answers!

Regards,
Matthias