[ome-users] Importing large image series and Cell^R data

[Roger Leigh]

On 13/05/2013 20:48, Colin Rickman wrote:
> Hi
>
> I'm just testing our new OMERO installation and have a few
> questions/problems with importing.

Which version of the OMERO client and server are you using?

> 1. I'm uploading a 740,000 frame tiff series with a total size just over
> 90GB. Using the importer it says uploading 0 of 1 so I assume it has
> decided to convert it to a single multidimensional ome.tif. After 15
> hours it failed with the statement file invalid. My questions are, is
> there a maximum upload size, is there a maximum ome.tif size, how can I
> choose to upload all files of a series as a series rather than convert
> them in to one file?

If the ome.tiff is a BigTIFF, then the maximum size will be the maximum
size of BigTIFF (2^64).  If it's not BigTIFF, then it will be the normal
2^32 (4 GiB).

Is your 740,000 frame tiff series a single file, or multiple files?  Is
it in any particular format which bioformats knows about ("showinf" will
tell you)?   If it's multiple files, are these structured in any
particular way (e.g. naming scheme) to give information about the
dimensionality?

Can you reproduce the failure with a smaller series?

We can certainly look at why the series fails to import, but we'll need
a copy of the original data for that, and 90GB is quite large.

> While this was uploading I attempted to convert the series to an ome.tif
> in FIJI and this failed at just over 4GB. FIJI was able to convert the
> series to a single .tif without any problems.

This is a known bug; we currently don't automatically enable BigTIFF
writing when the data is too big for normal TIFF, and we don't provide a
means to enable it by hand.

http://trac.openmicroscopy.org.uk/ome/ticket/6589

You can certainly convert using "bfconvert" and its "-bigtiff" option as
a workaround for the time being.

> 2. My other problem is uploading Olympus Cell^R/Cell Excellence data.
> Bioformats in FIJI seems to understand the database structure ok but in
> OMERO it uploads some of the data and names them after the database file
> names (e.g. xxxx.apl now contains image data) rather than the data names.

I'm afraid there's not much information to go on here.  We'll need some
more detail about exactly what you observed and what you expected.  If
you wish to provide us with an example, you can zip the data up and
upload it using http://qa.openmicroscopy.org.uk/qa/upload/ and we can
look at it in more detail.


Regards,
Roger

--
Dr Roger Leigh -- Open Microscopy Environment
Wellcome Trust Centre for Gene Regulation and Expression,
College of Life Sciences, University of Dundee, Dow Street,
Dundee DD1 5EH Scotland UK   Tel: (01382) 386364

The University of Dundee is a registered Scottish Charity, No: SC015096