[ome-users] OMERO Import issues...

[Curtis Rueden]

Hi Björn & David,

Thanks for the feedback, and sorry for your trouble importing large image
planes. As of Beta 4.0.3, the OMERO.importer defaults to a maximum heap size
of 1024 MB. So any image planes of 1 GB or more are certainly too large to
process successfully, and even image planes over 512 MB may have problems
due to temporary overhead or garbage collection issues.

One workaround is to increase the maximum heap size. With a 32-bit JVM, this
limit can often be 1536 or even ~1800 GB depending on your operating system.
The process for increasing the heap size is platform-dependent, however. For
example, on Mac OS X, right click the OMERO.importer icon, choose "Show
package contents," navigate into Contents, and edit the Info.plist file's
line "-Xmx1024M" to read "-Xmx1700M" or whatever value you wish to try.

That said, we have plans to improve the support for large image planes by
processing them in tiles, negating the need for such a large amount of RAM
at once. Unfortunately, this support is not planned until at least Beta 4.2.
The Bio-Formats readers already support processing an image plane in tiles,
but the writers do not, so the bfconvert command line tool does not yet
support this either.

-Curtis

On Wed, Sep 23, 2009 at 3:28 AM, Björn Quast
<bquast at evolution.uni-bonn.de>wrote:

> Hi David,
>
> I have had the same importing problems with one layer tif images, when they
> exceed a specific image size . There is no problem to import very large
> multi-
> layer tifs (about 500 Mb), but if one layer is to large the import stopped
> and
> I got a message on the console, that the java heap memory was exceeded.
> I can only guess, that the importer fails to read in large strips of image
> data. tif files may store all image data in one chunk or line by line or in
> smaller pieces. But even with line by line stored images I got the
> problems,
> if one layer is to large.
>
> It also happens, that a few images were imported but than the importer
> stops.
> Again, I can only guess that there is a memory issue.
>
> In the moment I can not reproduce it, because I have no omero running, but
> perhaps I can give more details when my server is up again.
>
> Björn
>
> David Gutman wrote:
> > I am having a lot of problems trying to import images into OMERO.    I
> > am using the 4.0.3beta code and having issues trying to import any of
> > my images.....
> >
> > The images are initially acquired as Aperio SVS files ....they are
> > basically a TIFF container with JPEG2000 tiles embedded in them.
> > I've tried the following scenarios and still can't get all of my
> > images to consistently  import, and get all sorts of errors, so wanted
> > to see if the community had any thoughts...
> >
> > 1) Split 6 channel SVS file into individual channels using
> > tiffsplit--- these images are just standard tiff's, and I can even
> > view them with imagemagick, and other graphics viewers (At least the
> > small ones)
> >    This produces 6 images, the really small images seem to import okay
> > sometimes, but not always (like 1024x512 or whatever-- these generally
> > are just an image of the label on the slide)...
> >
> > So some of the frames, but of course not the large frames (22090x27696
> > pixels) will not load either through the GUI or through the command
> > line interpreter--- I've restricted this now where I don't even try to
> > upload images greater than 250 MegaPixels (so well under 1Gigapixel
> > even considering all three channels) ......but I've had issues with
> > even much smaller images....
> >
> > Has anyone had any similar issues trying to load in large files with
> > the current beta of OMERO  I've even tried to transcode some of these
> > using the bioformats tools to  OME-XML.TIFF files with similar
> > disappointing results
> >
> > Oddly-- some of my images upload OK, and some do not...
> >
> > 2966455 2009-09-16 13:57 TCGA-06-0214-01B-03-BS3aad.tif
> > 158051 2009-09-16 13:57 TCGA-06-0214-01B-03-BS3aab.tif
> >
> > The "b" image loaded fine, the "d" image did not-- it's not a terribly
> > big image, but I keep getting weird errors when I try and import it--
> > has anyone had any issues lately with the OMEROIMPORTER function/gui?
>
> --
> Dr. Björn Quast
> Universität Bonn
> Institut für Evolutionsbiologie und Ökologie
> 53121 Bonn
> Tel.: 0228/735758   email: bquast at evolution.uni-bonn.de
> _______________________________________________
> ome-users mailing list
> ome-users at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20090923/01b1c592/attachment.html>