That's essentially the exact same issue I had-- metadata was visible, but displayed an X when trying to look at the image...<br><br><br><br><br><div class="gmail_quote">On Wed, Sep 23, 2009 at 5:15 PM, Björn Quast <span dir="ltr"><<a href="mailto:bquast@evolution.uni-bonn.de">bquast@evolution.uni-bonn.de</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div style="font-family: 'DejaVu Sans'; font-size: 9pt; font-weight: 400; font-style: normal;">
Hi,<br>
<p style="margin: 0px; text-indent: 0px;"><br></p>I just tried it with a number of grayscale tif images, all about 30 Mb, one layer. When I use the importer-gui the import hangs after 1 image (with java -Xmx256m). After changing to -Xmx2000m the importer stops after three files, but several test show me that the number of loaded files do not depend on the memory given by the -Xmx flag. The importer.log is attached.<br>
The "KeepAlive ping" looks, as if the importer looses the connection, but the client was started on the same machine with the server.<br>
<p style="margin: 0px; text-indent: 0px;"><br></p>Sometimes error messages of the Ice.Marshal occur: <br>
<p style="margin: 0px; text-indent: 0px;"><br></p>Ice.MarshalException<br>
reason = "OutOfMemoryError occurred while allocating a ByteBuffer"<br>
at IceInternal.Buffer.reserve(Buffer.java:163)<br>
at IceInternal.Buffer.resize(Buffer.java:72)<br>
at IceInternal.Buffer.expand(Buffer.java:59)<br>
at IceInternal.BasicStream.expand(BasicStream.java:2031)<br>
at IceInternal.BasicStream.writeInt(BasicStream.java:872)<br>
at omero.api._RawPixelsStoreDelM.setPlane(_RawPixelsStoreDelM.java:1128)<br>
at omero.api.RawPixelsStorePrxHelper.setPlane(RawPixelsStorePrxHelper.java:1546)<br>
at omero.api.RawPixelsStorePrxHelper.setPlane(RawPixelsStorePrxHelper.java:1518)<br>
at ome.formats.OMEROMetadataStoreClient.setPlane(OMEROMetadataStoreClient.java:2469)<br>
at ome.formats.importer.ImportLibrary.importData(ImportLibrary.java:450)<br>
at ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:354)<br>
at ome.formats.importer.ImportHandler.importImages(ImportHandler.java:192)<br>
at ome.formats.importer.ImportHandler.access$0(ImportHandler.java:110)<br>
at ome.formats.importer.ImportHandler$1.run(ImportHandler.java:91)<br>
Caused by: java.lang.OutOfMemoryError: Java heap space<br>
at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)<br>
at java.nio.ByteBuffer.allocate(ByteBuffer.java:329)<br>
at IceInternal.Buffer.reserve(Buffer.java:141)<br>
... 13 more<br>
<p style="margin: 0px; text-indent: 0px;"><br></p><p style="margin: 0px; text-indent: 0px;"><br></p><p style="margin: 0px; text-indent: 0px;"><br></p><p style="margin: 0px; text-indent: 0px;"><br></p>After connecting to the database with OMEROinsight_unix.sh and trying to open one of the loaded images another error message occurs: (again attached). Most of the large single layer images are shown by an X as thumbnail, but their metadata are viewable. Images, which are shown as thumbnails can also be shown in detail.<br>
<p style="margin: 0px; text-indent: 0px;"><br></p><p style="margin: 0px; text-indent: 0px;"><br></p>I hope this help to find the problem.<br><font color="#888888">
<p style="margin: 0px; text-indent: 0px;"><br></p>Björn</font><div><div></div><div class="h5"><br>
<p style="margin: 0px; text-indent: 0px;"><br></p><p style="margin: 0px; text-indent: 0px;"><br></p><p style="margin: 0px; text-indent: 0px;"><br></p>> On Wed, Sep 23, 2009 at 11:25 AM, Curtis Rueden <<a href="mailto:ctrueden@wisc.edu" target="_blank">ctrueden@wisc.edu</a>> wrote:<br>
> > Hi Björn & David,<br>
> ><br>
> > Thanks for the feedback, and sorry for your trouble importing large image<br>
> > planes. As of Beta 4.0.3, the OMERO.importer defaults to a maximum heap<br>
> > size of 1024 MB. So any image planes of 1 GB or more are certainly too<br>
> > large to process successfully, and even image planes over 512 MB may have<br>
> > problems due to temporary overhead or garbage collection issues.<br>
> ><br>
> > One workaround is to increase the maximum heap size. With a 32-bit JVM,<br>
> > this limit can often be 1536 or even ~1800 GB depending on your operating<br>
> > system. The process for increasing the heap size is platform-dependent,<br>
> > however. For example, on Mac OS X, right click the OMERO.importer icon,<br>
> > choose "Show package contents," navigate into Contents, and edit the<br>
> > Info.plist file's line "-Xmx1024M" to read "-Xmx1700M" or whatever value<br>
> > you wish to try.<br>
> ><br>
> > That said, we have plans to improve the support for large image planes by<br>
> > processing them in tiles, negating the need for such a large amount of<br>
> > RAM at once. Unfortunately, this support is not planned until at least<br>
> > Beta 4.2. The Bio-Formats readers already support processing an image<br>
> > plane in tiles, but the writers do not, so the bfconvert command line<br>
> > tool does not yet support this either.<br>
> ><br>
> > -Curtis<br>
> ><br>
> ><br>
> > On Wed, Sep 23, 2009 at 3:28 AM, Björn Quast<br>
> > <<a href="mailto:bquast@evolution.uni-bonn.de" target="_blank">bquast@evolution.uni-bonn.de</a><br>
> ><br>
> > > wrote:<br>
> >><br>
> >> Hi David,<br>
> >><br>
> >> I have had the same importing problems with one layer tif images, when<br>
> >> they<br>
> >> exceed a specific image size . There is no problem to import very large<br>
> >> multi-<br>
> >> layer tifs (about 500 Mb), but if one layer is to large the import<br>
> >> stopped and<br>
> >> I got a message on the console, that the java heap memory was exceeded.<br>
> >> I can only guess, that the importer fails to read in large strips of<br>
> >> image data. tif files may store all image data in one chunk or line by<br>
> >> line or in<br>
> >> smaller pieces. But even with line by line stored images I got the<br>
> >> problems,<br>
> >> if one layer is to large.<br>
> >><br>
> >> It also happens, that a few images were imported but than the importer<br>
> >> stops.<br>
> >> Again, I can only guess that there is a memory issue.<br>
> >><br>
> >> In the moment I can not reproduce it, because I have no omero running,<br>
> >> but perhaps I can give more details when my server is up again.<br>
> >><br>
> >> Björn<br>
> >><br>
> >> David Gutman wrote:<br>
> >> > I am having a lot of problems trying to import images into OMERO. I<br>
> >> > am using the 4.0.3beta code and having issues trying to import any of<br>
> >> > my images.....<br>
> >> ><br>
> >> > The images are initially acquired as Aperio SVS files ....they are<br>
> >> > basically a TIFF container with JPEG2000 tiles embedded in them.<br>
> >> > I've tried the following scenarios and still can't get all of my<br>
> >> > images to consistently import, and get all sorts of errors, so wanted<br>
> >> > to see if the community had any thoughts...<br>
> >> ><br>
> >> > 1) Split 6 channel SVS file into individual channels using<br>
> >> > tiffsplit--- these images are just standard tiff's, and I can even<br>
> >> > view them with imagemagick, and other graphics viewers (At least the<br>
> >> > small ones)<br>
> >> > This produces 6 images, the really small images seem to import okay<br>
> >> > sometimes, but not always (like 1024x512 or whatever-- these generally<br>
> >> > are just an image of the label on the slide)...<br>
> >> ><br>
> >> > So some of the frames, but of course not the large frames (22090x27696<br>
> >> > pixels) will not load either through the GUI or through the command<br>
> >> > line interpreter--- I've restricted this now where I don't even try to<br>
> >> > upload images greater than 250 MegaPixels (so well under 1Gigapixel<br>
> >> > even considering all three channels) ......but I've had issues with<br>
> >> > even much smaller images....<br>
> >> ><br>
> >> > Has anyone had any similar issues trying to load in large files with<br>
> >> > the current beta of OMERO I've even tried to transcode some of these<br>
> >> > using the bioformats tools to OME-XML.TIFF files with similar<br>
> >> > disappointing results<br>
> >> ><br>
> >> > Oddly-- some of my images upload OK, and some do not...<br>
> >> ><br>
> >> > 2966455 2009-09-16 13:57 TCGA-06-0214-01B-03-BS3aad.tif<br>
> >> > 158051 2009-09-16 13:57 TCGA-06-0214-01B-03-BS3aab.tif<br>
> >> ><br>
> >> > The "b" image loaded fine, the "d" image did not-- it's not a terribly<br>
> >> > big image, but I keep getting weird errors when I try and import it--<br>
> >> > has anyone had any issues lately with the OMEROIMPORTER function/gui?<br>
> >><br>
> >> --<br>
> >> Dr. Björn Quast<br>
> >> Universität Bonn<br>
> >> Institut für Evolutionsbiologie und Ökologie<br>
> >> 53121 Bonn<br>
> >> Tel.: 0228/735758 email: <a href="mailto:bquast@evolution.uni-bonn.de" target="_blank">bquast@evolution.uni-bonn.de</a><br>
> >> _______________________________________________<br>
> >> ome-users mailing list<br>
> >> <a href="mailto:ome-users@lists.openmicroscopy.org.uk" target="_blank">ome-users@lists.openmicroscopy.org.uk</a><br>
> >> <a href="http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users" target="_blank">http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users</a><br>
<p style="margin: 0px; text-indent: 0px;"><br></p><p style="margin: 0px; text-indent: 0px;"><br></p>-- <br>
Dr. Björn Quast<br>
Universität Bonn<br>
Institut für Evolutionsbiologie und Ökologie <br>
53121 Bonn<br>
Tel.: 0228/735758 email: <a href="mailto:bquast@evolution.uni-bonn.de" target="_blank">bquast@evolution.uni-bonn.de</a></div></div></div>
<br>_______________________________________________<br>
ome-users mailing list<br>
<a href="mailto:ome-users@lists.openmicroscopy.org.uk">ome-users@lists.openmicroscopy.org.uk</a><br>
<a href="http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users" target="_blank">http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users</a><br>
<br></blockquote></div><br><br clear="all"><br>-- <br>David A Gutman, M.D. Ph.D.<br>Center for Comprehensive Informatics<br>Emory University School of Medicine<br>