[ome-users] [Software Feedback] Bioformat cpp withitk
Matthias Noll
matthias.noll at igd.fraunhofer.de
Thu Jul 12 21:52:29 BST 2012
Hi Curtis,
I used the IORegion to set a subimage region for reading e.g.
> ImageType2D::SizeType smallSize;
> smallSize.Fill(500);
> ImageType2D::IndexType index;
> index.Fill(0);
> ImageType2D::RegionType region(index, smallSize);
> try{
> m_reader->GetOutput()->SetRequestedRegion(region);
> m_reader->Update();
> }catch(itk::ExceptionObject &e){return;}
The debug window shows that the region is set correctly. Also the read
command consists of filename and selected image region.
So I would say the itk side of the reader is well in order. At least as
far as I can tell right now.
> Debug: In
> j:\itk4_1_bio\itk4.1.0\src\modules\io\imagebase\include\itkImageIOBase.h,
> line 161
> BioFormatsImageIO (0000000003689390): setting IORegion to
> ImageIORegion (0000000000B1F2D8)
> Dimension: 2
> Index: 0 0
> Size: 500 500
>
> Debug: In
> J:\ITK4_1_Bio\bioformats\src\components\native\bf-itk-pipe\itkBioFormatsImageIO.cxx,
> line 647
> BioFormatsImageIO (0000000003689390): BioFormatsImageIO::Read
>
> Debug: In
> J:\ITK4_1_Bio\bioformats\src\components\native\bf-itk-pipe\itkBioFormatsImageIO.cxx,
> line 668
> BioFormatsImageIO (0000000003689390): BioFormatsImageIO::Read command:
> read D:/CD45ROdk_c03H1665-8_n1_RS2 - 2010-11-30-largest.ome.tif
> 0 500 0 500 0 1 0 1 0 1
>
> Debug: In
> J:\ITK4_1_Bio\bioformats\src\components\native\bf-itk-pipe\itkBioFormatsImageIO.cxx,
> line 284
> BioFormatsImageIO (0000000003689390):
> BioFormatsImageIO::DestroyJavaProcess killing java process
>
> Debug: In
> J:\ITK4_1_Bio\bioformats\src\components\native\bf-itk-pipe\itkBioFormatsImageIO.cxx,
> line 289
> BioFormatsImageIO (0000000003689390):
> BioFormatsImageIO::DestroyJavaProcess destroying java process
But looking at the following error message of the piped java exception,
the openBytes() function in FormatReader.java tries to allocate an array
of the dimension 69632 x 37888 x 3 x 1 whereas is should only allocate
the requested 500x500x3x1.
> itk::ERROR: BioFormatsImageIO(0000000003689390): BioFormatsImageIO:
> 'ITKBridgePipe read' exited abnormally. Exception in thread "main"
> loci.formats.FormatException: Image plane too large. Only 2GB of data
> can be extracted at one time. You can workaround the problem by
> opening the plane in tiles; for further details, see:
> http://www.openmicroscopy.org/site/support/faq/bio-formats/i-see-an-outofmemory-or-negativearraysize-error-message-when-attempting-to-open-an-svs-or-jpeg-2000-file.-what-does-this-mean
> at loci.formats.FormatReader.openBytes(FormatReader.java:760)
> at loci.formats.FormatReader.openBytes(FormatReader.java:739)
> at loci.formats.ImageReader.openBytes(ImageReader.java:393)
> at loci.formats.itk.ITKBridgePipes.read(ITKBridgePipes.java:325)
> at
> loci.formats.itk.ITKBridgePipes.executeCommand(ITKBridgePipes.java:133)
> at
> loci.formats.itk.ITKBridgePipes.waitForInput(ITKBridgePipes.java:94)
> at loci.formats.itk.ITKBridgePipes.main(ITKBridgePipes.java:554)
> Caused by: java.lang.IllegalArgumentException: Array size too large:
> 69632 x 37888 x 3 x 1
> at loci.common.DataTools.safeMultiply32(DataTools.java:888)
> at loci.common.DataTools.allocate(DataTools.java:862)
> at loci.formats.FormatReader.openBytes(FormatReader.java:757)
> ... 6 more
I'm not sure where this problem arises, since I can't debug the java
part, at least I'm not sure how to. Maybe the java part of the reader
resets the image region with the actual image header information, reads
out everything and crops the result later, if the reading worked.
I tested this with a smaller image of the dimensions 19000x8000 and the
same requested region. This worked but the read took a while. And also I
had to change the java vm memory from 265MB to 1024MB which points also
to a large amount of data allocation.
Best regards,
Matthias
Am 12/07/2012 21:44, schrieb Curtis Rueden:
> Hi Matthias,
>
>
>> Now I encountered another problem. With normal sized images everything
>> works fine. But since I pretty much only have images that are larger than
>> 2GB, the Java machine has not enough memory to "load" the whole image
>> plane. However this is not really what I want to do anyway. It seems that
>> itk has a way to manage large files by opening them using streams. An image
>> is split into pieces on read so that large images can be processes
>> sequentially. I'm not sure but this functionality seems not to be
>> implemented down to the wrapping bioformat reader for itk.
>
> As you observed, Bio-Formats has the capability to read in only a subset of
> image data, including a cropped region of each XY plane. The BF-ITK plugin
> is supposed to provide this capability as well, using the "IORegion"
> feature of ITK ImageIO. That said, we have not really tested this feature,
> and there may be a problem with it. I am actually not even sure how the
> calling code should go about specifying the IORegion. But I suggest you
> give it a try (ask on ITK-users if you need assistance) and then report
> back if you find any problems with it.
>
> HTH,
> Curtis
>
>
> On Wed, Jul 11, 2012 at 3:54 AM, Matthias Noll <
> matthias.noll at igd.fraunhofer.de> wrote:
>
>> Hi Churtis,
>>
>> I managed to apply the reader after deregistration of other readers that
>> were interfering in the reader selection.
>>
>> Now I encountered another problem. With normal sized images everything
>> works fine. But since I pretty much only have images that are larger than
>> 2GB, the Java machine has not enough memory to "load" the whole image
>> plane. However this is not really what I want to do anyway. It seems that
>> itk has a way to manage large files by opening them using streams. An image
>> is split into pieces on read so that large images can be processes
>> sequentially. I'm not sure but this functionality seems not to be
>> implemented down to the wrapping bioformat reader for itk.
>>
>> itk::ERROR: BioFormatsImageIO(**0000000003815B60): BioFormatsImageIO:
>> 'ITKBridgePipe read' exited abnormally. Exception in thread "main"
>> loci.formats.FormatException: Image plane too large. Only 2GB of data can
>> be extracted at one time.
>>
>> Looking at the error message, the Java wrapped code obviously wants to
>> read in the whole image. The cropping seems to be done after the complete
>> image is read.
>>
>> I also found this page using the error output.
>> http://www.openmicroscopy.org/**site/support/faq/bio-formats/**
>> i-see-an-outofmemory-or-**negativearraysize-error-**
>> message-when-attempting-to-**open-an-svs-or-jpeg-2000-file.**
>> -what-does-this-mean<http://www.openmicroscopy.org/site/support/faq/bio-formats/i-see-an-outofmemory-or-negativearraysize-error-message-when-attempting-to-open-an-svs-or-jpeg-2000-file.-what-does-this-mean>
>>
>> I could use the option to increase the JVM memory space but this would
>> only help me so far. With really large images that exceed my memory I would
>> still have a problem. Is there a way to get around the complete read-in of
>> the input with the itk reader that I've misses so far?
>>
>> Best Regards,
>> Matthias
>>
>> Am 7/2/2012 7:25 PM, schrieb Curtis Rueden:
>>
>>> Hi Matthias,
>>>
>>> I want to use the cpp wrapped bioformat reader with itk to read
>>> ome.tif files. The interfacing with itk however is kind of not
>>> working properly. First I have to unregister the base readers (jpg,
>>> png etc. )using the command itk::ObjectFactoryBase::__**ReHash(); If
>>> I
>>>
>>> don't do this, one of base readers will try to load the tiff image.
>>> Namely the TiffImageIO. Since the image in question is tiled, this
>>> read operation will fail. And therfore the bioformat reader will
>>> never be activated. If I use the ReHash() function, the bioformat
>>> reading works, but all other reader/writers are unregistered in the
>>> ObjectFactoryBase. In registering them manually for e.g. png saving
>>> a output region of the read ome.tiff image. The bioformat reader
>>> will always tell itk that he can write the input image. But he sould
>>> not, since I choose .png as the output format. I'm not quite shure
>>> what I'm doing wrong concerning the reader. I thought the plugin
>>> should behave as a normal itk reader/writer.
>>>
>>>
>>> I am CCing the ITK-users list, since I think your question is a more
>>> general one about the ITK ImageIO plugin mechanism. It seems to me that
>>> all you really need is a simple way to reshuffle the order of the
>>> available ImageIO plugins—in this case, to give the Bio-Formats ITK
>>> plugin a higher priority than the built-in TIFF ImageIO plugin. Does
>>> anyone know a good way to do this?
>>>
>>> Thanks,
>>> Curtis
>>>
>>> P.S. The ImageIO plugin Matthias is talking about is here:
>>> http://loci.wisc.edu/bio-**formats/itk<http://loci.wisc.edu/bio-formats/itk>
>>>
>>>
>>> On Mon, Jul 2, 2012 at 11:09 AM, <matthias.noll at igd.fraunhofer.**de<matthias.noll at igd.fraunhofer.de>
>>> <mailto:matthias.noll at igd.**fraunhofer.de<matthias.noll at igd.fraunhofer.de>>>
>>> wrote:
>>>
>>> Matthias Noll sent a message using the contact form at
>>> http://loci.wisc.edu/contact.
>>>
>>> Hello,
>>> I want to use the cpp wrapped bioformat reader with itk to read
>>> ome.tif files. The interfacing with itk however is kind of not
>>> working properly. First I have to unregister the base readers (jpg,
>>> png etc. )using the command itk::ObjectFactoryBase::__**ReHash(); If
>>> I
>>>
>>> don't do this, one of base readers will try to load the tiff image.
>>> Namely the TiffImageIO. Since the image in question is tiled, this
>>> read operation will fail. And therfore the bioformat reader will
>>> never be activated. If I use the ReHash() function, the bioformat
>>> reading works, but all other reader/writers are unregistered in the
>>> ObjectFactoryBase. In registering them manually for e.g. png saving
>>> a output region of the read ome.tiff image. The bioformat reader
>>> will always tell itk that he can write the input image. But he sould
>>> not, since I choose .png as the output format. I'm not quite shure
>>> what I'm doing wrong concerning the reader. I thought the plugin
>>> should behave as a normal itk reader/writer.
>>> I'm greatful for any kind of help.
>>> Thanks Matthias
>>>
>>>
>>>
More information about the ome-users
mailing list