[ome-users] [Software Feedback] Bioformat cpp withitk
Curtis Rueden
ctrueden at wisc.edu
Fri Jul 13 17:18:29 BST 2012
Hi Matthias,
> I tried the new loci_tools.jar you provided and I can say that the region
> selection is now working as expected.
Great, glad to hear it. The fix will propagate to the official build soon
(see https://github.com/openmicroscopy/bioformats/pull/104 for details if
interested).
Regards,
Curtis
On Fri, Jul 13, 2012 at 6:51 AM, Matthias Noll <
matthias.noll at igd.fraunhofer.de> wrote:
> Hi Curtis,
>
> I tried the new loci_tools.jar you provided and I can say that the region
> selection is now working as expected.
>
> Thank you for your fast replies.
> Regards,
> Matthias
>
> Am 7/12/2012 11:14 PM, schrieb Curtis Rueden:
>
>> Hi Matthias,
>>
>> Thanks for the additional information. The stack trace was helpful in
>> quickly finding the problem.
>>
>> It is a bug in ITKBridgePipes (the Java side of BF-ITK), which does not
>> call the method openBytes(no, x, y, w, h) but rather always
>> openBytes(no) and then crops after the fact, as you suspected.
>>
>> I have a pushed a proposed (untested) fix:
>> https://github.com/ctrueden/**bioformats/commit/**
>> eb83ace0f0a121c8c44e3fb8b95ec5**040f029c55<https://github.com/ctrueden/bioformats/commit/eb83ace0f0a121c8c44e3fb8b95ec5040f029c55>
>>
>> I have uploaded a build for you to test at:
>> http://dev.loci.wisc.edu/**curtis/2012-07-12-itk-crop/**loci_tools.jar<http://dev.loci.wisc.edu/curtis/2012-07-12-itk-crop/loci_tools.jar>
>>
>> Just replace your old loci_tools.jar and let us know how it goes! If all
>> is well, I will file a PR to merge the fix into the mainline.
>>
>> Regards,
>> Curtis
>>
>>
>> On Thu, Jul 12, 2012 at 3:52 PM, Matthias Noll
>> <matthias.noll at igd.fraunhofer.**de <matthias.noll at igd.fraunhofer.de>
>> <mailto:matthias.noll at igd.**fraunhofer.de<matthias.noll at igd.fraunhofer.de>>>
>> wrote:
>>
>> Hi Curtis,
>> I used the IORegion to set a subimage region for reading e.g.
>>
>> ImageType2D::SizeType smallSize;
>> smallSize.Fill(500);
>> ImageType2D::IndexType index;
>> index.Fill(0);
>> ImageType2D::RegionType region(index, smallSize);
>> try{
>> m_reader->GetOutput()->__**SetRequestedRegion(region);
>>
>> m_reader->Update();
>> }catch(itk::ExceptionObject &e){return;}
>>
>>
>>
>> The debug window shows that the region is set correctly. Also the
>> read command consists of filename and selected image region.
>> So I would say the itk side of the reader is well in order. At least
>> as far as I can tell right now.
>>
>> Debug: In
>> j:\itk4_1_bio\itk4.1.0\src\__**modules\io\imagebase\include\_**
>> _itkImageIOBase.h,
>>
>> line 161
>> BioFormatsImageIO (0000000003689390): setting IORegion to
>> ImageIORegion (0000000000B1F2D8)
>> Dimension: 2
>> Index: 0 0
>> Size: 500 500
>>
>> Debug: In
>> J:\ITK4_1_Bio\bioformats\src\_**_components\native\bf-itk-**
>> pipe\__itkBioFormatsImageIO.**cxx,
>>
>> line 647
>> BioFormatsImageIO (0000000003689390): BioFormatsImageIO::Read
>>
>> Debug: In
>> J:\ITK4_1_Bio\bioformats\src\_**_components\native\bf-itk-**
>> pipe\__itkBioFormatsImageIO.**cxx,
>>
>> line 668
>> BioFormatsImageIO (0000000003689390): BioFormatsImageIO::Read
>> command: read D:/CD45ROdk_c03H1665-8_n1_RS2 -
>> 2010-11-30-largest.ome.tif 0 500 0 500 0 1 0
>> 1 0 1
>>
>> Debug: In
>> J:\ITK4_1_Bio\bioformats\src\_**_components\native\bf-itk-**
>> pipe\__itkBioFormatsImageIO.**cxx,
>> line 284
>> BioFormatsImageIO (0000000003689390):
>> BioFormatsImageIO::__**DestroyJavaProcess killing java process
>>
>> Debug: In
>> J:\ITK4_1_Bio\bioformats\src\_**_components\native\bf-itk-**
>> pipe\__itkBioFormatsImageIO.**cxx,
>> line 289
>> BioFormatsImageIO (0000000003689390):
>> BioFormatsImageIO::__**DestroyJavaProcess destroying java process
>>
>>
>>
>> But looking at the following error message of the piped java
>> exception, the openBytes() function in FormatReader.java tries to
>> allocate an array of the dimension 69632 x 37888 x 3 x 1 whereas is
>> should only allocate the requested 500x500x3x1.
>>
>> itk::ERROR: BioFormatsImageIO(__**0000000003689390):
>>
>> BioFormatsImageIO: 'ITKBridgePipe read' exited abnormally.
>> Exception in thread "main" loci.formats.FormatException: Image
>> plane too large. Only 2GB of data can be extracted at one time.
>> You can workaround the problem by opening the plane in tiles;
>> for further details, see:
>> http://www.openmicroscopy.org/**__site/support/faq/bio-**
>> formats/__i-see-an-**outofmemory-or-__**negativearraysize-error-__**
>> message-when-attempting-to-__**open-an-svs-or-jpeg-2000-file.**
>> __-what-does-this-mean<http://www.openmicroscopy.org/__site/support/faq/bio-formats/__i-see-an-outofmemory-or-__negativearraysize-error-__message-when-attempting-to-__open-an-svs-or-jpeg-2000-file.__-what-does-this-mean>
>> <http://www.openmicroscopy.**org/site/support/faq/bio-**
>> formats/i-see-an-outofmemory-**or-negativearraysize-error-**
>> message-when-attempting-to-**open-an-svs-or-jpeg-2000-file.**
>> -what-does-this-mean<http://www.openmicroscopy.org/site/support/faq/bio-formats/i-see-an-outofmemory-or-negativearraysize-error-message-when-attempting-to-open-an-svs-or-jpeg-2000-file.-what-does-this-mean>
>> >
>> at
>> loci.formats.FormatReader.__**openBytes(FormatReader.java:__**
>> 760)
>> at
>> loci.formats.FormatReader.__**openBytes(FormatReader.java:__**
>> 739)
>> at
>> loci.formats.ImageReader.__**openBytes(ImageReader.java:__**393)
>> at
>> loci.formats.itk.__**ITKBridgePipes.read(__**
>> ITKBridgePipes.java:325)
>> at
>> loci.formats.itk.__**ITKBridgePipes.executeCommand(**
>> __ITKBridgePipes.java:133)
>> at
>> loci.formats.itk.__**ITKBridgePipes.waitForInput(__**
>> ITKBridgePipes.java:94)
>> at
>> loci.formats.itk.__**ITKBridgePipes.main(__**
>> ITKBridgePipes.java:554)
>> Caused by: java.lang.__**IllegalArgumentException: Array size too
>>
>> large: 69632 x 37888 x 3 x 1
>> at
>> loci.common.DataTools.__**safeMultiply32(DataTools.java:**__888)
>> at loci.common.DataTools.__**
>> allocate(DataTools.java:862)
>> at
>> loci.formats.FormatReader.__**openBytes(FormatReader.java:__**
>> 757)
>>
>> ... 6 more
>>
>>
>> I'm not sure where this problem arises, since I can't debug the java
>> part, at least I'm not sure how to. Maybe the java part of the
>> reader resets the image region with the actual image header
>> information, reads out everything and crops the result later, if the
>> reading worked.
>> I tested this with a smaller image of the dimensions 19000x8000 and
>> the same requested region. This worked but the read took a while.
>> And also I had to change the java vm memory from 265MB to 1024MB
>> which points also to a large amount of data allocation.
>>
>> Best regards,
>> Matthias
>>
>>
>> Am 12/07/2012 21:44, schrieb Curtis Rueden:
>>
>> Hi Matthias,
>>
>>
>> Now I encountered another problem. With normal sized images
>> everything
>> works fine. But since I pretty much only have images that
>> are larger than
>> 2GB, the Java machine has not enough memory to "load" the
>> whole image
>> plane. However this is not really what I want to do anyway.
>> It seems that
>> itk has a way to manage large files by opening them using
>> streams. An image
>> is split into pieces on read so that large images can be
>> processes
>> sequentially. I'm not sure but this functionality seems not
>> to be
>> implemented down to the wrapping bioformat reader for itk.
>>
>>
>> As you observed, Bio-Formats has the capability to read in only
>> a subset of
>> image data, including a cropped region of each XY plane. The
>> BF-ITK plugin
>> is supposed to provide this capability as well, using the
>> "IORegion"
>> feature of ITK ImageIO. That said, we have not really tested
>> this feature,
>> and there may be a problem with it. I am actually not even sure
>> how the
>> calling code should go about specifying the IORegion. But I
>> suggest you
>> give it a try (ask on ITK-users if you need assistance) and then
>> report
>> back if you find any problems with it.
>>
>> HTH,
>> Curtis
>>
>>
>> On Wed, Jul 11, 2012 at 3:54 AM, Matthias Noll <
>> matthias.noll at igd.fraunhofer._**_de
>>
>> <mailto:matthias.noll at igd.**fraunhofer.de<matthias.noll at igd.fraunhofer.de>>>
>> wrote:
>>
>> Hi Churtis,
>>
>> I managed to apply the reader after deregistration of other
>> readers that
>> were interfering in the reader selection.
>>
>> Now I encountered another problem. With normal sized images
>> everything
>> works fine. But since I pretty much only have images that
>> are larger than
>> 2GB, the Java machine has not enough memory to "load" the
>> whole image
>> plane. However this is not really what I want to do anyway.
>> It seems that
>> itk has a way to manage large files by opening them using
>> streams. An image
>> is split into pieces on read so that large images can be
>> processes
>> sequentially. I'm not sure but this functionality seems not
>> to be
>> implemented down to the wrapping bioformat reader for itk.
>>
>> itk::ERROR: BioFormatsImageIO(**__**0000000003815B60):
>>
>> BioFormatsImageIO:
>>
>> 'ITKBridgePipe read' exited abnormally. Exception in thread
>> "main"
>> loci.formats.FormatException: Image plane too large. Only
>> 2GB of data can
>> be extracted at one time.
>>
>> Looking at the error message, the Java wrapped code
>> obviously wants to
>> read in the whole image. The cropping seems to be done after
>> the complete
>> image is read.
>>
>> I also found this page using the error output.
>> http://www.openmicroscopy.org/**__**site/support/faq/bio-__**
>> formats/**<http://www.openmicroscopy.org/__**site/support/faq/bio-__formats/**>
>> <http://www.openmicroscopy.**org/**site/support/faq/bio-**
>> formats/**<http://www.openmicroscopy.org/**site/support/faq/bio-formats/**>
>> >
>> i-see-an-outofmemory-or-**__**negativearraysize-error-**
>> message-when-attempting-to-**_**_open-an-svs-or-jpeg-2000-**
>> file.__**
>> -what-does-this-mean<http://__**
>> www.openmicroscopy.org/site/__**support/faq/bio-formats/i-see-**
>> __an-outofmemory-or-__**negativearraysize-error-__**
>> message-when-attempting-to-__**open-an-svs-or-jpeg-2000-file.**
>> __-what-does-this-mean<http://www.openmicroscopy.org/site/__support/faq/bio-formats/i-see-__an-outofmemory-or-__negativearraysize-error-__message-when-attempting-to-__open-an-svs-or-jpeg-2000-file.__-what-does-this-mean>
>>
>> <http://www.openmicroscopy.**org/site/support/faq/bio-**
>> formats/i-see-an-outofmemory-**or-negativearraysize-error-**
>> message-when-attempting-to-**open-an-svs-or-jpeg-2000-file.**
>> -what-does-this-mean<http://www.openmicroscopy.org/site/support/faq/bio-formats/i-see-an-outofmemory-or-negativearraysize-error-message-when-attempting-to-open-an-svs-or-jpeg-2000-file.-what-does-this-mean>
>> >>
>>
>>
>> I could use the option to increase the JVM memory space but
>> this would
>> only help me so far. With really large images that exceed my
>> memory I would
>> still have a problem. Is there a way to get around the
>> complete read-in of
>> the input with the itk reader that I've misses so far?
>>
>> Best Regards,
>> Matthias
>>
>> Am 7/2/2012 7:25 PM, schrieb Curtis Rueden:
>>
>> Hi Matthias,
>>
>> I want to use the cpp wrapped bioformat reader
>> with itk to read
>> ome.tif files. The interfacing with itk however is
>> kind of not
>> working properly. First I have to unregister the
>> base readers (jpg,
>> png etc. )using the command
>> itk::ObjectFactoryBase::__**__**ReHash(); If
>>
>>
>> I
>>
>> don't do this, one of base readers will try to
>> load the tiff image.
>> Namely the TiffImageIO. Since the image in
>> question is tiled, this
>> read operation will fail. And therfore the
>> bioformat reader will
>> never be activated. If I use the ReHash()
>> function, the bioformat
>> reading works, but all other reader/writers are
>> unregistered in the
>> ObjectFactoryBase. In registering them manually
>> for e.g. png saving
>> a output region of the read ome.tiff image. The
>> bioformat reader
>> will always tell itk that he can write the input
>> image. But he sould
>> not, since I choose .png as the output format. I'm
>> not quite shure
>> what I'm doing wrong concerning the reader. I
>> thought the plugin
>> should behave as a normal itk reader/writer.
>>
>>
>> I am CCing the ITK-users list, since I think your
>> question is a more
>> general one about the ITK ImageIO plugin mechanism. It
>> seems to me that
>> all you really need is a simple way to reshuffle the
>> order of the
>> available ImageIO plugins—in this case, to give the
>> Bio-Formats ITK
>> plugin a higher priority than the built-in TIFF ImageIO
>> plugin. Does
>> anyone know a good way to do this?
>>
>> Thanks,
>> Curtis
>>
>> P.S. The ImageIO plugin Matthias is talking about is here:
>> http://loci.wisc.edu/bio-**__**formats/itk<http://loci.wisc.edu/bio-**__formats/itk>
>> <http://loci.wisc.edu/bio-****formats/itk<http://loci.wisc.edu/bio-**formats/itk>
>> ><http://loci.wisc.**__edu/bio-formats/itk
>>
>> <http://loci.wisc.edu/bio-**formats/itk<http://loci.wisc.edu/bio-formats/itk>
>> >>
>>
>>
>> On Mon, Jul 2, 2012 at 11:09 AM,
>> <matthias.noll at igd.fraunhofer.**__**de<matthias.noll at igd.
>> __fra**unhofer.de <http://fraunhofer.de>
>> <mailto:matthias.noll at igd.**fraunhofer.de<matthias.noll at igd.fraunhofer.de>
>> >>
>> <mailto:matthias.noll at igd.
>> <mailto:matthias.noll at igd.>**f**r__aunhofer.de<http://fr__aunhofer.de>
>> <http://fraunhofer.de><**matthias.noll at igd.__fraunhofer**
>> .de <http://fraunhofer.de>
>> <mailto:matthias.noll at igd.**fraunhofer.de<matthias.noll at igd.fraunhofer.de>
>> >>>>
>>
>>
>> wrote:
>>
>> Matthias Noll sent a message using the contact form
>> at
>> http://loci.wisc.edu/contact.
>>
>> Hello,
>> I want to use the cpp wrapped bioformat reader
>> with itk to read
>> ome.tif files. The interfacing with itk however is
>> kind of not
>> working properly. First I have to unregister the
>> base readers (jpg,
>> png etc. )using the command
>> itk::ObjectFactoryBase::__**__**ReHash(); If
>>
>>
>> I
>>
>> don't do this, one of base readers will try to
>> load the tiff image.
>> Namely the TiffImageIO. Since the image in
>> question is tiled, this
>> read operation will fail. And therfore the
>> bioformat reader will
>> never be activated. If I use the ReHash()
>> function, the bioformat
>> reading works, but all other reader/writers are
>> unregistered in the
>> ObjectFactoryBase. In registering them manually
>> for e.g. png saving
>> a output region of the read ome.tiff image. The
>> bioformat reader
>> will always tell itk that he can write the input
>> image. But he sould
>> not, since I choose .png as the output format. I'm
>> not quite shure
>> what I'm doing wrong concerning the reader. I
>> thought the plugin
>> should behave as a normal itk reader/writer.
>> I'm greatful for any kind of help.
>> Thanks Matthias
>>
>>
>>
>>
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20120713/3b7e1c64/attachment.html>
More information about the ome-users
mailing list