[ome-users] bioformats and ITK

Kishore Mosaliganti kishoreraom at gmail.com
Fri Oct 19 19:14:14 BST 2012 

Hi Mark,

Thats great! I will try the test at my end and confirm the result.

Thank you,

Kishore

On Fri, Oct 19, 2012 at 12:41 PM, Mark Hiner <hiner at wisc.edu> wrote:

> Hi Kishore,
>
>  I updated your test<https://gist.github.com/3900888/81064cf96fb06b4575a64e81481fda0936cd9056>to use an ExtractImageFilter in the IO pipeline. When a Reader's output is
> piped to this Filter, the reader->Update() call only reads the extracted
> region.
>
>  When I tested on the
> ConfocalMemCitrine516LeftEar18hpfAug17_2012_Fish1.lsm test image you
> uploaded, it correctly splits out the stacks for each time point. And when
> I tested on a 9GB image it processed each time point without reading the
> entire file.
>
>  So I'm hoping this solves your issue. If not, just let me know how it's
> failing for you and I'll try again!
>
> Thanks,
> Mark
>
>
> On Wed, Oct 17, 2012 at 2:34 PM, Mark Hiner <hiner at wisc.edu> wrote:
>
>> One thought is that, as the itkBioFormatsImageIO is a
>> StreamingImageIOBase<http://www.itk.org/Doxygen320/html/classitk_1_1StreamingImageIOBase.html>,
>> you could just set the desired region (as you were) and use *its* Read
>> and Write methods instead of wrapping it in a Reader or a Writer.
>>
>> I'll try to put a code example up using this approach. It would be a good
>> test case for the bf-itk-pipe plug-in anyway.
>>
>> - Mark
>>
>>
>>
>> On Wed, Oct 17, 2012 at 1:56 PM, Mark Hiner <hinerm at gmail.com> wrote:
>>
>>> Hi Kishore,
>>>
>>>  Just wanted to let you know that I'm still looking at your issue today.
>>> I updated the gist<https://gist.github.com/3900888/7aab8fddb38af87ea98cd49dd921524ace6430c8>to reflect the topics discussed in this e-mail, using the region setting
>>> method calls instead of pipelining, and added some (hopefully) helpful
>>> print statements to see what's going on with the regions.
>>>
>>>  From what I can tell there are two separate issues preventing your code
>>> from writing the desired regions.
>>>
>>>  First of all, the IORegion needs to be set on the Writer, for example:
>>>
>>>
>>>     writer->SetImageIO(io);
>>>     writer->SetInput(reader->GetOutput());
>>>
>>>
>>>     writer->SetIORegion(region);
>>>     writer->Write();
>>>
>>>  In the original code you sent to me, that wasn't happening and it was
>>> defaulting to the first 3D region and thus the first time point. I
>>> mentioned this, without providing a code example, in a past e-mail - so
>>> apologies if you've already updated your code!
>>>
>>>  Note that according to the itkImageFileWriter API<http://www.itk.org/Doxygen/html/classitk_1_1ImageFileWriter.html#ab2cba65f716aa75ec962171ee0b46fe3>Write() is a special form of Update() that respects the IORegion. However,
>>> in practice, for me it still just writes the largest possible region
>>> instead of what was specified. So that could suggest a bug in the
>>> bf-itk-pipes plug-in.
>>>
>>>  The second issue is that even after manually specifying the IO regions,
>>> the Reader output still has the largest possible dimensions (T=2) so the
>>> current gist test ends up creating 2 full copies of the image. I think this
>>> was always happening at the Reader, but then it was truncated to the first
>>> time point you saw by the Writer. Anyway, this implies that the full image
>>> is still being read, despite the region specifications.
>>>
>>>  I think this is a side effect of setting using the
>>> LargestPossibleRegion in the Update() method, as mentioned in the itkProcessObject
>>> specification<http://www.itk.org/Doxygen/html/classitk_1_1ProcessObject.html#a4041fb21e9105500eee311e265691bd5>
>>> .
>>>
>>>  Right now I'm going through the examples from this ITK Users thread<http://itk-insight-users.2283740.n2.nabble.com/Memory-limits-of-ITKImage-object-td5670422.html>about streaming for a better pipelining solution. I'm also testing to see
>>> what can be done at the ImageIO level, as when Read and Write are hit in
>>> the itkBioFormatsImageIO, GetIORegion() is already returning the largest
>>> possible region, instead of the specified region.
>>>
>>>  I'll keep you updated.
>>>
>>> - Mark
>>>
>>>
>>> On Tue, Oct 16, 2012 at 1:37 PM, Kishore Mosaliganti <
>>> kishoreraom at gmail.com> wrote:
>>>
>>>>  Dear Mark,
>>>>
>>>> Thanks for the clarification. My original problem remains unresolved.
>>>> For example, in the LSM gist on lines 104-107:
>>>>
>>>>     ReaderType::Pointer reader = ReaderType::New();
>>>>
>>>>
>>>>
>>>>
>>>>     reader->SetFileName(argv[1]);
>>>>     reader->SetImageIO(io);
>>>>
>>>>     reader->Update();
>>>>
>>>>
>>>> Here, calling reader->Update() loads the entire image into memory. If
>>>> you have a very large LSM file (several gigabytes large), then this code
>>>> will fail since such an image cannot get into the memory.
>>>>
>>>> Hence, I am trying to send the read region directly into the reader
>>>> using
>>>>
>>>> io->SetRegion(region);
>>>>
>>>> and then
>>>>
>>>> reader->SetImageIO(io);
>>>> reader->Update();
>>>>
>>>>
>>>> But this step is not giving me two different timepoints. It is writing
>>>> out the same timepoint.
>>>>
>>>> Kishore
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Oct 16, 2012 at 2:13 PM, Mark Hiner <hiner at wisc.edu> wrote:
>>>>
>>>>> Hello again,
>>>>>
>>>>>  I created a gist for your LSM test and updated it with my most recent
>>>>> code:
>>>>> https://gist.github.com/3900888
>>>>>
>>>>>  That example is currently working for me, in that it writes out 2
>>>>> images of 72 planes - one for T=0 and one for T=1. Since everything in ITK
>>>>> seems to be pipeline-oriented, I am not sure there is a solution to
>>>>> splitting out channels just using Setters like we were both trying.
>>>>>
>>>>>  Instead, I followed this ROI Filter example<http://www.vtk.org/Wiki/ITK/Examples/ImageProcessing/RegionOfInterestImageFilter>and used the logic from my last e-mail.
>>>>>
>>>>>  I do need to correct one thing I said in my last e-mail: SizeT should
>>>>> always be 1, as it is a relative offset to the Index value.
>>>>>
>>>>>  When I ran that test, both images went to grayscale.. I'm going to
>>>>> see if that's an error in the bf-itk-pipes logic, or just something that
>>>>> needs to be set up differently in using the ITK API.
>>>>>
>>>>>  Let me know if you run into any more problems with this code, or if
>>>>> you just want to discuss any part of it.
>>>>>
>>>>> Thanks again for using our software :)
>>>>> - Mark
>>>>>
>>>>>
>>>>> On Tue, Oct 16, 2012 at 11:00 AM, Mark Hiner <hiner at wisc.edu> wrote:
>>>>>
>>>>>> Hi Kishore,
>>>>>>
>>>>>>  I am sorry I wasn't able to get to your issue yesterday. Looking at
>>>>>> it today, I think there are a couple of things going on here.
>>>>>>
>>>>>>  First of all: in your test class
>>>>>> - I consolidated a BioFormatsImageIO and the ImageIOBase.
>>>>>> - In the region updating loop, the wrong region (dim-3 instead of
>>>>>> dim-2) was being set, and only the index is set. I suspect both the index
>>>>>> and size need to be set.
>>>>>>
>>>>>>  I attached an updated version of your test, where in the first
>>>>>> iteration it sets IndexT = 0, sizeT = 1. The second iteration sets IndexT =
>>>>>> 1, sizeT = 2.  I believe those regions will split the time points as you
>>>>>> wanted.
>>>>>>
>>>>>> But there is a second issue: although the Reader IORegion is correct
>>>>>> (I think) the Writer is not. It is defaulting to the ITK base 3D region.
>>>>>> When I tried setting the Writer's region to our desired region, it throws
>>>>>> an exception because the 5D region can't be contained by what it thinks is
>>>>>> the largest possible (3D) region. I am refreshing myself on the ITK API and
>>>>>> once we figure out how to set that region, I think it will write the images
>>>>>> correctly (or we can move on to the next problem).
>>>>>>
>>>>>> In your test I changed the output format to ome.tiff because I know
>>>>>> that will use the itkBioFormatsImageIO writer code, which I know is capable
>>>>>> of handling 5D writing.
>>>>>>
>>>>>> Also, in testing I found a couple of bugs in the ITKPipesBridge code.
>>>>>> I started a new branch to fix these and any other issues that may be
>>>>>> discovered. If you want to use the itkBioFormatsImageIO writer code you'll
>>>>>> have to use this new branch for now. You can find it here:
>>>>>> https://github.com/hinerm/bioformats/commits/bf-itk-fixes
>>>>>>
>>>>>> As always, let me know if this isn't fully addressing your issue.
>>>>>> Otherwise, I'll let you know as soon as I have the Writer region setting
>>>>>> properly.
>>>>>>
>>>>>> Thanks!
>>>>>> - Mark
>>>>>>
>>>>>>
>>>>>> On Fri, Oct 12, 2012 at 11:28 AM, Kishore Mosaliganti <
>>>>>> kishoreraom at gmail.com> wrote:
>>>>>>
>>>>>>> Hi Mark,
>>>>>>>
>>>>>>> Thank you for the email and invaluable help. The bioformats-itk
>>>>>>> plugin is wonderful. I am attaching my script here to help you debug. Let
>>>>>>> me know if you have any questions on my script. I am starting with a 5D
>>>>>>> (XYZTC) image and trying to write out all the 3D timepoints for the first
>>>>>>> channel.
>>>>>>>
>>>>>>> Kishore
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Oct 12, 2012 at 12:19 PM, Mark Hiner <hinerm at gmail.com>wrote:
>>>>>>>
>>>>>>>> Hi Kishore,
>>>>>>>>
>>>>>>>>  I helped develop the bf-itk-pipe plugin and am investigating your
>>>>>>>> issue right now.
>>>>>>>>
>>>>>>>>  What you're doing seems reasonable, so I'm trying to determine if
>>>>>>>> there's a bug in the plugin. My goal is to have some working example code
>>>>>>>> for you (and a fix if necessary) by the end of the day.
>>>>>>>>
>>>>>>>> Thank you for using our plugin! I hope we can get it working for
>>>>>>>> you soon.
>>>>>>>>
>>>>>>>> - Mark
>>>>>>>>
>>>>>>>> On Fri, Oct 12, 2012 at 10:54 AM, Kishore Mosaliganti <
>>>>>>>> kishoreraom at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Dear ITK and OME users,
>>>>>>>>>
>>>>>>>>> To further elaborate, I figured that this has nothing to do with
>>>>>>>>> the itkStreamingImageFilter.
>>>>>>>>>
>>>>>>>>> Instead, the itkImageFileReader seems to be extracting the same
>>>>>>>>> image region although I update the requested region in the
>>>>>>>>> BioformatsImageIO class :;
>>>>>>>>>
>>>>>>>>>     itk::BioFormatsImageIO::Pointer io =
>>>>>>>>> itk::BioFormatsImageIO::New();
>>>>>>>>>     io->SetIORegion( region ); // doesn't make a difference in
>>>>>>>>> terms of image data extracted. Its always from the beginning.
>>>>>>>>>
>>>>>>>>>     ReaderType::Pointer reader = ReaderType::New();
>>>>>>>>>     reader->SetFileName(argv[1]);
>>>>>>>>>     reader->SetImageIO(io);
>>>>>>>>>     reader->Update();
>>>>>>>>>
>>>>>>>>> So, my general question is whether the BioformatsImageIO expects
>>>>>>>>> the whole LSM image to be loaded into memory before writing it out? I would
>>>>>>>>> like to specify small image regions since my LSM is too large to be fully
>>>>>>>>> loaded into memory. How else can I stream data from large LSM?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Kishore
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Oct 12, 2012 at 8:53 AM, Kishore Mosaliganti <
>>>>>>>>> kishoreraom at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi all,
>>>>>>>>>>
>>>>>>>>>> I am trying to use bioformats and itk. I downloaded and compiled
>>>>>>>>>> bf-itk-pipe using cmake by linking against ITK 3.2:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> https://github.com/openmicroscopy/bioformats/tree/develop/components/native/bf-itk-pipe
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I tested out itkBFImageInfo and ./itkRGBBioFormatsImageIOTest on
>>>>>>>>>> a few simple LSM images. It works great and writes out all the associated
>>>>>>>>>> metadata and pixel data for the first timepoint.
>>>>>>>>>>
>>>>>>>>>> My LSM microscopy image is 5D. It is X-Y-Z-Time-Channel. I am
>>>>>>>>>> interested in extracting individual timepoints from the LSM file for
>>>>>>>>>> processing. The filter seems to be using itkStreamingImageFilter. I set the
>>>>>>>>>> NumberOfStreams to 10 but that still seems to still write out only the
>>>>>>>>>> first image.
>>>>>>>>>>
>>>>>>>>>> How do I set the output requestion region for an individual
>>>>>>>>>> timepoint?
>>>>>>>>>>
>>>>>>>>>> Thank you,
>>>>>>>>>>
>>>>>>>>>> Kishore
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> ome-users mailing list
>>>>>>>>> ome-users at lists.openmicroscopy.org.uk
>>>>>>>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20121019/77411dfe/attachment.html>

More information about the ome-users mailing list