[ome-users] bioformats and ITK
Mark Hiner
hiner at wisc.edu
Fri Oct 19 17:41:24 BST 2012
Hi Kishore,
I updated your
test<https://gist.github.com/3900888/81064cf96fb06b4575a64e81481fda0936cd9056>to
use an ExtractImageFilter in the IO pipeline. When a Reader's output
is
piped to this Filter, the reader->Update() call only reads the extracted
region.
When I tested on the ConfocalMemCitrine516LeftEar18hpfAug17_2012_Fish1.lsm
test image you uploaded, it correctly splits out the stacks for each time
point. And when I tested on a 9GB image it processed each time point
without reading the entire file.
So I'm hoping this solves your issue. If not, just let me know how it's
failing for you and I'll try again!
Thanks,
Mark
On Wed, Oct 17, 2012 at 2:34 PM, Mark Hiner <hiner at wisc.edu> wrote:
> One thought is that, as the itkBioFormatsImageIO is a StreamingImageIOBase<http://www.itk.org/Doxygen320/html/classitk_1_1StreamingImageIOBase.html>,
> you could just set the desired region (as you were) and use *its* Read
> and Write methods instead of wrapping it in a Reader or a Writer.
>
> I'll try to put a code example up using this approach. It would be a good
> test case for the bf-itk-pipe plug-in anyway.
>
> - Mark
>
>
>
> On Wed, Oct 17, 2012 at 1:56 PM, Mark Hiner <hinerm at gmail.com> wrote:
>
>> Hi Kishore,
>>
>> Just wanted to let you know that I'm still looking at your issue today.
>> I updated the gist<https://gist.github.com/3900888/7aab8fddb38af87ea98cd49dd921524ace6430c8>to reflect the topics discussed in this e-mail, using the region setting
>> method calls instead of pipelining, and added some (hopefully) helpful
>> print statements to see what's going on with the regions.
>>
>> From what I can tell there are two separate issues preventing your code
>> from writing the desired regions.
>>
>> First of all, the IORegion needs to be set on the Writer, for example:
>>
>>
>> writer->SetImageIO(io);
>> writer->SetInput(reader->GetOutput());
>>
>> writer->SetIORegion(region);
>> writer->Write();
>>
>> In the original code you sent to me, that wasn't happening and it was
>> defaulting to the first 3D region and thus the first time point. I
>> mentioned this, without providing a code example, in a past e-mail - so
>> apologies if you've already updated your code!
>>
>> Note that according to the itkImageFileWriter API<http://www.itk.org/Doxygen/html/classitk_1_1ImageFileWriter.html#ab2cba65f716aa75ec962171ee0b46fe3>Write() is a special form of Update() that respects the IORegion. However,
>> in practice, for me it still just writes the largest possible region
>> instead of what was specified. So that could suggest a bug in the
>> bf-itk-pipes plug-in.
>>
>> The second issue is that even after manually specifying the IO regions,
>> the Reader output still has the largest possible dimensions (T=2) so the
>> current gist test ends up creating 2 full copies of the image. I think this
>> was always happening at the Reader, but then it was truncated to the first
>> time point you saw by the Writer. Anyway, this implies that the full image
>> is still being read, despite the region specifications.
>>
>> I think this is a side effect of setting using the LargestPossibleRegion
>> in the Update() method, as mentioned in the itkProcessObject
>> specification<http://www.itk.org/Doxygen/html/classitk_1_1ProcessObject.html#a4041fb21e9105500eee311e265691bd5>
>> .
>>
>> Right now I'm going through the examples from this ITK Users thread<http://itk-insight-users.2283740.n2.nabble.com/Memory-limits-of-ITKImage-object-td5670422.html>about streaming for a better pipelining solution. I'm also testing to see
>> what can be done at the ImageIO level, as when Read and Write are hit in
>> the itkBioFormatsImageIO, GetIORegion() is already returning the largest
>> possible region, instead of the specified region.
>>
>> I'll keep you updated.
>>
>> - Mark
>>
>>
>> On Tue, Oct 16, 2012 at 1:37 PM, Kishore Mosaliganti <
>> kishoreraom at gmail.com> wrote:
>>
>>> Dear Mark,
>>>
>>> Thanks for the clarification. My original problem remains unresolved.
>>> For example, in the LSM gist on lines 104-107:
>>>
>>> ReaderType::Pointer reader = ReaderType::New();
>>>
>>>
>>>
>>> reader->SetFileName(argv[1]);
>>> reader->SetImageIO(io);
>>>
>>> reader->Update();
>>>
>>>
>>> Here, calling reader->Update() loads the entire image into memory. If
>>> you have a very large LSM file (several gigabytes large), then this code
>>> will fail since such an image cannot get into the memory.
>>>
>>> Hence, I am trying to send the read region directly into the reader using
>>>
>>> io->SetRegion(region);
>>>
>>> and then
>>>
>>> reader->SetImageIO(io);
>>> reader->Update();
>>>
>>>
>>> But this step is not giving me two different timepoints. It is writing
>>> out the same timepoint.
>>>
>>> Kishore
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Oct 16, 2012 at 2:13 PM, Mark Hiner <hiner at wisc.edu> wrote:
>>>
>>>> Hello again,
>>>>
>>>> I created a gist for your LSM test and updated it with my most recent
>>>> code:
>>>> https://gist.github.com/3900888
>>>>
>>>> That example is currently working for me, in that it writes out 2
>>>> images of 72 planes - one for T=0 and one for T=1. Since everything in ITK
>>>> seems to be pipeline-oriented, I am not sure there is a solution to
>>>> splitting out channels just using Setters like we were both trying.
>>>>
>>>> Instead, I followed this ROI Filter example<http://www.vtk.org/Wiki/ITK/Examples/ImageProcessing/RegionOfInterestImageFilter>and used the logic from my last e-mail.
>>>>
>>>> I do need to correct one thing I said in my last e-mail: SizeT should
>>>> always be 1, as it is a relative offset to the Index value.
>>>>
>>>> When I ran that test, both images went to grayscale.. I'm going to see
>>>> if that's an error in the bf-itk-pipes logic, or just something that needs
>>>> to be set up differently in using the ITK API.
>>>>
>>>> Let me know if you run into any more problems with this code, or if
>>>> you just want to discuss any part of it.
>>>>
>>>> Thanks again for using our software :)
>>>> - Mark
>>>>
>>>>
>>>> On Tue, Oct 16, 2012 at 11:00 AM, Mark Hiner <hiner at wisc.edu> wrote:
>>>>
>>>>> Hi Kishore,
>>>>>
>>>>> I am sorry I wasn't able to get to your issue yesterday. Looking at
>>>>> it today, I think there are a couple of things going on here.
>>>>>
>>>>> First of all: in your test class
>>>>> - I consolidated a BioFormatsImageIO and the ImageIOBase.
>>>>> - In the region updating loop, the wrong region (dim-3 instead of
>>>>> dim-2) was being set, and only the index is set. I suspect both the index
>>>>> and size need to be set.
>>>>>
>>>>> I attached an updated version of your test, where in the first
>>>>> iteration it sets IndexT = 0, sizeT = 1. The second iteration sets IndexT =
>>>>> 1, sizeT = 2. I believe those regions will split the time points as you
>>>>> wanted.
>>>>>
>>>>> But there is a second issue: although the Reader IORegion is correct
>>>>> (I think) the Writer is not. It is defaulting to the ITK base 3D region.
>>>>> When I tried setting the Writer's region to our desired region, it throws
>>>>> an exception because the 5D region can't be contained by what it thinks is
>>>>> the largest possible (3D) region. I am refreshing myself on the ITK API and
>>>>> once we figure out how to set that region, I think it will write the images
>>>>> correctly (or we can move on to the next problem).
>>>>>
>>>>> In your test I changed the output format to ome.tiff because I know
>>>>> that will use the itkBioFormatsImageIO writer code, which I know is capable
>>>>> of handling 5D writing.
>>>>>
>>>>> Also, in testing I found a couple of bugs in the ITKPipesBridge code.
>>>>> I started a new branch to fix these and any other issues that may be
>>>>> discovered. If you want to use the itkBioFormatsImageIO writer code you'll
>>>>> have to use this new branch for now. You can find it here:
>>>>> https://github.com/hinerm/bioformats/commits/bf-itk-fixes
>>>>>
>>>>> As always, let me know if this isn't fully addressing your issue.
>>>>> Otherwise, I'll let you know as soon as I have the Writer region setting
>>>>> properly.
>>>>>
>>>>> Thanks!
>>>>> - Mark
>>>>>
>>>>>
>>>>> On Fri, Oct 12, 2012 at 11:28 AM, Kishore Mosaliganti <
>>>>> kishoreraom at gmail.com> wrote:
>>>>>
>>>>>> Hi Mark,
>>>>>>
>>>>>> Thank you for the email and invaluable help. The bioformats-itk
>>>>>> plugin is wonderful. I am attaching my script here to help you debug. Let
>>>>>> me know if you have any questions on my script. I am starting with a 5D
>>>>>> (XYZTC) image and trying to write out all the 3D timepoints for the first
>>>>>> channel.
>>>>>>
>>>>>> Kishore
>>>>>>
>>>>>>
>>>>>> On Fri, Oct 12, 2012 at 12:19 PM, Mark Hiner <hinerm at gmail.com>wrote:
>>>>>>
>>>>>>> Hi Kishore,
>>>>>>>
>>>>>>> I helped develop the bf-itk-pipe plugin and am investigating your
>>>>>>> issue right now.
>>>>>>>
>>>>>>> What you're doing seems reasonable, so I'm trying to determine if
>>>>>>> there's a bug in the plugin. My goal is to have some working example code
>>>>>>> for you (and a fix if necessary) by the end of the day.
>>>>>>>
>>>>>>> Thank you for using our plugin! I hope we can get it working for you
>>>>>>> soon.
>>>>>>>
>>>>>>> - Mark
>>>>>>>
>>>>>>> On Fri, Oct 12, 2012 at 10:54 AM, Kishore Mosaliganti <
>>>>>>> kishoreraom at gmail.com> wrote:
>>>>>>>
>>>>>>>> Dear ITK and OME users,
>>>>>>>>
>>>>>>>> To further elaborate, I figured that this has nothing to do with
>>>>>>>> the itkStreamingImageFilter.
>>>>>>>>
>>>>>>>> Instead, the itkImageFileReader seems to be extracting the same
>>>>>>>> image region although I update the requested region in the
>>>>>>>> BioformatsImageIO class :;
>>>>>>>>
>>>>>>>> itk::BioFormatsImageIO::Pointer io =
>>>>>>>> itk::BioFormatsImageIO::New();
>>>>>>>> io->SetIORegion( region ); // doesn't make a difference in
>>>>>>>> terms of image data extracted. Its always from the beginning.
>>>>>>>>
>>>>>>>> ReaderType::Pointer reader = ReaderType::New();
>>>>>>>> reader->SetFileName(argv[1]);
>>>>>>>> reader->SetImageIO(io);
>>>>>>>> reader->Update();
>>>>>>>>
>>>>>>>> So, my general question is whether the BioformatsImageIO expects
>>>>>>>> the whole LSM image to be loaded into memory before writing it out? I would
>>>>>>>> like to specify small image regions since my LSM is too large to be fully
>>>>>>>> loaded into memory. How else can I stream data from large LSM?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Kishore
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Oct 12, 2012 at 8:53 AM, Kishore Mosaliganti <
>>>>>>>> kishoreraom at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> I am trying to use bioformats and itk. I downloaded and compiled
>>>>>>>>> bf-itk-pipe using cmake by linking against ITK 3.2:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> https://github.com/openmicroscopy/bioformats/tree/develop/components/native/bf-itk-pipe
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I tested out itkBFImageInfo and ./itkRGBBioFormatsImageIOTest on a
>>>>>>>>> few simple LSM images. It works great and writes out all the associated
>>>>>>>>> metadata and pixel data for the first timepoint.
>>>>>>>>>
>>>>>>>>> My LSM microscopy image is 5D. It is X-Y-Z-Time-Channel. I am
>>>>>>>>> interested in extracting individual timepoints from the LSM file for
>>>>>>>>> processing. The filter seems to be using itkStreamingImageFilter. I set the
>>>>>>>>> NumberOfStreams to 10 but that still seems to still write out only the
>>>>>>>>> first image.
>>>>>>>>>
>>>>>>>>> How do I set the output requestion region for an individual
>>>>>>>>> timepoint?
>>>>>>>>>
>>>>>>>>> Thank you,
>>>>>>>>>
>>>>>>>>> Kishore
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> ome-users mailing list
>>>>>>>> ome-users at lists.openmicroscopy.org.uk
>>>>>>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20121019/8c9ca040/attachment.html>
More information about the ome-users
mailing list