<div dir="ltr"><div><div><div><div><div><div><div><div><div><div>Hi Joe,<br><br></div> First of all, in case you are using the bf-itk-pipes plugin, you should know that we currently have a <a href="http://review.source.kitware.com/9450">patch under review</a> by the ITK developers. This patch adds a true Bio-Formats (SCIFIO) ITK ImageIO, which will be available by default in ITK.<br>
<br></div> This means that the old bf-itk-pipes plugin won't be supported, as all development efforts are going towards the SCIFIOImageIO.<br><br></div> As this code is still under review, there are issues that I'm currently working on resolving.<br>
<br></div> So first I'd ask that you check out the latest patch code using the links on that review site and rework the splitting code to use <a href="http://review.source.kitware.com/#/c/9450/3/Modules/IO/SCIFIO/include/itkSCIFIOImageIO.h">SCIFIOImageIO</a> instead of the BioFormatsImageIO. You can also look at the <a href="http://review.source.kitware.com/#/c/9450/3/Modules/IO/SCIFIO/test/itkSCIFIOImageIOTest.cxx">test directory</a> for more sample code.<br>
<br></div> If you have any trouble updating your code, please let me know and I'll be happy to help.<br><br></div> As to your actual issue of the .oif image hanging, I actually just fixed a bug in the last week regarding hanging when plane size is miscalculated by the plugin, so with any luck that was your issue. :)<br>
<br></div> If the latest SCIFIOImageIO still gives you problems with your .oif dataset though let me know and I'll investigate.<br><br></div> As for the 2chZT data, that sounds unrelated to the hanging so I'll take a look at it locally when I get a chance.<br>
<br></div> Thanks for using our software and reporting these issues!<br><br></div>- Mark<br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Apr 23, 2013 at 2:40 PM, Joe Ping-Lin Hsiao <span dir="ltr"><<a href="mailto:phsiao@cs.unc.edu" target="_blank">phsiao@cs.unc.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Mark,<div> I tested your code with the .lsm download from <a href="http://www.loci.wisc.edu/files/software/data/2chZT.zip" target="_blank">http://www.loci.wisc.edu/files/software/data/2chZT.zip</a>. The program just crashed with the error message</div>
<div><br></div><div>"This application has requested the Runtime to terminate it in an unusual way.</div><div>Please contact the application's support team for more information."</div><div class="gmail_extra">
<br>If I tested with my own 5D .oif image collection, the execution hangs at 'extractor->Update()' and it does not even use any CPU. Any idea how to fix it?</div><div class="gmail_extra"><br></div><div class="gmail_extra">
Thanks,</div><div class="gmail_extra">Joe</div><div><div class="h5"><div class="gmail_extra"><br></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Oct 19, 2012 at 12:41 PM, Mark Hiner <span dir="ltr"><<a href="mailto:hiner@wisc.edu" target="_blank">hiner@wisc.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Kishore,<br><br> I updated <a href="https://gist.github.com/3900888/81064cf96fb06b4575a64e81481fda0936cd9056" target="_blank">your test</a> to use an ExtractImageFilter in the IO pipeline. When a Reader's output is piped to this Filter, the reader->Update() call only reads the extracted region.<br>
<br> When I tested on the ConfocalMemCitrine516LeftEar18hpfAug17_2012_Fish1.lsm test image you uploaded, it correctly splits out the stacks for each time point. And when I tested on a 9GB image it processed each time point without reading the entire file.<br>
<br> So I'm hoping this solves your issue. If not, just let me know how it's failing for you and I'll try again!<br><br>Thanks,<br>Mark<div><div><br><br><div class="gmail_quote">On Wed, Oct 17, 2012 at 2:34 PM, Mark Hiner <span dir="ltr"><<a href="mailto:hiner@wisc.edu" target="_blank">hiner@wisc.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">One thought is that, as the itkBioFormatsImageIO is a <a href="http://www.itk.org/Doxygen320/html/classitk_1_1StreamingImageIOBase.html" target="_blank">StreamingImageIOBase</a>, you could just set the desired region (as you were) and use <b>its</b> Read and Write methods instead of wrapping it in a Reader or a Writer.<br>
<br>I'll try to put a code example up using this approach. It would be a good test case for the bf-itk-pipe plug-in anyway.<span><font color="#888888"><br><br>- Mark</font></span><div><div>
<br><br><br><div class="gmail_quote">On Wed, Oct 17, 2012 at 1:56 PM, Mark Hiner <span dir="ltr"><<a href="mailto:hinerm@gmail.com" target="_blank">hinerm@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Kishore,<br><br> Just wanted to let you know that I'm still looking at your issue today. I updated <a href="https://gist.github.com/3900888/7aab8fddb38af87ea98cd49dd921524ace6430c8" target="_blank">the gist</a> to reflect the topics discussed in this e-mail, using the region setting method calls instead of pipelining, and added some (hopefully) helpful print statements to see what's going on with the regions.<br>
<br> From what I can tell there are two separate issues preventing your code from writing the desired regions.<br><br> First of all, the IORegion needs to be set on the Writer, for example:<br><pre><div>
<span>writer</span><span>-></span><span>SetImageIO</span><span>(</span><span>io</span><span>);</span></div><div> <span>writer</span><span>-></span><span>SetInput</span><span>(</span><span>reader</span><span>-></span><span>GetOutput</span><span>());</span></div>
<div> <span>writer</span><span>-></span><span>SetIORegion</span><span>(</span><span>region</span><span>);<br> writer->Write();<br>
</span></div></pre> In the original code you sent to me, that wasn't happening and it was defaulting to the first 3D region and thus the first time point. I mentioned this, without providing a code example, in a past e-mail - so apologies if you've already updated your code!<br>
<br> Note that according to the <a href="http://www.itk.org/Doxygen/html/classitk_1_1ImageFileWriter.html#ab2cba65f716aa75ec962171ee0b46fe3" target="_blank">itkImageFileWriter API</a> Write() is a special form of Update() that respects the IORegion. However, in practice, for me it still just writes the largest possible region instead of what was specified. So that could suggest a bug in the bf-itk-pipes plug-in.<br>
<br> The second issue is that even after manually specifying the IO regions, the Reader output still has the largest possible dimensions (T=2) so the current gist test ends up creating 2 full copies of the image. I think this was always happening at the Reader, but then it was truncated to the first time point you saw by the Writer. Anyway, this implies that the full image is still being read, despite the region specifications.<br>
<br> I think this is a side effect of setting using the LargestPossibleRegion in the Update() method, as mentioned in the <a href="http://www.itk.org/Doxygen/html/classitk_1_1ProcessObject.html#a4041fb21e9105500eee311e265691bd5" target="_blank">itkProcessObject specification</a>.<br>
<br> Right now I'm going through the examples from <a href="http://itk-insight-users.2283740.n2.nabble.com/Memory-limits-of-ITKImage-object-td5670422.html" target="_blank">this ITK Users thread</a> about streaming for a better pipelining solution. I'm also testing to see what can be done at the ImageIO level, as when Read and Write are hit in the itkBioFormatsImageIO, GetIORegion() is already returning the largest possible region, instead of the specified region.<br>
<br> I'll keep you updated.<span><font color="#888888"><br><br>- Mark</font></span><div><div><br><br><div class="gmail_quote">On Tue, Oct 16, 2012 at 1:37 PM, Kishore Mosaliganti <span dir="ltr"><<a href="mailto:kishoreraom@gmail.com" target="_blank">kishoreraom@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span><span name="Mark"></span> </span><div><img alt=""></div><div>
<div><div>Dear Mark,<br><br>Thanks for the clarification. My original problem remains unresolved. For example, in the LSM gist on lines 104-107:</div><div><br><div><pre><div> <span>ReaderType</span><span>::</span><span>Pointer</span> <span>reader</span> <span>=</span> <span>ReaderType</span><span>::</span><span>New</span><span>();</span></div>
<div> <span>reader</span><span>-></span><span>SetFileName</span><span>(</span><span>argv</span><span>[</span><span>1</span><span>]);</span></div>
<div> <span>reader</span><span>-></span><span>SetImageIO</span><span>(</span><span>io</span><span>);</span></div><div>
<span>reader</span><span>-></span><span>Update</span><span>();</span></div></pre><br></div></div><div>Here,
calling reader->Update() loads the entire image into memory. If you
have a very large LSM file (several gigabytes large), then this code
will fail since such an image cannot get into the memory. <br>
<br>Hence, I am trying to send the read region directly into the reader using<br><br>io->SetRegion(region);<br><br>and then <br><br>reader->SetImageIO(io);<br>reader->Update();<br><br><br></div>But this step is not giving me two different timepoints. It is writing out the same timepoint.<br>
<br>Kishore<div><div><img><br><br><br></div></div></div></div><br><br><div class="gmail_quote"><div>On Tue, Oct 16, 2012 at 2:13 PM, Mark Hiner <span dir="ltr"><<a href="mailto:hiner@wisc.edu" target="_blank">hiner@wisc.edu</a>></span> wrote:<br>
</div><div><div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hello again,<br><br> I created a gist for your LSM test and updated it with my most recent code:<br>
<a href="https://gist.github.com/3900888" target="_blank">https://gist.github.com/3900888</a><br>
<br> That example is currently working for me, in that it writes out 2 images of 72 planes - one for T=0 and one for T=1. Since everything in ITK seems to be pipeline-oriented, I am not sure there is a solution to splitting out channels just using Setters like we were both trying.<br>
<br> Instead, I followed <a href="http://www.vtk.org/Wiki/ITK/Examples/ImageProcessing/RegionOfInterestImageFilter" target="_blank">this ROI Filter example</a> and used the logic from my last e-mail.<br><br> I do need to correct one thing I said in my last e-mail: SizeT should always be 1, as it is a relative offset to the Index value.<br>
<br> When I ran that test, both images went to grayscale.. I'm going to see if that's an error in the bf-itk-pipes logic, or just something that needs to be set up differently in using the ITK API.<br><br> Let me know if you run into any more problems with this code, or if you just want to discuss any part of it.<br>
<br>Thanks again for using our software :)<span><font color="#888888"><br>- Mark</font></span><div><div><br><br><div class="gmail_quote">On Tue, Oct 16, 2012 at 11:00 AM, Mark Hiner <span dir="ltr"><<a href="mailto:hiner@wisc.edu" target="_blank">hiner@wisc.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Kishore,<br><br> I am sorry I wasn't able to get to your issue yesterday. Looking at it today, I think there are a couple of things going on here.<br>
<br> First of all: in your test class<br>- I consolidated a BioFormatsImageIO and the ImageIOBase.<br>
- In the region updating loop, the wrong region (dim-3 instead of dim-2) was being set, and only the index is set. I suspect both the index and size need to be set.<br><br> I attached an updated version of your test, where in the first iteration it sets IndexT = 0, sizeT = 1. The second iteration sets IndexT = 1, sizeT = 2. I believe those regions will split the time points as you wanted.<br>
<br>But there is a second issue: although the Reader IORegion is correct (I think) the Writer is not. It is defaulting to the ITK base 3D region. When I tried setting the Writer's region to our desired region, it throws an exception because the 5D region can't be contained by what it thinks is the largest possible (3D) region. I am refreshing myself on the ITK API and once we figure out how to set that region, I think it will write the images correctly (or we can move on to the next problem).<br>
<br>In your test I changed the output format to ome.tiff because I know that will use the itkBioFormatsImageIO writer code, which I know is capable of handling 5D writing.<br><br>Also, in testing I found a couple of bugs in the ITKPipesBridge code. I started a new branch to fix these and any other issues that may be discovered. If you want to use the itkBioFormatsImageIO writer code you'll have to use this new branch for now. You can find it here:<br>
<a href="https://github.com/hinerm/bioformats/commits/bf-itk-fixes" target="_blank">https://github.com/hinerm/bioformats/commits/bf-itk-fixes</a><br><br>As always, let me know if this isn't fully addressing your issue. Otherwise, I'll let you know as soon as I have the Writer region setting properly.<br>
<br>Thanks!<span><font color="#888888"><br>- Mark</font></span><div><div><br><br><div class="gmail_quote">On Fri, Oct 12, 2012 at 11:28 AM, Kishore Mosaliganti <span dir="ltr"><<a href="mailto:kishoreraom@gmail.com" target="_blank">kishoreraom@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Mark,<br><br>Thank you for the email and invaluable help. The bioformats-itk plugin is wonderful. I am attaching my script here to help you debug. Let me know if you have any questions on my script. I am starting with a 5D (XYZTC) image and trying to write out all the 3D timepoints for the first channel.<span><font color="#888888"><br>
<br>Kishore</font></span><div><div><br><br><div class="gmail_quote">On Fri, Oct 12, 2012 at 12:19 PM, Mark Hiner <span dir="ltr"><<a href="mailto:hinerm@gmail.com" target="_blank">hinerm@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Kishore,<br><br> I helped develop the bf-itk-pipe plugin and am investigating your issue right now. <br><br> What you're doing seems reasonable, so I'm trying to determine if there's a bug in the plugin. My goal is to have some working example code for you (and a fix if necessary) by the end of the day.<br>
<br>Thank you for using our plugin! I hope we can get it working for you soon.<br><br>- Mark<br><br><div class="gmail_quote"><div><div>On Fri, Oct 12, 2012 at 10:54 AM, Kishore Mosaliganti <span dir="ltr"><<a href="mailto:kishoreraom@gmail.com" target="_blank">kishoreraom@gmail.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div>Dear ITK and OME users,<br><br>To further elaborate, I figured that this has nothing to do with the itkStreamingImageFilter.<br>
<br>Instead, the itkImageFileReader seems to be extracting the same image region although I update the requested region in the BioformatsImageIO class :;<br>
<br> itk::BioFormatsImageIO::Pointer io = itk::BioFormatsImageIO::New();<br> io->SetIORegion( region ); // doesn't make a difference in terms of image data extracted. Its always from the beginning.<br><br> ReaderType::Pointer reader = ReaderType::New();<br>
reader->SetFileName(argv[1]);<br> reader->SetImageIO(io);<br> reader->Update();<br><br>So, my general question is whether the BioformatsImageIO expects the whole LSM image to be loaded into memory before writing it out? I would like to specify small image regions since my LSM is too large to be fully loaded into memory. How else can I stream data from large LSM?<br>
<br>Thanks,<br><br>Kishore<br><br><br><div class="gmail_quote">On Fri, Oct 12, 2012 at 8:53 AM, Kishore Mosaliganti <span dir="ltr"><<a href="mailto:kishoreraom@gmail.com" target="_blank">kishoreraom@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi all,<br><br>I am trying to use bioformats and itk. I downloaded
and compiled bf-itk-pipe using cmake by linking against ITK 3.2:<br><br><a href="https://github.com/openmicroscopy/bioformats/tree/develop/components/native/bf-itk-pipe" target="_blank">https://github.com/openmicroscopy/bioformats/tree/develop/components/native/bf-itk-pipe</a><br>
<br><br>I tested out itkBFImageInfo
and ./itkRGBBioFormatsImageIOTest on a few simple LSM images. It works
great and writes out all the associated metadata and pixel data for the first timepoint. <br>
<br>My LSM microscopy image is 5D. It is X-Y-Z-Time-Channel. I am interested in extracting individual timepoints from the LSM file for processing. The filter seems to be using itkStreamingImageFilter. I set the NumberOfStreams
to 10 but that still seems to still write out only the first image. <br><br>How do I set the output requestion region for an individual timepoint?<br>
<br>Thank you,<br><br>Kishore
</blockquote></div><br>
<br></div></div>_______________________________________________<br>
ome-users mailing list<br>
<a href="mailto:ome-users@lists.openmicroscopy.org.uk" target="_blank">ome-users@lists.openmicroscopy.org.uk</a><br>
<a href="http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users" target="_blank">http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users</a><br>
<br></blockquote></div><br>
</blockquote></div><br>
</div></div></blockquote></div><br>
</div></div></blockquote></div><br>
</div></div></blockquote></div></div></div><br>
</blockquote></div><br>
</div></div></blockquote></div><br>
</div></div></blockquote></div><br>
</div></div><br>_______________________________________________<br>
ome-users mailing list<br>
<a href="mailto:ome-users@lists.openmicroscopy.org.uk" target="_blank">ome-users@lists.openmicroscopy.org.uk</a><br>
<a href="http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users" target="_blank">http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users</a><br>
<br></blockquote></div><br></div></div></div></div>
</blockquote></div><br></div>