[ome-users] bioformats and ITK

Joe Ping-Lin Hsiao phsiao at cs.unc.edu
Thu May 23 14:52:39 BST 2013 

Mark,

Adding the SCIFIO_PATH environment variable doesn't work for me. I
experimented with 'build/lib/jars' and 'build/lib/jars/', but I still have
the same error.
I can wait for the bug fix in ITK to get approved and try again.

Thanks,
Joe


On Wed, May 22, 2013 at 3:00 PM, Mark Hiner <hiner at wisc.edu> wrote:

> Hi Joe,
>
>  Just wanted to send you an update... I eventually realized it wasn't a
> Debug/Release issue, but rather a classpath separator issue. The Linux/OSX
> separator was hard-coded in the SCIFIO ImageIO.
>
>  So the scifio-imageio <https://github.com/scifio/scifio-imageio>repository has been updated to choose the correct separator for OSX or
> Windows. I also filed this bug fix
> <http://review.source.kitware.com/#/c/11375/>updating ITK to download the
> updated scifio-imageio commit.
>
>  Also, while looking through the code, I had forgotten that the JVM
> classpath (where those jars need to be) isn't pulled from the PATH... it's
> taken from a special environment variable: SCIFIO_PATH. If you set
> SCIFIO_PATH to the directory containing the itk-bridge jar and
> loci_tools.jar (build/lib/jars) it should work without the update. (*
> should* work..)
>
>  Note that the SCIFIO_PATH is prepended onto each jar dependency.
>
> Anyway, I hope this resolves everything for you. If you run into more
> trouble, let me know.
>
> Thanks,
> Mark
>
>
> On Fri, May 17, 2013 at 3:39 PM, Mark Hiner <hiner at wisc.edu> wrote:
>
>> Joe,
>>
>>  That's great! It's almost certainly an issue of the jars being put in
>> the wrong place because of the Debug/Release addition on Windows.
>>
>>  I confirmed that it doesn't work on Windows :-p I think it's in the
>> DownloadSCIFIO.cmake.in which downloads to ${jarsDownloadDirectory}...
>> but the jars are installed from
>> ${CMAKE_LIBRARY_OUTPUT_DIRECTORY}/${jarsDownloadDirectory} which I'm
>> assuming points to lib/Debug/jars, which doesn't exist.
>>
>> Unfortunately I don't have time to verify that fixes things today. I'll
>> try to test it on the weekend or Monday.
>>
>>
>> On Fri, May 17, 2013 at 2:29 PM, Joe Ping-Lin Hsiao <phsiao at cs.unc.edu>wrote:
>>
>>> Mark,
>>>
>>> Just want to let you know that I repeated the same steps on my macbook
>>> with Mac OSX 10.8.3, and program works fine.
>>>
>>> Joe
>>>
>>>
>>> On Fri, May 17, 2013 at 10:45 AM, Mark Hiner <hinerm at gmail.com> wrote:
>>>
>>>> Joe,
>>>>
>>>>  That's odd.. not sure why manually adding them to the classpath didn't
>>>> work. I will test locally on a Windows 7 box and let you know if I can
>>>> reproduce and/or resolve the issue.
>>>>
>>>> Thanks for reporting it, hopefully we can get it working for you soon.
>>>>
>>>> - Mark
>>>>
>>>>
>>>> On Thu, May 16, 2013 at 4:30 PM, Joe Ping-Lin Hsiao <phsiao at cs.unc.edu>wrote:
>>>>
>>>>> Mark,
>>>>>
>>>>> Yes, I was using the command "ITKIOSCIFIOTestDriver
>>>>> itkSCIFIOImageIOTest inputfile outputfile".
>>>>>
>>>>> I am on Windows 7. The two jars (loci_tools.jar and
>>>>> scifio-itk-bridge-1.0.0.jar) are in ITK/lib/jars.
>>>>>
>>>>> I manually added the path to classpath, and my classpath is now:
>>>>> CLASSPATH=.;C:\Program Files
>>>>> (x86)\Java\jre6\lib\ext\QTJava.zip;D:\Joe\dev\ITK\lib\jars
>>>>>
>>>>> But I still have no luck.
>>>>>
>>>>> After running the command, I first get a dialog of
>>>>>  "Could not find the main class: loci.scifio.itk.SCIFIOITKBridge.
>>>>> Program will exit."
>>>>> Please see attached image.
>>>>>
>>>>> And there are also error messages in the console:
>>>>>
>>>>> D:\Joe\dev\ITK\bin\Release>ITKIOSCIFIOTestDriver itkSCIFIOImageIOTest
>>>>> C:\Users\phsiao\Desktop\itk-scifio-build\Release\
>>>>> chZT.lsm C:\Users\phsiao\Desktop\itk-scifio-build\Release\abc.tif
>>>>> reader->GetUseStreaming(): 1
>>>>> done checking streaming usage
>>>>> itk::ExceptionObject (0000000000A4EF10)
>>>>> Location: "unknown"
>>>>> File: .\itkSCIFIOImageIO.cxx
>>>>> Line: 172
>>>>> Description: itk::ERROR: SCIFIOImageIO(0000000002A5C080):
>>>>> SCIFIOImageIO exited abnormally. java.lang.NoClassDefFoundErr
>>>>> r: loci/scifio/itk/SCIFIOITKBridge
>>>>> Caused by: java.lang.ClassNotFoundException:
>>>>> loci.scifio.itk.SCIFIOITKBridge
>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>>>>>         at
>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>>>>> Exception in thread "main"
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Thu, May 16, 2013 at 4:46 PM, Mark Hiner <hiner at wisc.edu> wrote:
>>>>>
>>>>>> Hi Joe,
>>>>>>
>>>>>>  Hmm.. well that is the correct package specification. What was the
>>>>>> exact command you used? I'm assuming it was something like
>>>>>>
>>>>>> "ITKIOSCIFIOTestDriver itkSCIFIOImageIOTest inputfile outputfile"
>>>>>>
>>>>>> ?
>>>>>>
>>>>>> Anyway it sounds like the scifio-itk-bridge jar isn't on your
>>>>>> classpath. I just tested a clean build off ITK.git master (commit
>>>>>> 2e056e0cee1271d3080a9bfc4f7353c1d9165ef9) and it worked for me... but
>>>>>> that's on OSX.
>>>>>>
>>>>>> What's in your build/lib/jars directory? It should have 2 things:
>>>>>> loci_tools.jar
>>>>>> scifio-itk-bridge-1.0.0.jar
>>>>>>
>>>>>> It's possible that they aren't being added to the classpath correctly
>>>>>> though, if on Windows there's another directory level (e.g. is it
>>>>>> build/Release/lib/jars ?). So if they are in a different directory, let me
>>>>>> know.
>>>>>>
>>>>>> If the jars are there, try manually adding them to your classpath for
>>>>>> now and let me know how that goes. If they're not then it's a problem with
>>>>>> the download script.
>>>>>>
>>>>>> You can download the jars manually from:
>>>>>>
>>>>>> http://cvs.openmicroscopy.org.uk/snapshots/bioformats/4.4.5/loci_tools.jar
>>>>>>
>>>>>> http://jenkins.imagej.net/view/SCIFIO/job/SCIFIOITKBridge/lastSuccessfulBuild/artifact/target/scifio-itk-bridge-1.0.0.jar
>>>>>>
>>>>>> That's all I can think of at the moment...
>>>>>>
>>>>>> - Mark
>>>>>>
>>>>>>
>>>>>> On Thu, May 16, 2013 at 1:02 PM, Joe Ping-Lin Hsiao <
>>>>>> phsiao at cs.unc.edu> wrote:
>>>>>>
>>>>>>> Hi Mark,
>>>>>>>
>>>>>>> Glad to see SCIFIO is now becoming ITK's remote module. Thanks for
>>>>>>> your continuous working.
>>>>>>> I did a clean git clone of the latest ITK, and configure it with
>>>>>>>
>>>>>>>    Fetch_SCIFIO
>>>>>>>    BUILD_TESTING
>>>>>>>
>>>>>>> set to ON. Then I run ITKIOSCIFIOTestDriver.exe on an image file,
>>>>>>> but I got the error message
>>>>>>>
>>>>>>> "Could not find the main class: loci.scifio.itk.SCIFIOITKBridge.
>>>>>>> Program will exit."
>>>>>>>
>>>>>>> Do I miss setting some path, or it still requires Bio-Formats?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Joe
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Apr 26, 2013 at 2:43 PM, Mark Hiner <hiner at wisc.edu> wrote:
>>>>>>>
>>>>>>>> Hi again Joe,
>>>>>>>>
>>>>>>>>  I noticed a couple things that may help you: in the
>>>>>>>> itkVectorImageSCIFIOImageIOTest you sent me (and in all the test classes I
>>>>>>>> mentioned in my last e-mail) the PixelType and Dimension variables must be
>>>>>>>> set carefully.
>>>>>>>>
>>>>>>>>  For example, with the code in the state you sent me, PixelType is
>>>>>>>> an unsigned short (which is 16 bits) and the dimension is 3.
>>>>>>>>
>>>>>>>>  These settings would be incorrect for the 2chZT.lsm sample data,
>>>>>>>> as it has an 8 bit unsigned pixel type (so PixelType should be defined as
>>>>>>>> unsigned char). Also if you're using the SCIFIOImageIO you can typically
>>>>>>>> set dimensions to 5.
>>>>>>>>
>>>>>>>>  I don't know the specifications of your .oif dataset, but the
>>>>>>>> PixelType and dimensions may need to be adjusted for it.
>>>>>>>>
>>>>>>>>   Sorry if you already knew this and were updating these values!
>>>>>>>>
>>>>>>>>   Anyway, I've been working on improving the stability of the
>>>>>>>> SCIFIOImageIO plugin. Using the itkSCIFIOImageIOTest I mentioned
>>>>>>>> previously, I confirmed that I was able to read and write the 2chsZT.lsm
>>>>>>>> file and the .oif test images I have access to.
>>>>>>>>
>>>>>>>>   These changes aren't in an ITK patch set yet because they include
>>>>>>>> changes to both the c++ and java code, which will may require some
>>>>>>>> restructuring of SCIFIOImageIO.
>>>>>>>>
>>>>>>>> However, if you're eager to test these changes, you could build off
>>>>>>>> my development branches:
>>>>>>>> 1. Clone my ITK repo. <https://github.com/hinerm/ITK> The default
>>>>>>>> branch is what you want here.
>>>>>>>> 2. Build ITK as normal (with tests enabled)
>>>>>>>> 3. Clone my Bio-Formats repo <https://github.com/hinerm/bioformats>
>>>>>>>> .
>>>>>>>> 4. Check out the itk-bridge-fixes branch in Bio-Formats
>>>>>>>> 5. From the top level of Bio-Formats, run "ant tools"
>>>>>>>> 6. Copy the loci_tools.jar from ${bio-formats}/artifacts to
>>>>>>>> ${ITK_build}/lib/jars/
>>>>>>>>    (this will overwrite the loci_tools.jar that was downloaded
>>>>>>>> automatically when ITK built)
>>>>>>>>
>>>>>>>>  At this point you can link in the ImageToVTKFilter you were using
>>>>>>>> or run the SCIFIO ImageIO test programs included in ITK.
>>>>>>>>
>>>>>>>>  Whether you build these manually or not, I'll try to get a new
>>>>>>>> official patch set released soon.
>>>>>>>>
>>>>>>>>  Let me know if you run into any problems.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Mark
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Apr 25, 2013 at 4:12 PM, Mark Hiner <hinerm at gmail.com>wrote:
>>>>>>>>
>>>>>>>>> Hi Joe,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Here are my steps to compile the program, in case I missed
>>>>>>>>>> something:
>>>>>>>>>> 1. Copy the git commend from the 'Patch set 3' review page, and
>>>>>>>>>> paste it under the master ITK source, which downloaded a bunch of files and
>>>>>>>>>> switched me to the branch 'SCIFIOImageIO'.
>>>>>>>>>> 2. Compile the new ITK source in CMake, with Module_ITKVtkGlue,
>>>>>>>>>> ITK_USE_REVIEW, and BUILD_SHARED_LIBS checked.
>>>>>>>>>> 3. Link the program with the ITK with SCIFIOImageIO.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> This looks good. Note that a Patch set 4 is now out, and I'm
>>>>>>>>> working on Patch 5. I'm hoping to include any necessary fixes for your
>>>>>>>>> issues in patch 5.
>>>>>>>>>
>>>>>>>>> So, to clarify here - you're end goal is to convert your .oif
>>>>>>>>> dataset to .ome.tiff? Is that right? Are you specifically wanting to
>>>>>>>>> extract a subset of the image or anything? If you're just trying to do a
>>>>>>>>> simple conversion it may be better to use the tests that are part of the
>>>>>>>>> ITK SCIFIOImageIO module. You can take a look at them at:
>>>>>>>>> $ITK/Modules/IO/SCIFIO/test
>>>>>>>>>
>>>>>>>>> To build them, just turn the BUILD_TESTING flag on when
>>>>>>>>> configuring cmake. The ITKIOSCIFIOTestDriver is built to $build/bin
>>>>>>>>>
>>>>>>>>> I've been using the itkSCIFIOImageIOTest to test image conversion,
>>>>>>>>> for example.
>>>>>>>>>
>>>>>>>>> The hanging happens after the line printing "post-update".
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Looking at the script that suggests it read your dataset
>>>>>>>>> successfully but failed to write.
>>>>>>>>>
>>>>>>>>> Using the ImageIOTest mentioned above I tried converting the
>>>>>>>>> 2chsZT.lsm to an .ome.tiff. It got through half the planes and then crashed
>>>>>>>>> (which sounds like an improvement over your report, so I'm taking that as a
>>>>>>>>> small victory).
>>>>>>>>>
>>>>>>>>> There are some oddities in the ITK <-> Bio-Formats axis
>>>>>>>>> conversion.. it looks like only 1 of the 2 channels was detected for some
>>>>>>>>> reason. So I'll fix that in the next patch set.
>>>>>>>>>
>>>>>>>>> I got similar results with an .oif file locally.
>>>>>>>>>
>>>>>>>>> So I apologize, as I don't feel I've helped you much yet. I just
>>>>>>>>> wanted you to know that I'm still looking at these issues. Once I'm to the
>>>>>>>>> point that I can convert that 2chsZT.lsm and local .oif files I'll try to
>>>>>>>>> finalize another patch and see if that fixes things for you.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Mark
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Apr 24, 2013 at 2:26 PM, Joe Ping-Lin Hsiao <
>>>>>>>>> phsiao at cs.unc.edu> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Mark,
>>>>>>>>>>
>>>>>>>>>>     Thanks for the explanation. I have tried switching to
>>>>>>>>>> SCIFIOImageIO in the cpp code, but unfortunately the program still hangs
>>>>>>>>>> during writing files out (with a .oif input). I have attached the source
>>>>>>>>>> code. What I did was just replacing itk::BioFormatsImageIO with
>>>>>>>>>> itk::SCIFIOImageIO and a change of the included header file.
>>>>>>>>>>
>>>>>>>>>> Here are my steps to compile the program, in case I missed
>>>>>>>>>> something:
>>>>>>>>>> 1. Copy the git commend from the 'Patch set 3' review page, and
>>>>>>>>>> paste it under the master ITK source, which downloaded a bunch of files and
>>>>>>>>>> switched me to the branch 'SCIFIOImageIO'.
>>>>>>>>>> 2. Compile the new ITK source in CMake, with Module_ITKVtkGlue,
>>>>>>>>>> ITK_USE_REVIEW, and BUILD_SHARED_LIBS checked.
>>>>>>>>>> 3. Link the program with the ITK with SCIFIOImageIO.
>>>>>>>>>>
>>>>>>>>>> The hanging happens after the line printing "post-update".
>>>>>>>>>> By the way, I am using VS 2008 sp1 on Windows 7 64-bit.
>>>>>>>>>>
>>>>>>>>>> I also tested the latest bf-itk-pipe code. The hanging happens
>>>>>>>>>> after the line printing "pre-update" in this case.
>>>>>>>>>> Note that I had to link bf-itk-pipe against the SCIFIOImageIO ITK
>>>>>>>>>> otherwise there'd be a link error if linked against the master ITK.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Joe
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Apr 23, 2013 at 4:11 PM, Mark Hiner <hiner at wisc.edu>wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Joe,
>>>>>>>>>>>
>>>>>>>>>>>  First of all, in case you are using the bf-itk-pipes plugin,
>>>>>>>>>>> you should know that we currently have a patch under review<http://review.source.kitware.com/9450>by the ITK developers. This patch adds a true Bio-Formats (SCIFIO) ITK
>>>>>>>>>>> ImageIO, which will be available by default in ITK.
>>>>>>>>>>>
>>>>>>>>>>>  This means that the old bf-itk-pipes plugin won't be supported,
>>>>>>>>>>> as all development efforts are going towards the SCIFIOImageIO.
>>>>>>>>>>>
>>>>>>>>>>>  As this code is still under review, there are issues that I'm
>>>>>>>>>>> currently working on resolving.
>>>>>>>>>>>
>>>>>>>>>>>  So first I'd ask that you check out the latest patch code using
>>>>>>>>>>> the links on that review site and rework the splitting code to use
>>>>>>>>>>> SCIFIOImageIO<http://review.source.kitware.com/#/c/9450/3/Modules/IO/SCIFIO/include/itkSCIFIOImageIO.h>instead of the BioFormatsImageIO. You can also look at the test
>>>>>>>>>>> directory<http://review.source.kitware.com/#/c/9450/3/Modules/IO/SCIFIO/test/itkSCIFIOImageIOTest.cxx>for more sample code.
>>>>>>>>>>>
>>>>>>>>>>>  If you have any trouble updating your code, please let me know
>>>>>>>>>>> and I'll be happy to help.
>>>>>>>>>>>
>>>>>>>>>>>  As to your actual issue of the .oif image hanging, I actually
>>>>>>>>>>> just fixed a bug in the last week regarding hanging when plane size is
>>>>>>>>>>> miscalculated by the plugin, so with any luck that was your issue. :)
>>>>>>>>>>>
>>>>>>>>>>>  If the latest SCIFIOImageIO still gives you problems with your
>>>>>>>>>>> .oif dataset though let me know and I'll investigate.
>>>>>>>>>>>
>>>>>>>>>>>  As for the 2chZT data, that sounds unrelated to the hanging so
>>>>>>>>>>> I'll take a look at it locally when I get a chance.
>>>>>>>>>>>
>>>>>>>>>>>  Thanks for using our software and reporting these issues!
>>>>>>>>>>>
>>>>>>>>>>> - Mark
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Apr 23, 2013 at 2:40 PM, Joe Ping-Lin Hsiao <
>>>>>>>>>>> phsiao at cs.unc.edu> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Mark,
>>>>>>>>>>>>      I tested your code with the .lsm download from
>>>>>>>>>>>> http://www.loci.wisc.edu/files/software/data/2chZT.zip. The
>>>>>>>>>>>> program just crashed with the error message
>>>>>>>>>>>>
>>>>>>>>>>>> "This application has requested the Runtime to terminate it in
>>>>>>>>>>>> an unusual way.
>>>>>>>>>>>> Please contact the application's support team for more
>>>>>>>>>>>> information."
>>>>>>>>>>>>
>>>>>>>>>>>> If I tested with my own 5D .oif image collection, the execution
>>>>>>>>>>>> hangs at 'extractor->Update()' and it does not even use any CPU. Any idea
>>>>>>>>>>>> how to fix it?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Joe
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Oct 19, 2012 at 12:41 PM, Mark Hiner <hiner at wisc.edu>wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Kishore,
>>>>>>>>>>>>>
>>>>>>>>>>>>>  I updated your test<https://gist.github.com/3900888/81064cf96fb06b4575a64e81481fda0936cd9056>to use an ExtractImageFilter in the IO pipeline. When a Reader's output is
>>>>>>>>>>>>> piped to this Filter, the reader->Update() call only reads the extracted
>>>>>>>>>>>>> region.
>>>>>>>>>>>>>
>>>>>>>>>>>>>  When I tested on the
>>>>>>>>>>>>> ConfocalMemCitrine516LeftEar18hpfAug17_2012_Fish1.lsm test image you
>>>>>>>>>>>>> uploaded, it correctly splits out the stacks for each time point. And when
>>>>>>>>>>>>> I tested on a 9GB image it processed each time point without reading the
>>>>>>>>>>>>> entire file.
>>>>>>>>>>>>>
>>>>>>>>>>>>>  So I'm hoping this solves your issue. If not, just let me
>>>>>>>>>>>>> know how it's failing for you and I'll try again!
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Mark
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Oct 17, 2012 at 2:34 PM, Mark Hiner <hiner at wisc.edu>wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> One thought is that, as the itkBioFormatsImageIO is a
>>>>>>>>>>>>>> StreamingImageIOBase<http://www.itk.org/Doxygen320/html/classitk_1_1StreamingImageIOBase.html>,
>>>>>>>>>>>>>> you could just set the desired region (as you were) and use *
>>>>>>>>>>>>>> its* Read and Write methods instead of wrapping it in a
>>>>>>>>>>>>>> Reader or a Writer.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'll try to put a code example up using this approach. It
>>>>>>>>>>>>>> would be a good test case for the bf-itk-pipe plug-in anyway.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Mark
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Oct 17, 2012 at 1:56 PM, Mark Hiner <hinerm at gmail.com
>>>>>>>>>>>>>> > wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Kishore,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  Just wanted to let you know that I'm still looking at your
>>>>>>>>>>>>>>> issue today. I updated the gist<https://gist.github.com/3900888/7aab8fddb38af87ea98cd49dd921524ace6430c8>to reflect the topics discussed in this e-mail, using the region setting
>>>>>>>>>>>>>>> method calls instead of pipelining, and added some (hopefully) helpful
>>>>>>>>>>>>>>> print statements to see what's going on with the regions.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  From what I can tell there are two separate issues
>>>>>>>>>>>>>>> preventing your code from writing the desired regions.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  First of all, the IORegion needs to be set on the Writer,
>>>>>>>>>>>>>>> for example:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>     writer->SetImageIO(io);
>>>>>>>>>>>>>>>     writer->SetInput(reader->GetOutput());
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>     writer->SetIORegion(region);
>>>>>>>>>>>>>>>     writer->Write();
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  In the original code you sent to me, that wasn't happening
>>>>>>>>>>>>>>> and it was defaulting to the first 3D region and thus the first time point.
>>>>>>>>>>>>>>> I mentioned this, without providing a code example, in a past e-mail - so
>>>>>>>>>>>>>>> apologies if you've already updated your code!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  Note that according to the itkImageFileWriter API<http://www.itk.org/Doxygen/html/classitk_1_1ImageFileWriter.html#ab2cba65f716aa75ec962171ee0b46fe3>Write() is a special form of Update() that respects the IORegion. However,
>>>>>>>>>>>>>>> in practice, for me it still just writes the largest possible region
>>>>>>>>>>>>>>> instead of what was specified. So that could suggest a bug in the
>>>>>>>>>>>>>>> bf-itk-pipes plug-in.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  The second issue is that even after manually specifying the
>>>>>>>>>>>>>>> IO regions, the Reader output still has the largest possible dimensions
>>>>>>>>>>>>>>> (T=2) so the current gist test ends up creating 2 full copies of the image.
>>>>>>>>>>>>>>> I think this was always happening at the Reader, but then it was truncated
>>>>>>>>>>>>>>> to the first time point you saw by the Writer. Anyway, this implies that
>>>>>>>>>>>>>>> the full image is still being read, despite the region specifications.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  I think this is a side effect of setting using the
>>>>>>>>>>>>>>> LargestPossibleRegion in the Update() method, as mentioned in the itkProcessObject
>>>>>>>>>>>>>>> specification<http://www.itk.org/Doxygen/html/classitk_1_1ProcessObject.html#a4041fb21e9105500eee311e265691bd5>
>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  Right now I'm going through the examples from this ITK
>>>>>>>>>>>>>>> Users thread<http://itk-insight-users.2283740.n2.nabble.com/Memory-limits-of-ITKImage-object-td5670422.html>about streaming for a better pipelining solution. I'm also testing to see
>>>>>>>>>>>>>>> what can be done at the ImageIO level, as when Read and Write are hit in
>>>>>>>>>>>>>>> the itkBioFormatsImageIO, GetIORegion() is already returning the largest
>>>>>>>>>>>>>>> possible region, instead of the specified region.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  I'll keep you updated.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - Mark
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Oct 16, 2012 at 1:37 PM, Kishore Mosaliganti <
>>>>>>>>>>>>>>> kishoreraom at gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  Dear Mark,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks for the clarification. My original problem remains
>>>>>>>>>>>>>>>> unresolved. For example, in the LSM gist on lines 104-107:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>     ReaderType::Pointer reader = ReaderType::New();
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>     reader->SetFileName(argv[1]);
>>>>>>>>>>>>>>>>     reader->SetImageIO(io);
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>     reader->Update();
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Here, calling reader->Update() loads the entire image into
>>>>>>>>>>>>>>>> memory. If you have a very large LSM file (several gigabytes large), then
>>>>>>>>>>>>>>>> this code will fail since such an image cannot get into the memory.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hence, I am trying to send the read region directly into
>>>>>>>>>>>>>>>> the reader using
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> io->SetRegion(region);
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> and then
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> reader->SetImageIO(io);
>>>>>>>>>>>>>>>> reader->Update();
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> But this step is not giving me two different timepoints. It
>>>>>>>>>>>>>>>> is writing out the same timepoint.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Kishore
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Oct 16, 2012 at 2:13 PM, Mark Hiner <hiner at wisc.edu
>>>>>>>>>>>>>>>> > wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hello again,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>  I created a gist for your LSM test and updated it with my
>>>>>>>>>>>>>>>>> most recent code:
>>>>>>>>>>>>>>>>> https://gist.github.com/3900888
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>  That example is currently working for me, in that it
>>>>>>>>>>>>>>>>> writes out 2 images of 72 planes - one for T=0 and one for T=1. Since
>>>>>>>>>>>>>>>>> everything in ITK seems to be pipeline-oriented, I am not sure there is a
>>>>>>>>>>>>>>>>> solution to splitting out channels just using Setters like we were both
>>>>>>>>>>>>>>>>> trying.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>  Instead, I followed this ROI Filter example<http://www.vtk.org/Wiki/ITK/Examples/ImageProcessing/RegionOfInterestImageFilter>and used the logic from my last e-mail.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>  I do need to correct one thing I said in my last e-mail:
>>>>>>>>>>>>>>>>> SizeT should always be 1, as it is a relative offset to the Index value.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>  When I ran that test, both images went to grayscale.. I'm
>>>>>>>>>>>>>>>>> going to see if that's an error in the bf-itk-pipes logic, or just
>>>>>>>>>>>>>>>>> something that needs to be set up differently in using the ITK API.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>  Let me know if you run into any more problems with this
>>>>>>>>>>>>>>>>> code, or if you just want to discuss any part of it.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks again for using our software :)
>>>>>>>>>>>>>>>>> - Mark
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tue, Oct 16, 2012 at 11:00 AM, Mark Hiner <
>>>>>>>>>>>>>>>>> hiner at wisc.edu> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi Kishore,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>  I am sorry I wasn't able to get to your issue yesterday.
>>>>>>>>>>>>>>>>>> Looking at it today, I think there are a couple of things going on here.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>  First of all: in your test class
>>>>>>>>>>>>>>>>>> - I consolidated a BioFormatsImageIO and the ImageIOBase.
>>>>>>>>>>>>>>>>>> - In the region updating loop, the wrong region (dim-3
>>>>>>>>>>>>>>>>>> instead of dim-2) was being set, and only the index is set. I suspect both
>>>>>>>>>>>>>>>>>> the index and size need to be set.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>  I attached an updated version of your test, where in the
>>>>>>>>>>>>>>>>>> first iteration it sets IndexT = 0, sizeT = 1. The second iteration sets
>>>>>>>>>>>>>>>>>> IndexT = 1, sizeT = 2.  I believe those regions will split the time points
>>>>>>>>>>>>>>>>>> as you wanted.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> But there is a second issue: although the Reader IORegion
>>>>>>>>>>>>>>>>>> is correct (I think) the Writer is not. It is defaulting to the ITK base 3D
>>>>>>>>>>>>>>>>>> region. When I tried setting the Writer's region to our desired region, it
>>>>>>>>>>>>>>>>>> throws an exception because the 5D region can't be contained by what it
>>>>>>>>>>>>>>>>>> thinks is the largest possible (3D) region. I am refreshing myself on the
>>>>>>>>>>>>>>>>>> ITK API and once we figure out how to set that region, I think it will
>>>>>>>>>>>>>>>>>> write the images correctly (or we can move on to the next problem).
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> In your test I changed the output format to ome.tiff
>>>>>>>>>>>>>>>>>> because I know that will use the itkBioFormatsImageIO writer code, which I
>>>>>>>>>>>>>>>>>> know is capable of handling 5D writing.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Also, in testing I found a couple of bugs in the
>>>>>>>>>>>>>>>>>> ITKPipesBridge code. I started a new branch to fix these and any other
>>>>>>>>>>>>>>>>>> issues that may be discovered. If you want to use the itkBioFormatsImageIO
>>>>>>>>>>>>>>>>>> writer code you'll have to use this new branch for now. You can find it
>>>>>>>>>>>>>>>>>> here:
>>>>>>>>>>>>>>>>>> https://github.com/hinerm/bioformats/commits/bf-itk-fixes
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> As always, let me know if this isn't fully addressing
>>>>>>>>>>>>>>>>>> your issue. Otherwise, I'll let you know as soon as I have the Writer
>>>>>>>>>>>>>>>>>> region setting properly.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>> - Mark
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Fri, Oct 12, 2012 at 11:28 AM, Kishore Mosaliganti <
>>>>>>>>>>>>>>>>>> kishoreraom at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi Mark,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thank you for the email and invaluable help. The
>>>>>>>>>>>>>>>>>>> bioformats-itk plugin is wonderful. I am attaching my script here to help
>>>>>>>>>>>>>>>>>>> you debug. Let me know if you have any questions on my script. I am
>>>>>>>>>>>>>>>>>>> starting with a 5D (XYZTC) image and trying to write out all the 3D
>>>>>>>>>>>>>>>>>>> timepoints for the first channel.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Kishore
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Fri, Oct 12, 2012 at 12:19 PM, Mark Hiner <
>>>>>>>>>>>>>>>>>>> hinerm at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hi Kishore,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>  I helped develop the bf-itk-pipe plugin and am
>>>>>>>>>>>>>>>>>>>> investigating your issue right now.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>  What you're doing seems reasonable, so I'm trying to
>>>>>>>>>>>>>>>>>>>> determine if there's a bug in the plugin. My goal is to have some working
>>>>>>>>>>>>>>>>>>>> example code for you (and a fix if necessary) by the end of the day.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thank you for using our plugin! I hope we can get it
>>>>>>>>>>>>>>>>>>>> working for you soon.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> - Mark
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Fri, Oct 12, 2012 at 10:54 AM, Kishore Mosaliganti <
>>>>>>>>>>>>>>>>>>>> kishoreraom at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Dear ITK and OME users,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> To further elaborate, I figured that this has nothing
>>>>>>>>>>>>>>>>>>>>> to do with the itkStreamingImageFilter.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Instead, the itkImageFileReader seems to be extracting
>>>>>>>>>>>>>>>>>>>>> the same image region although I update the requested region in the
>>>>>>>>>>>>>>>>>>>>> BioformatsImageIO class :;
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>     itk::BioFormatsImageIO::Pointer io =
>>>>>>>>>>>>>>>>>>>>> itk::BioFormatsImageIO::New();
>>>>>>>>>>>>>>>>>>>>>     io->SetIORegion( region ); // doesn't make a
>>>>>>>>>>>>>>>>>>>>> difference in terms of image data extracted. Its always from the beginning.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>     ReaderType::Pointer reader = ReaderType::New();
>>>>>>>>>>>>>>>>>>>>>     reader->SetFileName(argv[1]);
>>>>>>>>>>>>>>>>>>>>>     reader->SetImageIO(io);
>>>>>>>>>>>>>>>>>>>>>     reader->Update();
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> So, my general question is whether the
>>>>>>>>>>>>>>>>>>>>> BioformatsImageIO expects the whole LSM image to be loaded into memory
>>>>>>>>>>>>>>>>>>>>> before writing it out? I would like to specify small image regions since my
>>>>>>>>>>>>>>>>>>>>> LSM is too large to be fully loaded into memory. How else can I stream data
>>>>>>>>>>>>>>>>>>>>> from large LSM?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Kishore
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Fri, Oct 12, 2012 at 8:53 AM, Kishore Mosaliganti <
>>>>>>>>>>>>>>>>>>>>> kishoreraom at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I am trying to use bioformats and itk. I downloaded
>>>>>>>>>>>>>>>>>>>>>> and compiled bf-itk-pipe using cmake by linking against ITK 3.2:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> https://github.com/openmicroscopy/bioformats/tree/develop/components/native/bf-itk-pipe
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I tested out itkBFImageInfo and
>>>>>>>>>>>>>>>>>>>>>> ./itkRGBBioFormatsImageIOTest on a few simple LSM images. It works great
>>>>>>>>>>>>>>>>>>>>>> and writes out all the associated metadata and pixel data for the first
>>>>>>>>>>>>>>>>>>>>>> timepoint.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> My LSM microscopy image is 5D. It is
>>>>>>>>>>>>>>>>>>>>>> X-Y-Z-Time-Channel. I am interested in extracting individual timepoints
>>>>>>>>>>>>>>>>>>>>>> from the LSM file for processing. The filter seems to be using
>>>>>>>>>>>>>>>>>>>>>> itkStreamingImageFilter. I set the NumberOfStreams to 10 but that still
>>>>>>>>>>>>>>>>>>>>>> seems to still write out only the first image.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> How do I set the output requestion region for an
>>>>>>>>>>>>>>>>>>>>>> individual timepoint?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thank you,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Kishore
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>>>>>> ome-users mailing list
>>>>>>>>>>>>>>>>>>>>> ome-users at lists.openmicroscopy.org.uk
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> ome-users mailing list
>>>>>>>>>>>>> ome-users at lists.openmicroscopy.org.uk
>>>>>>>>>>>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20130523/a4acd718/attachment.html>

More information about the ome-users mailing list