[ome-users] bioformats and ITK

[Joe Ping-Lin Hsiao]

I got it working.
My local repository doesn't have the new variable SCIFIO_SEP, and it still
uses ":" for separator. I change it to ";" and it works. The program also
works without SCIFIO_PATH being set. So the separator fix should resolve
this issue.

Thanks for you help!
Joe


On Thu, May 23, 2013 at 9:52 AM, Joe Ping-Lin Hsiao <phsiao at cs.unc.edu>wrote:

> Mark,
>
> Adding the SCIFIO_PATH environment variable doesn't work for me. I
> experimented with 'build/lib/jars' and 'build/lib/jars/', but I still have
> the same error.
> I can wait for the bug fix in ITK to get approved and try again.
>
> Thanks,
> Joe
>
>
> On Wed, May 22, 2013 at 3:00 PM, Mark Hiner <hiner at wisc.edu> wrote:
>
>> Hi Joe,
>>
>>  Just wanted to send you an update... I eventually realized it wasn't a
>> Debug/Release issue, but rather a classpath separator issue. The Linux/OSX
>> separator was hard-coded in the SCIFIO ImageIO.
>>
>>  So the scifio-imageio <https://github.com/scifio/scifio-imageio>repository has been updated to choose the correct separator for OSX or
>> Windows. I also filed this bug fix
>> <http://review.source.kitware.com/#/c/11375/>updating ITK to download
>> the updated scifio-imageio commit.
>>
>>  Also, while looking through the code, I had forgotten that the JVM
>> classpath (where those jars need to be) isn't pulled from the PATH... it's
>> taken from a special environment variable: SCIFIO_PATH. If you set
>> SCIFIO_PATH to the directory containing the itk-bridge jar and
>> loci_tools.jar (build/lib/jars) it should work without the update. (*
>> should* work..)
>>
>>  Note that the SCIFIO_PATH is prepended onto each jar dependency.
>>
>> Anyway, I hope this resolves everything for you. If you run into more
>> trouble, let me know.
>>
>> Thanks,
>> Mark
>>
>>
>> On Fri, May 17, 2013 at 3:39 PM, Mark Hiner <hiner at wisc.edu> wrote:
>>
>>> Joe,
>>>
>>>  That's great! It's almost certainly an issue of the jars being put in
>>> the wrong place because of the Debug/Release addition on Windows.
>>>
>>>  I confirmed that it doesn't work on Windows :-p I think it's in the
>>> DownloadSCIFIO.cmake.in which downloads to ${jarsDownloadDirectory}...
>>> but the jars are installed from
>>> ${CMAKE_LIBRARY_OUTPUT_DIRECTORY}/${jarsDownloadDirectory} which I'm
>>> assuming points to lib/Debug/jars, which doesn't exist.
>>>
>>> Unfortunately I don't have time to verify that fixes things today. I'll
>>> try to test it on the weekend or Monday.
>>>
>>>
>>> On Fri, May 17, 2013 at 2:29 PM, Joe Ping-Lin Hsiao <phsiao at cs.unc.edu>wrote:
>>>
>>>> Mark,
>>>>
>>>> Just want to let you know that I repeated the same steps on my macbook
>>>> with Mac OSX 10.8.3, and program works fine.
>>>>
>>>> Joe
>>>>
>>>>
>>>> On Fri, May 17, 2013 at 10:45 AM, Mark Hiner <hinerm at gmail.com> wrote:
>>>>
>>>>> Joe,
>>>>>
>>>>>  That's odd.. not sure why manually adding them to the classpath
>>>>> didn't work. I will test locally on a Windows 7 box and let you know if I
>>>>> can reproduce and/or resolve the issue.
>>>>>
>>>>> Thanks for reporting it, hopefully we can get it working for you soon.
>>>>>
>>>>> - Mark
>>>>>
>>>>>
>>>>> On Thu, May 16, 2013 at 4:30 PM, Joe Ping-Lin Hsiao <phsiao at cs.unc.edu
>>>>> > wrote:
>>>>>
>>>>>> Mark,
>>>>>>
>>>>>> Yes, I was using the command "ITKIOSCIFIOTestDriver
>>>>>> itkSCIFIOImageIOTest inputfile outputfile".
>>>>>>
>>>>>> I am on Windows 7. The two jars (loci_tools.jar and
>>>>>> scifio-itk-bridge-1.0.0.jar) are in ITK/lib/jars.
>>>>>>
>>>>>> I manually added the path to classpath, and my classpath is now:
>>>>>> CLASSPATH=.;C:\Program Files
>>>>>> (x86)\Java\jre6\lib\ext\QTJava.zip;D:\Joe\dev\ITK\lib\jars
>>>>>>
>>>>>> But I still have no luck.
>>>>>>
>>>>>> After running the command, I first get a dialog of
>>>>>>  "Could not find the main class: loci.scifio.itk.SCIFIOITKBridge.
>>>>>> Program will exit."
>>>>>> Please see attached image.
>>>>>>
>>>>>> And there are also error messages in the console:
>>>>>>
>>>>>> D:\Joe\dev\ITK\bin\Release>ITKIOSCIFIOTestDriver itkSCIFIOImageIOTest
>>>>>> C:\Users\phsiao\Desktop\itk-scifio-build\Release\
>>>>>> chZT.lsm C:\Users\phsiao\Desktop\itk-scifio-build\Release\abc.tif
>>>>>> reader->GetUseStreaming(): 1
>>>>>> done checking streaming usage
>>>>>> itk::ExceptionObject (0000000000A4EF10)
>>>>>> Location: "unknown"
>>>>>> File: .\itkSCIFIOImageIO.cxx
>>>>>> Line: 172
>>>>>> Description: itk::ERROR: SCIFIOImageIO(0000000002A5C080):
>>>>>> SCIFIOImageIO exited abnormally. java.lang.NoClassDefFoundErr
>>>>>> r: loci/scifio/itk/SCIFIOITKBridge
>>>>>> Caused by: java.lang.ClassNotFoundException:
>>>>>> loci.scifio.itk.SCIFIOITKBridge
>>>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>>>>>>         at
>>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>>>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>>>>>> Exception in thread "main"
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, May 16, 2013 at 4:46 PM, Mark Hiner <hiner at wisc.edu> wrote:
>>>>>>
>>>>>>> Hi Joe,
>>>>>>>
>>>>>>>  Hmm.. well that is the correct package specification. What was the
>>>>>>> exact command you used? I'm assuming it was something like
>>>>>>>
>>>>>>> "ITKIOSCIFIOTestDriver itkSCIFIOImageIOTest inputfile outputfile"
>>>>>>>
>>>>>>> ?
>>>>>>>
>>>>>>> Anyway it sounds like the scifio-itk-bridge jar isn't on your
>>>>>>> classpath. I just tested a clean build off ITK.git master (commit
>>>>>>> 2e056e0cee1271d3080a9bfc4f7353c1d9165ef9) and it worked for me... but
>>>>>>> that's on OSX.
>>>>>>>
>>>>>>> What's in your build/lib/jars directory? It should have 2 things:
>>>>>>> loci_tools.jar
>>>>>>> scifio-itk-bridge-1.0.0.jar
>>>>>>>
>>>>>>> It's possible that they aren't being added to the classpath
>>>>>>> correctly though, if on Windows there's another directory level (e.g. is it
>>>>>>> build/Release/lib/jars ?). So if they are in a different directory, let me
>>>>>>> know.
>>>>>>>
>>>>>>> If the jars are there, try manually adding them to your classpath
>>>>>>> for now and let me know how that goes. If they're not then it's a problem
>>>>>>> with the download script.
>>>>>>>
>>>>>>> You can download the jars manually from:
>>>>>>>
>>>>>>> http://cvs.openmicroscopy.org.uk/snapshots/bioformats/4.4.5/loci_tools.jar
>>>>>>>
>>>>>>> http://jenkins.imagej.net/view/SCIFIO/job/SCIFIOITKBridge/lastSuccessfulBuild/artifact/target/scifio-itk-bridge-1.0.0.jar
>>>>>>>
>>>>>>> That's all I can think of at the moment...
>>>>>>>
>>>>>>> - Mark
>>>>>>>
>>>>>>>
>>>>>>> On Thu, May 16, 2013 at 1:02 PM, Joe Ping-Lin Hsiao <
>>>>>>> phsiao at cs.unc.edu> wrote:
>>>>>>>
>>>>>>>> Hi Mark,
>>>>>>>>
>>>>>>>> Glad to see SCIFIO is now becoming ITK's remote module. Thanks for
>>>>>>>> your continuous working.
>>>>>>>> I did a clean git clone of the latest ITK, and configure it with
>>>>>>>>
>>>>>>>>    Fetch_SCIFIO
>>>>>>>>    BUILD_TESTING
>>>>>>>>
>>>>>>>> set to ON. Then I run ITKIOSCIFIOTestDriver.exe on an image file,
>>>>>>>> but I got the error message
>>>>>>>>
>>>>>>>> "Could not find the main class: loci.scifio.itk.SCIFIOITKBridge.
>>>>>>>> Program will exit."
>>>>>>>>
>>>>>>>> Do I miss setting some path, or it still requires Bio-Formats?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Joe
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Apr 26, 2013 at 2:43 PM, Mark Hiner <hiner at wisc.edu> wrote:
>>>>>>>>
>>>>>>>>> Hi again Joe,
>>>>>>>>>
>>>>>>>>>  I noticed a couple things that may help you: in the
>>>>>>>>> itkVectorImageSCIFIOImageIOTest you sent me (and in all the test classes I
>>>>>>>>> mentioned in my last e-mail) the PixelType and Dimension variables must be
>>>>>>>>> set carefully.
>>>>>>>>>
>>>>>>>>>  For example, with the code in the state you sent me, PixelType is
>>>>>>>>> an unsigned short (which is 16 bits) and the dimension is 3.
>>>>>>>>>
>>>>>>>>>  These settings would be incorrect for the 2chZT.lsm sample data,
>>>>>>>>> as it has an 8 bit unsigned pixel type (so PixelType should be defined as
>>>>>>>>> unsigned char). Also if you're using the SCIFIOImageIO you can typically
>>>>>>>>> set dimensions to 5.
>>>>>>>>>
>>>>>>>>>  I don't know the specifications of your .oif dataset, but the
>>>>>>>>> PixelType and dimensions may need to be adjusted for it.
>>>>>>>>>
>>>>>>>>>   Sorry if you already knew this and were updating these values!
>>>>>>>>>
>>>>>>>>>   Anyway, I've been working on improving the stability of the
>>>>>>>>> SCIFIOImageIO plugin. Using the itkSCIFIOImageIOTest I mentioned
>>>>>>>>> previously, I confirmed that I was able to read and write the 2chsZT.lsm
>>>>>>>>> file and the .oif test images I have access to.
>>>>>>>>>
>>>>>>>>>   These changes aren't in an ITK patch set yet because they
>>>>>>>>> include changes to both the c++ and java code, which will may require some
>>>>>>>>> restructuring of SCIFIOImageIO.
>>>>>>>>>
>>>>>>>>> However, if you're eager to test these changes, you could build
>>>>>>>>> off my development branches:
>>>>>>>>> 1. Clone my ITK repo. <https://github.com/hinerm/ITK> The default
>>>>>>>>> branch is what you want here.
>>>>>>>>> 2. Build ITK as normal (with tests enabled)
>>>>>>>>> 3. Clone my Bio-Formats repo<https://github.com/hinerm/bioformats>
>>>>>>>>> .
>>>>>>>>> 4. Check out the itk-bridge-fixes branch in Bio-Formats
>>>>>>>>> 5. From the top level of Bio-Formats, run "ant tools"
>>>>>>>>> 6. Copy the loci_tools.jar from ${bio-formats}/artifacts to
>>>>>>>>> ${ITK_build}/lib/jars/
>>>>>>>>>    (this will overwrite the loci_tools.jar that was downloaded
>>>>>>>>> automatically when ITK built)
>>>>>>>>>
>>>>>>>>>  At this point you can link in the ImageToVTKFilter you were using
>>>>>>>>> or run the SCIFIO ImageIO test programs included in ITK.
>>>>>>>>>
>>>>>>>>>  Whether you build these manually or not, I'll try to get a new
>>>>>>>>> official patch set released soon.
>>>>>>>>>
>>>>>>>>>  Let me know if you run into any problems.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Mark
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Apr 25, 2013 at 4:12 PM, Mark Hiner <hinerm at gmail.com>wrote:
>>>>>>>>>
>>>>>>>>>> Hi Joe,
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Here are my steps to compile the program, in case I missed
>>>>>>>>>>> something:
>>>>>>>>>>> 1. Copy the git commend from the 'Patch set 3' review page, and
>>>>>>>>>>> paste it under the master ITK source, which downloaded a bunch of files and
>>>>>>>>>>> switched me to the branch 'SCIFIOImageIO'.
>>>>>>>>>>> 2. Compile the new ITK source in CMake, with Module_ITKVtkGlue,
>>>>>>>>>>> ITK_USE_REVIEW, and BUILD_SHARED_LIBS checked.
>>>>>>>>>>> 3. Link the program with the ITK with SCIFIOImageIO.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> This looks good. Note that a Patch set 4 is now out, and I'm
>>>>>>>>>> working on Patch 5. I'm hoping to include any necessary fixes for your
>>>>>>>>>> issues in patch 5.
>>>>>>>>>>
>>>>>>>>>> So, to clarify here - you're end goal is to convert your .oif
>>>>>>>>>> dataset to .ome.tiff? Is that right? Are you specifically wanting to
>>>>>>>>>> extract a subset of the image or anything? If you're just trying to do a
>>>>>>>>>> simple conversion it may be better to use the tests that are part of the
>>>>>>>>>> ITK SCIFIOImageIO module. You can take a look at them at:
>>>>>>>>>> $ITK/Modules/IO/SCIFIO/test
>>>>>>>>>>
>>>>>>>>>> To build them, just turn the BUILD_TESTING flag on when
>>>>>>>>>> configuring cmake. The ITKIOSCIFIOTestDriver is built to $build/bin
>>>>>>>>>>
>>>>>>>>>> I've been using the itkSCIFIOImageIOTest to test image
>>>>>>>>>> conversion, for example.
>>>>>>>>>>
>>>>>>>>>> The hanging happens after the line printing "post-update".
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Looking at the script that suggests it read your dataset
>>>>>>>>>> successfully but failed to write.
>>>>>>>>>>
>>>>>>>>>> Using the ImageIOTest mentioned above I tried converting the
>>>>>>>>>> 2chsZT.lsm to an .ome.tiff. It got through half the planes and then crashed
>>>>>>>>>> (which sounds like an improvement over your report, so I'm taking that as a
>>>>>>>>>> small victory).
>>>>>>>>>>
>>>>>>>>>> There are some oddities in the ITK <-> Bio-Formats axis
>>>>>>>>>> conversion.. it looks like only 1 of the 2 channels was detected for some
>>>>>>>>>> reason. So I'll fix that in the next patch set.
>>>>>>>>>>
>>>>>>>>>> I got similar results with an .oif file locally.
>>>>>>>>>>
>>>>>>>>>> So I apologize, as I don't feel I've helped you much yet. I just
>>>>>>>>>> wanted you to know that I'm still looking at these issues. Once I'm to the
>>>>>>>>>> point that I can convert that 2chsZT.lsm and local .oif files I'll try to
>>>>>>>>>> finalize another patch and see if that fixes things for you.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Mark
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Apr 24, 2013 at 2:26 PM, Joe Ping-Lin Hsiao <
>>>>>>>>>> phsiao at cs.unc.edu> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Mark,
>>>>>>>>>>>
>>>>>>>>>>>     Thanks for the explanation. I have tried switching to
>>>>>>>>>>> SCIFIOImageIO in the cpp code, but unfortunately the program still hangs
>>>>>>>>>>> during writing files out (with a .oif input). I have attached the source
>>>>>>>>>>> code. What I did was just replacing itk::BioFormatsImageIO with
>>>>>>>>>>> itk::SCIFIOImageIO and a change of the included header file.
>>>>>>>>>>>
>>>>>>>>>>> Here are my steps to compile the program, in case I missed
>>>>>>>>>>> something:
>>>>>>>>>>> 1. Copy the git commend from the 'Patch set 3' review page, and
>>>>>>>>>>> paste it under the master ITK source, which downloaded a bunch of files and
>>>>>>>>>>> switched me to the branch 'SCIFIOImageIO'.
>>>>>>>>>>> 2. Compile the new ITK source in CMake, with Module_ITKVtkGlue,
>>>>>>>>>>> ITK_USE_REVIEW, and BUILD_SHARED_LIBS checked.
>>>>>>>>>>> 3. Link the program with the ITK with SCIFIOImageIO.
>>>>>>>>>>>
>>>>>>>>>>> The hanging happens after the line printing "post-update".
>>>>>>>>>>> By the way, I am using VS 2008 sp1 on Windows 7 64-bit.
>>>>>>>>>>>
>>>>>>>>>>> I also tested the latest bf-itk-pipe code. The hanging happens
>>>>>>>>>>> after the line printing "pre-update" in this case.
>>>>>>>>>>> Note that I had to link bf-itk-pipe against the SCIFIOImageIO
>>>>>>>>>>> ITK otherwise there'd be a link error if linked against the master ITK.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Joe
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Apr 23, 2013 at 4:11 PM, Mark Hiner <hiner at wisc.edu>wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Joe,
>>>>>>>>>>>>
>>>>>>>>>>>>  First of all, in case you are using the bf-itk-pipes plugin,
>>>>>>>>>>>> you should know that we currently have a patch under review<http://review.source.kitware.com/9450>by the ITK developers. This patch adds a true Bio-Formats (SCIFIO) ITK
>>>>>>>>>>>> ImageIO, which will be available by default in ITK.
>>>>>>>>>>>>
>>>>>>>>>>>>  This means that the old bf-itk-pipes plugin won't be
>>>>>>>>>>>> supported, as all development efforts are going towards the SCIFIOImageIO.
>>>>>>>>>>>>
>>>>>>>>>>>>  As this code is still under review, there are issues that I'm
>>>>>>>>>>>> currently working on resolving.
>>>>>>>>>>>>
>>>>>>>>>>>>  So first I'd ask that you check out the latest patch code
>>>>>>>>>>>> using the links on that review site and rework the splitting code to use
>>>>>>>>>>>> SCIFIOImageIO<http://review.source.kitware.com/#/c/9450/3/Modules/IO/SCIFIO/include/itkSCIFIOImageIO.h>instead of the BioFormatsImageIO. You can also look at the test
>>>>>>>>>>>> directory<http://review.source.kitware.com/#/c/9450/3/Modules/IO/SCIFIO/test/itkSCIFIOImageIOTest.cxx>for more sample code.
>>>>>>>>>>>>
>>>>>>>>>>>>  If you have any trouble updating your code, please let me know
>>>>>>>>>>>> and I'll be happy to help.
>>>>>>>>>>>>
>>>>>>>>>>>>  As to your actual issue of the .oif image hanging, I actually
>>>>>>>>>>>> just fixed a bug in the last week regarding hanging when plane size is
>>>>>>>>>>>> miscalculated by the plugin, so with any luck that was your issue. :)
>>>>>>>>>>>>
>>>>>>>>>>>>  If the latest SCIFIOImageIO still gives you problems with your
>>>>>>>>>>>> .oif dataset though let me know and I'll investigate.
>>>>>>>>>>>>
>>>>>>>>>>>>  As for the 2chZT data, that sounds unrelated to the hanging so
>>>>>>>>>>>> I'll take a look at it locally when I get a chance.
>>>>>>>>>>>>
>>>>>>>>>>>>  Thanks for using our software and reporting these issues!
>>>>>>>>>>>>
>>>>>>>>>>>> - Mark
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Apr 23, 2013 at 2:40 PM, Joe Ping-Lin Hsiao <
>>>>>>>>>>>> phsiao at cs.unc.edu> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Mark,
>>>>>>>>>>>>>      I tested your code with the .lsm download from
>>>>>>>>>>>>> http://www.loci.wisc.edu/files/software/data/2chZT.zip. The
>>>>>>>>>>>>> program just crashed with the error message
>>>>>>>>>>>>>
>>>>>>>>>>>>> "This application has requested the Runtime to terminate it in
>>>>>>>>>>>>> an unusual way.
>>>>>>>>>>>>> Please contact the application's support team for more
>>>>>>>>>>>>> information."
>>>>>>>>>>>>>
>>>>>>>>>>>>> If I tested with my own 5D .oif image collection, the
>>>>>>>>>>>>> execution hangs at 'extractor->Update()' and it does not even use any CPU.
>>>>>>>>>>>>> Any idea how to fix it?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Joe
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Oct 19, 2012 at 12:41 PM, Mark Hiner <hiner at wisc.edu>wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Kishore,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>  I updated your test<https://gist.github.com/3900888/81064cf96fb06b4575a64e81481fda0936cd9056>to use an ExtractImageFilter in the IO pipeline. When a Reader's output is
>>>>>>>>>>>>>> piped to this Filter, the reader->Update() call only reads the extracted
>>>>>>>>>>>>>> region.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>  When I tested on the
>>>>>>>>>>>>>> ConfocalMemCitrine516LeftEar18hpfAug17_2012_Fish1.lsm test image you
>>>>>>>>>>>>>> uploaded, it correctly splits out the stacks for each time point. And when
>>>>>>>>>>>>>> I tested on a 9GB image it processed each time point without reading the
>>>>>>>>>>>>>> entire file.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>  So I'm hoping this solves your issue. If not, just let me
>>>>>>>>>>>>>> know how it's failing for you and I'll try again!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Mark
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Oct 17, 2012 at 2:34 PM, Mark Hiner <hiner at wisc.edu>wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> One thought is that, as the itkBioFormatsImageIO is a
>>>>>>>>>>>>>>> StreamingImageIOBase<http://www.itk.org/Doxygen320/html/classitk_1_1StreamingImageIOBase.html>,
>>>>>>>>>>>>>>> you could just set the desired region (as you were) and use
>>>>>>>>>>>>>>> *its* Read and Write methods instead of wrapping it in a
>>>>>>>>>>>>>>> Reader or a Writer.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'll try to put a code example up using this approach. It
>>>>>>>>>>>>>>> would be a good test case for the bf-itk-pipe plug-in anyway.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - Mark
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Oct 17, 2012 at 1:56 PM, Mark Hiner <
>>>>>>>>>>>>>>> hinerm at gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Kishore,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  Just wanted to let you know that I'm still looking at your
>>>>>>>>>>>>>>>> issue today. I updated the gist<https://gist.github.com/3900888/7aab8fddb38af87ea98cd49dd921524ace6430c8>to reflect the topics discussed in this e-mail, using the region setting
>>>>>>>>>>>>>>>> method calls instead of pipelining, and added some (hopefully) helpful
>>>>>>>>>>>>>>>> print statements to see what's going on with the regions.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  From what I can tell there are two separate issues
>>>>>>>>>>>>>>>> preventing your code from writing the desired regions.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  First of all, the IORegion needs to be set on the Writer,
>>>>>>>>>>>>>>>> for example:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>     writer->SetImageIO(io);
>>>>>>>>>>>>>>>>     writer->SetInput(reader->GetOutput());
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>     writer->SetIORegion(region);
>>>>>>>>>>>>>>>>     writer->Write();
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  In the original code you sent to me, that wasn't happening
>>>>>>>>>>>>>>>> and it was defaulting to the first 3D region and thus the first time point.
>>>>>>>>>>>>>>>> I mentioned this, without providing a code example, in a past e-mail - so
>>>>>>>>>>>>>>>> apologies if you've already updated your code!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  Note that according to the itkImageFileWriter API<http://www.itk.org/Doxygen/html/classitk_1_1ImageFileWriter.html#ab2cba65f716aa75ec962171ee0b46fe3>Write() is a special form of Update() that respects the IORegion. However,
>>>>>>>>>>>>>>>> in practice, for me it still just writes the largest possible region
>>>>>>>>>>>>>>>> instead of what was specified. So that could suggest a bug in the
>>>>>>>>>>>>>>>> bf-itk-pipes plug-in.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  The second issue is that even after manually specifying
>>>>>>>>>>>>>>>> the IO regions, the Reader output still has the largest possible dimensions
>>>>>>>>>>>>>>>> (T=2) so the current gist test ends up creating 2 full copies of the image.
>>>>>>>>>>>>>>>> I think this was always happening at the Reader, but then it was truncated
>>>>>>>>>>>>>>>> to the first time point you saw by the Writer. Anyway, this implies that
>>>>>>>>>>>>>>>> the full image is still being read, despite the region specifications.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  I think this is a side effect of setting using the
>>>>>>>>>>>>>>>> LargestPossibleRegion in the Update() method, as mentioned in the itkProcessObject
>>>>>>>>>>>>>>>> specification<http://www.itk.org/Doxygen/html/classitk_1_1ProcessObject.html#a4041fb21e9105500eee311e265691bd5>
>>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  Right now I'm going through the examples from this ITK
>>>>>>>>>>>>>>>> Users thread<http://itk-insight-users.2283740.n2.nabble.com/Memory-limits-of-ITKImage-object-td5670422.html>about streaming for a better pipelining solution. I'm also testing to see
>>>>>>>>>>>>>>>> what can be done at the ImageIO level, as when Read and Write are hit in
>>>>>>>>>>>>>>>> the itkBioFormatsImageIO, GetIORegion() is already returning the largest
>>>>>>>>>>>>>>>> possible region, instead of the specified region.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  I'll keep you updated.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> - Mark
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Oct 16, 2012 at 1:37 PM, Kishore Mosaliganti <
>>>>>>>>>>>>>>>> kishoreraom at gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>  Dear Mark,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks for the clarification. My original problem remains
>>>>>>>>>>>>>>>>> unresolved. For example, in the LSM gist on lines 104-107:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>     ReaderType::Pointer reader = ReaderType::New();
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>     reader->SetFileName(argv[1]);
>>>>>>>>>>>>>>>>>     reader->SetImageIO(io);
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>     reader->Update();
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Here, calling reader->Update() loads the entire image into
>>>>>>>>>>>>>>>>> memory. If you have a very large LSM file (several gigabytes large), then
>>>>>>>>>>>>>>>>> this code will fail since such an image cannot get into the memory.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hence, I am trying to send the read region directly into
>>>>>>>>>>>>>>>>> the reader using
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> io->SetRegion(region);
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> and then
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> reader->SetImageIO(io);
>>>>>>>>>>>>>>>>> reader->Update();
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> But this step is not giving me two different timepoints.
>>>>>>>>>>>>>>>>> It is writing out the same timepoint.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Kishore
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Tue, Oct 16, 2012 at 2:13 PM, Mark Hiner <
>>>>>>>>>>>>>>>>> hiner at wisc.edu> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hello again,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>  I created a gist for your LSM test and updated it with
>>>>>>>>>>>>>>>>>> my most recent code:
>>>>>>>>>>>>>>>>>> https://gist.github.com/3900888
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>  That example is currently working for me, in that it
>>>>>>>>>>>>>>>>>> writes out 2 images of 72 planes - one for T=0 and one for T=1. Since
>>>>>>>>>>>>>>>>>> everything in ITK seems to be pipeline-oriented, I am not sure there is a
>>>>>>>>>>>>>>>>>> solution to splitting out channels just using Setters like we were both
>>>>>>>>>>>>>>>>>> trying.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>  Instead, I followed this ROI Filter example<http://www.vtk.org/Wiki/ITK/Examples/ImageProcessing/RegionOfInterestImageFilter>and used the logic from my last e-mail.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>  I do need to correct one thing I said in my last e-mail:
>>>>>>>>>>>>>>>>>> SizeT should always be 1, as it is a relative offset to the Index value.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>  When I ran that test, both images went to grayscale..
>>>>>>>>>>>>>>>>>> I'm going to see if that's an error in the bf-itk-pipes logic, or just
>>>>>>>>>>>>>>>>>> something that needs to be set up differently in using the ITK API.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>  Let me know if you run into any more problems with this
>>>>>>>>>>>>>>>>>> code, or if you just want to discuss any part of it.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks again for using our software :)
>>>>>>>>>>>>>>>>>> - Mark
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Tue, Oct 16, 2012 at 11:00 AM, Mark Hiner <
>>>>>>>>>>>>>>>>>> hiner at wisc.edu> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi Kishore,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>  I am sorry I wasn't able to get to your issue
>>>>>>>>>>>>>>>>>>> yesterday. Looking at it today, I think there are a couple of things going
>>>>>>>>>>>>>>>>>>> on here.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>  First of all: in your test class
>>>>>>>>>>>>>>>>>>> - I consolidated a BioFormatsImageIO and the ImageIOBase.
>>>>>>>>>>>>>>>>>>> - In the region updating loop, the wrong region (dim-3
>>>>>>>>>>>>>>>>>>> instead of dim-2) was being set, and only the index is set. I suspect both
>>>>>>>>>>>>>>>>>>> the index and size need to be set.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>  I attached an updated version of your test, where in
>>>>>>>>>>>>>>>>>>> the first iteration it sets IndexT = 0, sizeT = 1. The second iteration
>>>>>>>>>>>>>>>>>>> sets IndexT = 1, sizeT = 2.  I believe those regions will split the time
>>>>>>>>>>>>>>>>>>> points as you wanted.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> But there is a second issue: although the Reader
>>>>>>>>>>>>>>>>>>> IORegion is correct (I think) the Writer is not. It is defaulting to the
>>>>>>>>>>>>>>>>>>> ITK base 3D region. When I tried setting the Writer's region to our desired
>>>>>>>>>>>>>>>>>>> region, it throws an exception because the 5D region can't be contained by
>>>>>>>>>>>>>>>>>>> what it thinks is the largest possible (3D) region. I am refreshing myself
>>>>>>>>>>>>>>>>>>> on the ITK API and once we figure out how to set that region, I think it
>>>>>>>>>>>>>>>>>>> will write the images correctly (or we can move on to the next problem).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> In your test I changed the output format to ome.tiff
>>>>>>>>>>>>>>>>>>> because I know that will use the itkBioFormatsImageIO writer code, which I
>>>>>>>>>>>>>>>>>>> know is capable of handling 5D writing.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Also, in testing I found a couple of bugs in the
>>>>>>>>>>>>>>>>>>> ITKPipesBridge code. I started a new branch to fix these and any other
>>>>>>>>>>>>>>>>>>> issues that may be discovered. If you want to use the itkBioFormatsImageIO
>>>>>>>>>>>>>>>>>>> writer code you'll have to use this new branch for now. You can find it
>>>>>>>>>>>>>>>>>>> here:
>>>>>>>>>>>>>>>>>>> https://github.com/hinerm/bioformats/commits/bf-itk-fixes
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> As always, let me know if this isn't fully addressing
>>>>>>>>>>>>>>>>>>> your issue. Otherwise, I'll let you know as soon as I have the Writer
>>>>>>>>>>>>>>>>>>> region setting properly.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>> - Mark
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Fri, Oct 12, 2012 at 11:28 AM, Kishore Mosaliganti <
>>>>>>>>>>>>>>>>>>> kishoreraom at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hi Mark,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Thank you for the email and invaluable help. The
>>>>>>>>>>>>>>>>>>>> bioformats-itk plugin is wonderful. I am attaching my script here to help
>>>>>>>>>>>>>>>>>>>> you debug. Let me know if you have any questions on my script. I am
>>>>>>>>>>>>>>>>>>>> starting with a 5D (XYZTC) image and trying to write out all the 3D
>>>>>>>>>>>>>>>>>>>> timepoints for the first channel.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Kishore
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Fri, Oct 12, 2012 at 12:19 PM, Mark Hiner <
>>>>>>>>>>>>>>>>>>>> hinerm at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi Kishore,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>  I helped develop the bf-itk-pipe plugin and am
>>>>>>>>>>>>>>>>>>>>> investigating your issue right now.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>  What you're doing seems reasonable, so I'm trying to
>>>>>>>>>>>>>>>>>>>>> determine if there's a bug in the plugin. My goal is to have some working
>>>>>>>>>>>>>>>>>>>>> example code for you (and a fix if necessary) by the end of the day.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thank you for using our plugin! I hope we can get it
>>>>>>>>>>>>>>>>>>>>> working for you soon.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> - Mark
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On Fri, Oct 12, 2012 at 10:54 AM, Kishore Mosaliganti
>>>>>>>>>>>>>>>>>>>>> <kishoreraom at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Dear ITK and OME users,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> To further elaborate, I figured that this has nothing
>>>>>>>>>>>>>>>>>>>>>> to do with the itkStreamingImageFilter.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Instead, the itkImageFileReader seems to be
>>>>>>>>>>>>>>>>>>>>>> extracting the same image region although I update the requested region in
>>>>>>>>>>>>>>>>>>>>>> the BioformatsImageIO class :;
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>     itk::BioFormatsImageIO::Pointer io =
>>>>>>>>>>>>>>>>>>>>>> itk::BioFormatsImageIO::New();
>>>>>>>>>>>>>>>>>>>>>>     io->SetIORegion( region ); // doesn't make a
>>>>>>>>>>>>>>>>>>>>>> difference in terms of image data extracted. Its always from the beginning.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>     ReaderType::Pointer reader = ReaderType::New();
>>>>>>>>>>>>>>>>>>>>>>     reader->SetFileName(argv[1]);
>>>>>>>>>>>>>>>>>>>>>>     reader->SetImageIO(io);
>>>>>>>>>>>>>>>>>>>>>>     reader->Update();
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> So, my general question is whether the
>>>>>>>>>>>>>>>>>>>>>> BioformatsImageIO expects the whole LSM image to be loaded into memory
>>>>>>>>>>>>>>>>>>>>>> before writing it out? I would like to specify small image regions since my
>>>>>>>>>>>>>>>>>>>>>> LSM is too large to be fully loaded into memory. How else can I stream data
>>>>>>>>>>>>>>>>>>>>>> from large LSM?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Kishore
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Fri, Oct 12, 2012 at 8:53 AM, Kishore Mosaliganti
>>>>>>>>>>>>>>>>>>>>>> <kishoreraom at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I am trying to use bioformats and itk. I downloaded
>>>>>>>>>>>>>>>>>>>>>>> and compiled bf-itk-pipe using cmake by linking against ITK 3.2:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> https://github.com/openmicroscopy/bioformats/tree/develop/components/native/bf-itk-pipe
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I tested out itkBFImageInfo and
>>>>>>>>>>>>>>>>>>>>>>> ./itkRGBBioFormatsImageIOTest on a few simple LSM images. It works great
>>>>>>>>>>>>>>>>>>>>>>> and writes out all the associated metadata and pixel data for the first
>>>>>>>>>>>>>>>>>>>>>>> timepoint.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> My LSM microscopy image is 5D. It is
>>>>>>>>>>>>>>>>>>>>>>> X-Y-Z-Time-Channel. I am interested in extracting individual timepoints
>>>>>>>>>>>>>>>>>>>>>>> from the LSM file for processing. The filter seems to be using
>>>>>>>>>>>>>>>>>>>>>>> itkStreamingImageFilter. I set the NumberOfStreams to 10 but that still
>>>>>>>>>>>>>>>>>>>>>>> seems to still write out only the first image.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> How do I set the output requestion region for an
>>>>>>>>>>>>>>>>>>>>>>> individual timepoint?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Thank you,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Kishore
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>>>>>>> ome-users mailing list
>>>>>>>>>>>>>>>>>>>>>> ome-users at lists.openmicroscopy.org.uk
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> ome-users mailing list
>>>>>>>>>>>>>> ome-users at lists.openmicroscopy.org.uk
>>>>>>>>>>>>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20130523/9df64c7e/attachment.html>