[ome-users] convert .lif file with bfconvert

[Mark Hiner]

Hi Michael,

So far I've used "File > Open". Sorry.
>

No need to apologize! This is a common misunderstanding. When you use "File
> Open", your .ome.tiff gets picked up by the ImageJ TIFF reader instead of
Bio-Formats. So that's why metadata is missing. This is actually something
we're improving with the ImageJ2 release this week: "File > Open" will be
able to work with Bio-Formats automatically.

I just tried "File > Import > Bio-Formats" and it works as well.
>

This made me suspect that there was something going wrong on my end. I
should have asked which version of Bio-Formats you were using bfconvert
from.. I was using it from v5.0.1. I updated to v5.0.2 this morning and
tried bfconvert, and now my output .ome.tiffs open just fine in Fiji's
Bio-Formats Importer. So I'm sorry for any misguidance there.

Anyway, as for the itkSCIFIOImageIO use, I was still getting
OutOfMemoryErrors. So you'll want to increase the amount of memory Java can
use. By default, the ImageIO creates a JVM with 256m. You can override this
by setting a "JAVA_FLAGS" environment variable. Once I went over 3g, I was
actually able to convert your data appropriately, finally! To increase
available memory, I used the command:

export JAVA_FLAGS=-Xmx3400m

Also! I forgot there were some important considerations with other flags to
use with the itkSCIFIOImageIO:

   - use "-d 5". I know your data is 3D + channels, but as an .ome.tiff it
   will be considered 5D by the imageio and the wrong dimensions could get
   truncated if you set -d less than 5.
   - If necessary, set pixel type with "-t". The default is unsigned short
   (an ITK pixel type, which is equivalent to Bio-Formats's uint16). Your data
   is uint16 so there's no need to use this flag, just wanted you to be aware
   of it.

So, the command that worked for me was:

/SCIFIOTestDriver itkSCIFIOImageIOTest
bfconvert_output/output_series_0.ome.tiff test.ome.tiff -w -d 5

Let me know if you have any more problems,

Mark


On Sun, Jun 1, 2014 at 1:19 AM, Michael Meuli <michael.meuli at gmail.com>
wrote:

> Hi Mark,
>
> I just tried "File > Import > Bio-Formats" and it works as well.
> If I select one of the files produced with ...%s.ome.tiff in the first
> open dialog box and then Hyperstack in the "Bio-Formats Import
> Options", I can open any of the series files.
> If I separate on of these files into another directory and try to open
> it the same way, Fiji crashes.
> I can also open the file containing all the series, but made some
> coffee in between.
> So far I've used "File > Open". Sorry. "File > Open" also works for
> the separated file, in case this matters, but shows them in gray and
> plane by plane (no channell selection).
>
> Thanks,
> Michael
>
>
> On 1 June 2014 03:08, Mark Hiner <hiner at wisc.edu> wrote:
> > Hi Michael,
> >
> >
> >> I can import the bfconvert output into Fiji with no problem.
> >
> >
> > I just wanted to confirm, you're using "File > Import > Bio-Formats" or
> the
> > Bio-Formats importer in Fiji, correct? It definitely didn't work for me
> when
> > I tried yesterday. I can check next week to see if I was using an
> outdated
> > Bio-Formats or something..
> >
> > Thanks,
> > Mark
> >
> >
> > On Sat, May 31, 2014 at 7:52 AM, Michael Meuli <michael.meuli at gmail.com>
> > wrote:
> >>
> >> Hi Mark
> >>
> >> I can import the bfconvert output into Fiji with no problem.
> >> (If I do "./bfconvert /path/to/wt-dead-A.lif /path/to/output.ome.tiff"
> or
> >> "./bfconvert /path/to/wt-dead-A.lif /path/to/output_series_%s.ome.tiff"
> >> I can open the file(s) with Fiji.)
> >> If I open the .lif file with Fiji and export one series to ome.tiff I
> >> can read the output with ./SCIFIOTestDriver itkSCIFIOImageIOTest ...
> >> whereas I'm not able to read the output of bfconvert.
> >>
> >> Thanks a lot and best regards
> >> Michael
> >>
> >>
> >>
> >> On 30 May 2014 16:18, Mark Hiner <hinerm at gmail.com> wrote:
> >> > Hi Michael,
> >> >
> >> >> "./SCIFIOTestDriver
> >> >> itkSCIFIOImageIOTest/path/to/output_series_0.ome.tiff
> >> >> /path/to/scifio-test.ome.tiff"
> >> >
> >> > Just a reminder, since you're writing ome.tiff you should have a "-w"
> >> > flag
> >> > at the end of this command. It sounds like you were using -w, just
> >> > wanted to
> >> > clarify for anyone else reading this thread.
> >> >
> >> >> Exception in thread "main" java.lang.OutOfMemoryError: Java heap
> space
> >> >
> >> >  There are two problems here. First, it seems that in SCIFIOImageIO
> the
> >> > use
> >> > of Bio-Formats is overly aggressive in pulling in all the series, thus
> >> > the
> >> > OutOfMemoryError. If you physically isolate a single series (e.g.
> delete
> >> > or
> >> > move to another directory series 1-19) the application will run, but
> we
> >> > get
> >> > another exception (at least on my end):
> >> >
> >> > ./SCIFIOTestDriver itkSCIFIOImageIOTest output_series_0.ome.tiff
> >> > scifio-test.ome.tiff -w
> >> >
> >> > reader->GetUseStreaming(): 1
> >> > done checking streaming usage
> >> > ITK test driver caught an ITK exception:
> >> >
> >> > itk::ExceptionObject (0x7ff8838b6ae8)
> >> > Location: "unknown"
> >> > File:
> >> >
> >> >
> /Users/mhiner/loci/ITK/build/Modules/Remote/SCIFIO/src/itkSCIFIOImageIO.cxx
> >> > Line: 180
> >> > Description: itk::ERROR: SCIFIOImageIO(0x7ff8838b4f00): SCIFIOImageIO
> >> > exited
> >> > abnormally. Creating new reader for output_series_0.ome.tiff
> >> >
> >> > log4j:WARN No appenders could be found for logger
> >> > (loci.common.services.ServiceFactory).
> >> > log4j:WARN Please initialize the log4j system properly.
> >> > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
> for
> >> > more info.
> >> > Exception in thread "main" loci.formats.FormatException: Unmatched
> UUID:
> >> > urn:uuid:b72fe97b-02c9-4029-a876-ca2bae46cbac
> >> >     at loci.formats.in.OMETiffReader.initFile(OMETiffReader.java:456)
> >> >
> >> >     at loci.formats.FormatReader.setId(FormatReader.java:1244)
> >> >     at loci.formats.ImageReader.setId(ImageReader.java:727)
> >> >     at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:529)
> >> >     at loci.formats.ChannelFiller.setId(ChannelFiller.java:263)
> >> >     at
> >> > loci.scifio.itk.SCIFIOITKBridge.createReader(SCIFIOITKBridge.java:617)
> >> >     at
> >> >
> loci.scifio.itk.SCIFIOITKBridge.readImageInfo(SCIFIOITKBridge.java:268)
> >> >     at
> >> >
> loci.scifio.itk.SCIFIOITKBridge.executeCommand(SCIFIOITKBridge.java:133)
> >> >
> >> >     at
> >> >
> loci.scifio.itk.SCIFIOITKBridge.executeCommand(SCIFIOITKBridge.java:108)
> >> >     at
> >> > loci.scifio.itk.SCIFIOITKBridge.waitForInput(SCIFIOITKBridge.java:83)
> >> >     at
> >> >
> loci.scifio.itk.SCIFIOITKBridge.executeCommand(SCIFIOITKBridge.java:163)
> >> >     at loci.scifio.itk.SCIFIOITKBridge.main(SCIFIOITKBridge.java:688)
> >> >
> >> > I also get this exception when importing with Bio-Formats into Fiji.
> >> >
> >> > As a side note, there are flags for itkSCIFIOImageIOTest that, in
> >> > theory,
> >> > should help the out of memory side:
> >> >
> >> > "-s <n1, n2>" should limit the series read.
> >> > "-v <n>" should divide the data into divisions, which might mitigate
> the
> >> > risk of OutOfMemoryErrors.
> >> >
> >> > But neither of these will help the UUID exception. Also, I still got
> >> > OutOfMemoryErrors with these, so it was still trying to initialize
> using
> >> > all
> >> > the series. So I created a new issue in the SCIFIO-ImageIO tracker.
> >> >
> >> > Anyway, try importing the bfconvert output into Fiji with Bio-Formats
> >> > and
> >> > let me know if you don't get the UUID exception. If you do, then I
> think
> >> > the
> >> > bfconvert output is just invalid..
> >> >
> >> > Regards,
> >> > Mark
> >> >
> >> >
> >> > On Thu, May 29, 2014 at 12:17 PM, Michael Meuli
> >> > <michael.meuli at gmail.com>
> >> > wrote:
> >> >>
> >> >> Hi Mark,
> >> >>
> >> >> Thanks for the quick response. I can see I was using
> >> >> itkSCIFIOImageIOTest the wrong way and it doesn't make sense to use
> it
> >> >> like this.
> >> >> If I use the -w flag I get an error in all cases but at the moment I
> >> >> don't care as my aim now is to read ome.tiff files with itk.
> >> >> I've written a small program (mainly by copy paste from examples /
> >> >> https://github.com/michaelmeuli/bactelize) which so far reads the
> >> >> multi-channel-z-series.ome.tif file, does a maximum intensity
> >> >> projection of one channel and writes a 2D image and it seems to work
> >> >> with the multi-channel-z-series.ome.tif but not with the files
> >> >> converted with bfconvert.
> >> >>
> >> >> Expressed with itkSCIFIOImageIOTest:
> >> >> "./SCIFIOTestDriver itkSCIFIOImageIOTest
> >> >> /path/to/multi-channel-z-series.ome.tif
> /path/to/scifio-test.ome.tiff"
> >> >> doesn't give an error and seems to produce a tiff file of the first
> >> >> z-plane, but
> >> >> "./SCIFIOTestDriver itkSCIFIOImageIOTest
> >> >> /path/to/output_series_0.ome.tiff /path/to/scifio-test.ome.tiff"
> gives
> >> >> an error:
> >> >>
> >> >> reader->GetUseStreaming(): 1
> >> >> done checking streaming usage
> >> >> ITK test driver caught an ITK exception:
> >> >>
> >> >> itk::ExceptionObject (0x2f87680)
> >> >> Location: "unknown"
> >> >> File:
> >> >>
> >> >>
> /home/michael/Colocalization/ITK-Release-build/Modules/Remote/SCIFIO/src/itkSCIFIOImageIO.cxx
> >> >> Line: 180
> >> >> Description: itk::ERROR: SCIFIOImageIO(0x2f78970): SCIFIOImageIO
> >> >> exited abnormally. Creating new reader for
> >> >>
> >> >>
> >> >>
> /home/michael/bioimage/ome/data/wt-dead-A-ome-tiff/output_series_0.ome.tiff
> >> >> log4j:WARN No appenders could be found for logger
> >> >> (loci.common.services.ServiceFactory).
> >> >> log4j:WARN Please initialize the log4j system properly.
> >> >> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
> >> >> for more info.
> >> >> Exception in thread "main" java.lang.OutOfMemoryError: Java heap
> space
> >> >> at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:39)
> >> >> at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
> >> >> at
> >> >>
> >> >>
> loci.common.NIOByteBufferProvider.allocateDirect(NIOByteBufferProvider.java:132)
> >> >> at
> >> >>
> >> >>
> loci.common.NIOByteBufferProvider.allocate(NIOByteBufferProvider.java:118)
> >> >> at loci.common.NIOFileHandle.buffer(NIOFileHandle.java:532)
> >> >> at loci.common.NIOFileHandle.seek(NIOFileHandle.java:254)
> >> >> at
> >> >>
> >> >>
> loci.common.RandomAccessInputStream.seek(RandomAccessInputStream.java:140)
> >> >> at loci.formats.tiff.TiffParser.getIFDOffsets(TiffParser.java:308)
> >> >> at loci.formats.tiff.TiffParser.getIFDs(TiffParser.java:226)
> >> >> at
> >> >>
> loci.formats.in.MinimalTiffReader.initFile(MinimalTiffReader.java:426)
> >> >> at loci.formats.FormatReader.setId(FormatReader.java:1244)
> >> >> at loci.formats.in.OMETiffReader.initFile(OMETiffReader.java:704)
> >> >> at loci.formats.FormatReader.setId(FormatReader.java:1244)
> >> >> at loci.formats.ImageReader.setId(ImageReader.java:727)
> >> >> at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:529)
> >> >> at loci.formats.ChannelFiller.setId(ChannelFiller.java:263)
> >> >> at
> >> >>
> loci.scifio.itk.SCIFIOITKBridge.createReader(SCIFIOITKBridge.java:622)
> >> >> at
> >> >>
> loci.scifio.itk.SCIFIOITKBridge.readImageInfo(SCIFIOITKBridge.java:266)
> >> >> at
> >> >>
> >> >>
> loci.scifio.itk.SCIFIOITKBridge.executeCommand(SCIFIOITKBridge.java:131)
> >> >> at
> >> >>
> >> >>
> loci.scifio.itk.SCIFIOITKBridge.executeCommand(SCIFIOITKBridge.java:108)
> >> >> at
> >> >> loci.scifio.itk.SCIFIOITKBridge.waitForInput(SCIFIOITKBridge.java:83)
> >> >> at
> >> >>
> >> >>
> loci.scifio.itk.SCIFIOITKBridge.executeCommand(SCIFIOITKBridge.java:161)
> >> >> at loci.scifio.itk.SCIFIOITKBridge.main(SCIFIOITKBridge.java:693)
> >> >>
> >> >> Best regards
> >> >> Michael Meuli
> >> >>
> >> >>
> >> >>
> >> >> On 29 May 2014 17:41, Mark Hiner <hiner at wisc.edu> wrote:
> >> >> > Hi Michael,
> >> >> >
> >> >> >
> >> >> >> The problem is that ./SCIFIOTestDriver itkSCIFIOImageIOTest
> >> >> >> /path/to/output_series_0.ome.tiff /path/to/scifiotest.ome.tiff
> >> >> >> (and my little program) throws an exception.
> >> >> > ...
> >> >> >
> >> >> >> How do I use bfconvert the correct way for my case?
> >> >> >
> >> >> > You used bfconvert correctly to convert your dataset. Just so it's
> >> >> > clear, if
> >> >> > your goal was to produce an OME-TIFF:
> >> >> >
> >> >> >> /SCIFIOTestDriver itkSCIFIOImageIOTest
> >> >> >> /path/to/output_series_0.ome.tiff
> >> >> >> /path/to/scifiotest.ome.tiff
> >> >> >
> >> >> > would be unnecessary, as you already have the OME-TIFF output from
> >> >> > bfconvert. The itkSCIFIOImageIOTest is an analogous tool to
> bfconvert
> >> >> > written in the Bio-Formats C++ framework. (although comparing the
> >> >> > outputs of
> >> >> > the itkSCIFIOImageIOTest conversion would be a good way to check
> for
> >> >> > errors
> >> >> > in the conversion process)
> >> >> >
> >> >> > That said, I believe your conversion failed because the output was
> >> >> > picked up
> >> >> > by the ITK TIFF ImageIO, which I believe only supports 3D data. The
> >> >> > itkSCIFIOImageIOTest utility does basically no automation - you
> have
> >> >> > to
> >> >> > tell
> >> >> > it exactly how to run, and because the SCIFIOImageIO is an external
> >> >> > module
> >> >> > it is not prioritized by the ITK format detection framework over
> >> >> > internal
> >> >> > modules (I'm not sure it's part of the automatic detection
> framework
> >> >> > at
> >> >> > all,
> >> >> > actually). So you need to force the use of the SCIFIOImageIO for
> >> >> > writing.
> >> >> >
> >> >> > If you run:
> >> >> >
> >> >> >  /SCIFIOTestDriver itkSCIFIOImageIOTest
> >> >> >
> >> >> > with no arguments from the command line, it will print the usage +
> >> >> > available
> >> >> > flags. The flag you wanted here is "-w"  to write with the
> >> >> > SCIFIOImageIO
> >> >> > (which will use Bio-Formats, and thus the OME-TIFF writer).
> >> >> >
> >> >> > So the final syntax is:
> >> >> >
> >> >> > /SCIFIOTestDriver itkSCIFIOImageIOTest
> >> >> > /path/to/output_series_0.ome.tiff
> >> >> > /path/to/scifiotest.ome.tiff -w
> >> >> >
> >> >> > which worked for me locally.
> >> >> >
> >> >> > Let me know if you have any other questions or issues!
> >> >> >
> >> >> > Regards,
> >> >> > Mark
> >> >> >
> >> >> > On Thu, May 29, 2014 at 4:40 AM, Michael Meuli
> >> >> > <michael.meuli at gmail.com>
> >> >> > wrote:
> >> >> >>
> >> >> >> ./SCIFIOTestDriver itkSCIFIOImageIOTest
> >> >> >> /path/to/output_series_0.ome.tiff /path/to/scifiotest.ome.tiff
> >> >> >
> >> >> >
> >> >> >
> >> >
> >> >
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20140602/5ccd933d/attachment.html>