[ome-devel] Bio-Formats problem with AxioScan CZI files (uncompressed)

[Melissa Linkert]

Hi Niko and Kai,

> > The problematic 100 GB file consist of: 3ch, 3z, 8 scenes
> > 
> > Opening this file fails in the following way:
> > 
> > - Bio-Formats recognizes the CZI as an image sequence with 1008
> > images instead of 8 individual stitched images (one per scene).
> > - When opening one of the images in this sequence, the number of
> > planes and channels are correct, but pixel values are all ZERO, i.e
> > the entire image is black.
> > - The size of this image corresponds to the size of the full CZI,
> > not the size of a scene, further reflecting that scenes are no more
> > recognized as such.
> > - Metadata opens correctly and is attached to this mail (hoping the
> > mailing list server lets the 7z compressed file pass).
> > 
> > According to our tests the critical parameter that leads to this
> > behaviour seems to be purely the size of the CZI, as smaller files
> > with a comparable complexity in structure are opened correctly.

Thanks again for uploading the problematic file, and apologies for the
delay.

We're testing a fix for this now:

https://github.com/openmicroscopy/bioformats/pull/2305

and I'd expect the fix to be included in the 5.2.0 release.

Regards,
-Melissa

On Wed, Feb 24, 2016 at 12:49:46PM -0600, Melissa Linkert wrote:
> Hi Niko and Kai,
> 
> > we are required to deal with CZI data created by an Zeiss
> > AxioScan.Z1 SlideScanner:
> > 
> > * acquisition software is ZEN 2 (blue)
> > * tile-stitching was done directly after acquisition in "offline mode"
> > * results were saved as uncompressed CZI files
> > 
> > Trying to open them in Fiji using the latest Bio-Formats (5.1.8-DEV)
> > works well for relatively small files (up to 12 GB tested) but fails
> > for big files (100 GB tested).
> > 
> > The biggest working CZI consists of:
> > * 3 channels
> > * 9 z-slices
> > * 1 scene (Zeiss-speak for connected regions, i.e. tilings)
> > 
> > We tested other files containing more than one scene, they opened
> > correctly as one image per scene.
> 
> OK, good to know that it's at least not a general problem with multiple
> scenes.
> 
> > The problematic 100 GB file consist of: 3ch, 3z, 8 scenes
> > 
> > Opening this file fails in the following way:
> > 
> > - Bio-Formats recognizes the CZI as an image sequence with 1008
> > images instead of 8 individual stitched images (one per scene).
> > - When opening one of the images in this sequence, the number of
> > planes and channels are correct, but pixel values are all ZERO, i.e
> > the entire image is black.
> > - The size of this image corresponds to the size of the full CZI,
> > not the size of a scene, further reflecting that scenes are no more
> > recognized as such.
> > - Metadata opens correctly and is attached to this mail (hoping the
> > mailing list server lets the 7z compressed file pass).
> > 
> > According to our tests the critical parameter that leads to this
> > behaviour seems to be purely the size of the CZI, as smaller files
> > with a comparable complexity in structure are opened correctly.
> 
> In looking at the metadata output, it seems like the scenes are
> detected, but that each one is stored as a set of tiles (per
> "Information|Image|SizeM" near the bottom of the .txt).  My guess is
> that's what is confusing the dimensions - can you confirm that the
> working datasets show the SizeM to be 1?
> 
> > To further put this into context, we wish to use OMERO to handle the
> > massive amount of data that is created by the Slidescanner. From an
> > experimenters perspective, the way this 100 GB data set was recorded
> > is a realistic scenario.
> 
> All makes sense.
> 
> > We'd be very happy if you could help us in accomplishing this goal!
> > Please let me know if there is a convenient way that we could make
> > these datasets available to you.
> 
> If you're able to upload the data, that would certainly help us to
> reproduce and fix the problem - I know that's asking a lot given the
> size of the data though, so if not possible just let us know.  I've sent
> upload instructions in a private email to you both.
> 
> Regards,
> -Melissa
> 
> On Tue, Feb 23, 2016 at 12:55:29PM +0100, Niko Ehrenfeuchter wrote:
> > Dear all, dear Melissa,
> > 
> > we are required to deal with CZI data created by an Zeiss
> > AxioScan.Z1 SlideScanner:
> > 
> > * acquisition software is ZEN 2 (blue)
> > * tile-stitching was done directly after acquisition in "offline mode"
> > * results were saved as uncompressed CZI files
> > 
> > Trying to open them in Fiji using the latest Bio-Formats (5.1.8-DEV)
> > works well for relatively small files (up to 12 GB tested) but fails
> > for big files (100 GB tested).
> > 
> > The biggest working CZI consists of:
> > * 3 channels
> > * 9 z-slices
> > * 1 scene (Zeiss-speak for connected regions, i.e. tilings)
> > 
> > We tested other files containing more than one scene, they opened
> > correctly as one image per scene.
> > 
> > The problematic 100 GB file consist of: 3ch, 3z, 8 scenes
> > 
> > Opening this file fails in the following way:
> > 
> > - Bio-Formats recognizes the CZI as an image sequence with 1008
> > images instead of 8 individual stitched images (one per scene).
> > - When opening one of the images in this sequence, the number of
> > planes and channels are correct, but pixel values are all ZERO, i.e
> > the entire image is black.
> > - The size of this image corresponds to the size of the full CZI,
> > not the size of a scene, further reflecting that scenes are no more
> > recognized as such.
> > - Metadata opens correctly and is attached to this mail (hoping the
> > mailing list server lets the 7z compressed file pass).
> > 
> > According to our tests the critical parameter that leads to this
> > behaviour seems to be purely the size of the CZI, as smaller files
> > with a comparable complexity in structure are opened correctly.
> > 
> > To further put this into context, we wish to use OMERO to handle the
> > massive amount of data that is created by the Slidescanner. From an
> > experimenters perspective, the way this 100 GB data set was recorded
> > is a realistic scenario.
> > 
> > We'd be very happy if you could help us in accomplishing this goal!
> > Please let me know if there is a convenient way that we could make
> > these datasets available to you.
> > 
> > Thanks and cheers,
> > Kai & Niko
> 
> 
> > _______________________________________________
> > ome-devel mailing list
> > ome-devel at lists.openmicroscopy.org.uk
> > http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>