[ome-devel] Bio-Formats problem with AxioScan CZI files (uncompressed)

[Melissa Linkert]

Hi Niko and Kai,

> we are required to deal with CZI data created by an Zeiss
> AxioScan.Z1 SlideScanner:
> 
> * acquisition software is ZEN 2 (blue)
> * tile-stitching was done directly after acquisition in "offline mode"
> * results were saved as uncompressed CZI files
> 
> Trying to open them in Fiji using the latest Bio-Formats (5.1.8-DEV)
> works well for relatively small files (up to 12 GB tested) but fails
> for big files (100 GB tested).
> 
> The biggest working CZI consists of:
> * 3 channels
> * 9 z-slices
> * 1 scene (Zeiss-speak for connected regions, i.e. tilings)
> 
> We tested other files containing more than one scene, they opened
> correctly as one image per scene.

OK, good to know that it's at least not a general problem with multiple
scenes.

> The problematic 100 GB file consist of: 3ch, 3z, 8 scenes
> 
> Opening this file fails in the following way:
> 
> - Bio-Formats recognizes the CZI as an image sequence with 1008
> images instead of 8 individual stitched images (one per scene).
> - When opening one of the images in this sequence, the number of
> planes and channels are correct, but pixel values are all ZERO, i.e
> the entire image is black.
> - The size of this image corresponds to the size of the full CZI,
> not the size of a scene, further reflecting that scenes are no more
> recognized as such.
> - Metadata opens correctly and is attached to this mail (hoping the
> mailing list server lets the 7z compressed file pass).
> 
> According to our tests the critical parameter that leads to this
> behaviour seems to be purely the size of the CZI, as smaller files
> with a comparable complexity in structure are opened correctly.

In looking at the metadata output, it seems like the scenes are
detected, but that each one is stored as a set of tiles (per
"Information|Image|SizeM" near the bottom of the .txt).  My guess is
that's what is confusing the dimensions - can you confirm that the
working datasets show the SizeM to be 1?

> To further put this into context, we wish to use OMERO to handle the
> massive amount of data that is created by the Slidescanner. From an
> experimenters perspective, the way this 100 GB data set was recorded
> is a realistic scenario.

All makes sense.

> We'd be very happy if you could help us in accomplishing this goal!
> Please let me know if there is a convenient way that we could make
> these datasets available to you.

If you're able to upload the data, that would certainly help us to
reproduce and fix the problem - I know that's asking a lot given the
size of the data though, so if not possible just let us know.  I've sent
upload instructions in a private email to you both.

Regards,
-Melissa

On Tue, Feb 23, 2016 at 12:55:29PM +0100, Niko Ehrenfeuchter wrote:
> Dear all, dear Melissa,
> 
> we are required to deal with CZI data created by an Zeiss
> AxioScan.Z1 SlideScanner:
> 
> * acquisition software is ZEN 2 (blue)
> * tile-stitching was done directly after acquisition in "offline mode"
> * results were saved as uncompressed CZI files
> 
> Trying to open them in Fiji using the latest Bio-Formats (5.1.8-DEV)
> works well for relatively small files (up to 12 GB tested) but fails
> for big files (100 GB tested).
> 
> The biggest working CZI consists of:
> * 3 channels
> * 9 z-slices
> * 1 scene (Zeiss-speak for connected regions, i.e. tilings)
> 
> We tested other files containing more than one scene, they opened
> correctly as one image per scene.
> 
> The problematic 100 GB file consist of: 3ch, 3z, 8 scenes
> 
> Opening this file fails in the following way:
> 
> - Bio-Formats recognizes the CZI as an image sequence with 1008
> images instead of 8 individual stitched images (one per scene).
> - When opening one of the images in this sequence, the number of
> planes and channels are correct, but pixel values are all ZERO, i.e
> the entire image is black.
> - The size of this image corresponds to the size of the full CZI,
> not the size of a scene, further reflecting that scenes are no more
> recognized as such.
> - Metadata opens correctly and is attached to this mail (hoping the
> mailing list server lets the 7z compressed file pass).
> 
> According to our tests the critical parameter that leads to this
> behaviour seems to be purely the size of the CZI, as smaller files
> with a comparable complexity in structure are opened correctly.
> 
> To further put this into context, we wish to use OMERO to handle the
> massive amount of data that is created by the Slidescanner. From an
> experimenters perspective, the way this 100 GB data set was recorded
> is a realistic scenario.
> 
> We'd be very happy if you could help us in accomplishing this goal!
> Please let me know if there is a convenient way that we could make
> these datasets available to you.
> 
> Thanks and cheers,
> Kai & Niko


> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel