[ome-users] Invalid values stored in TIFF-tag 'StripByteCounts' when writing striped OME-TIFF
Melissa Linkert
melissa at glencoesoftware.com
Tue Sep 11 01:56:42 BST 2012
Hi Matthias,
> However, when using LZW compression, it nearly ever results in an
> OME-TIFF which cannot be opened.
> To locate the error I analyzed the IFD of the TIFF and observed that the
> tag 'StripByteCounts' doesn’t contain valid values.
> In the case when there are just a few stripes in the OME-TIFF, the
> values of 'StripByteCounts' are too large, even exceeding the number of
> bytes when the strip would be uncompressed.
> In the case when there are more stripes in the OME-TIFF, the values of
> 'StripByteCounts' are zero.
> So I had a look at the IFD-class in the loci.formats.tiff package an
> encountered the following lines in the getStripByteCounts() method:
>
> if (getCompression() == TiffCompression.LZW) {
> for (int i=0; i<byteCounts.length; i++) {
> counts[i] = byteCounts[i] * 2;
> }
> }
>
> I removed the multiplication (* 2) and it seems to solve the error with
> the wrong values of 'StripByteCounts'.
This logic was originally introduced to work around the fact that some files
(notably, many Zeiss .lsm files) are LZW-compressed but are missing the
RowsPerStrip entry. Doubling the byte count ensures that all of the
data is read, even if the number of "compressed" bytes is greater than the
number of raw bytes (which occasionally happens).
Simply removing this isn't sufficient, as there are still cases where it
is needed. This commit should be an effective compromise:
https://github.com/melissalinkert/bioformats/commit/0144587f34bd5110528152cfcbb2498d381cde00
> The OME-TIFFs can be opened now (even containing many stripes).
> The multiplication increases the values of 'StripByteCounts' every time
> a strip gets stored in the OME-TIFF.
> This would explain the very large numbers and, when more strips are
> added, the zeros because of an overflow.
>
> Below, I added an example-code (TiffStripWriter.java) to reproduce this
> error.
> Example settings:
> TIFF_WIDTH = 2000, TIFF_HEIGHT = 800, STRIP_WIDTH=2000, STRIP_HEIGHT =
> 80 (= values of 'StripByteCounts' are very large)
> TIFF_WIDTH = 2000, TIFF_HEIGHT = 800, STRIP_WIDTH=2000, STRIP_HEIGHT = 8
> (= values of 'StripByteCounts' are zero)
Thank you for the test case. I do see an error when reading back an
OME-TIFF created with the default settings:
----
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException
at java.lang.System.arraycopy(Native Method)
at loci.formats.tiff.TiffParser.getSamples(TiffParser.java:896)
at loci.formats.tiff.TiffParser.getSamples(TiffParser.java:731)
at loci.formats.in.OMETiffReader.openBytes(OMETiffReader.java:250)
at loci.formats.FormatReader.openBytes(FormatReader.java:777)
at loci.formats.ImageReader.openBytes(ImageReader.java:400)
at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:305)
at loci.formats.gui.BufferedImageReader.openImage(BufferedImageReader.java:94)
at loci.formats.tools.ImageInfo.readPixels(ImageInfo.java:778)
at loci.formats.tools.ImageInfo.testRead(ImageInfo.java:1000)
at loci.formats.tools.ImageInfo.main(ImageInfo.java:1031)
----
...which is a separate small bug fixed by this commit:
https://github.com/melissalinkert/bioformats/commit/c16261d3d3b1f7434634ef30c075a4e5836e01c6
Regards,
-Melissa
On Fri, Sep 07, 2012 at 05:31:12PM +0200, Matthias Baldauf wrote:
> Hello all,
>
> I am using the Bio-Formats Java library (trunk build) to write an
> OME-TIFF sequentially strip per strip.
> It is basically the same approach I mentioned in a previous post
> (http://lists.openmicroscopy.org.uk/pipermail/ome-users/2012-August/003178.html),
> but now I am reading the JPEG in stripes and store them strip per strip
> in an OME-TIFF. It works great for striped uncompressed OME-TIFFs.
>
> However, when using LZW compression, it nearly ever results in an
> OME-TIFF which cannot be opened.
> To locate the error I analyzed the IFD of the TIFF and observed that the
> tag 'StripByteCounts' doesn’t contain valid values.
> In the case when there are just a few stripes in the OME-TIFF, the
> values of 'StripByteCounts' are too large, even exceeding the number of
> bytes when the strip would be uncompressed.
> In the case when there are more stripes in the OME-TIFF, the values of
> 'StripByteCounts' are zero.
> So I had a look at the IFD-class in the loci.formats.tiff package an
> encountered the following lines in the getStripByteCounts() method:
>
> if (getCompression() == TiffCompression.LZW) {
> for (int i=0; i<byteCounts.length; i++) {
> counts[i] = byteCounts[i] * 2;
> }
> }
>
> I removed the multiplication (* 2) and it seems to solve the error with
> the wrong values of 'StripByteCounts'.
> The OME-TIFFs can be opened now (even containing many stripes).
> The multiplication increases the values of 'StripByteCounts' every time
> a strip gets stored in the OME-TIFF.
> This would explain the very large numbers and, when more strips are
> added, the zeros because of an overflow.
>
> Below, I added an example-code (TiffStripWriter.java) to reproduce this
> error.
> Example settings:
> TIFF_WIDTH = 2000, TIFF_HEIGHT = 800, STRIP_WIDTH=2000, STRIP_HEIGHT =
> 80 (= values of 'StripByteCounts' are very large)
> TIFF_WIDTH = 2000, TIFF_HEIGHT = 800, STRIP_WIDTH=2000, STRIP_HEIGHT = 8
> (= values of 'StripByteCounts' are zero)
>
>
> Thanks in advance for having a look at it!
>
> Regards,
> Matthias
>
>
>
> ------------------------------------------------
> TiffStripWriter.java
> ------------------------------------------------
> import loci.common.services.ServiceFactory;
> import loci.formats.FormatTools;
> import loci.formats.MetadataTools;
> import loci.formats.meta.IMetadata;
> import loci.formats.out.OMETiffWriter;
> import loci.formats.out.TiffWriter;
> import loci.formats.services.OMEXMLService;
> import loci.formats.tiff.IFD;
>
> public class TiffStripWriter {
>
> static final int PIXEL_TYPE = FormatTools.UINT8;
> static final int COMPONENTS = 3;
> static final int TIFF_WIDTH = 2000;
> static final int TIFF_HEIGHT = 800;
> static final int STRIP_WIDTH = 2000;
> static final int STRIP_HEIGHT = 80;
>
> public static void main(String[] args) throws Exception {
> if (args.length < 1) {
> System.out.println("Please specify an output file name.");
> System.exit(1);
> }
> String id = args[0];
>
> // create metadata object with minimum required metadata fields
> ServiceFactory factory = new ServiceFactory();
> OMEXMLService service =
> factory.getInstance(OMEXMLService.class);
> IMetadata meta = service.createOMEXMLMetadata();
> MetadataTools.populateMetadata(meta, 0, null, false, "XYZCT",
> FormatTools.getPixelTypeString(PIXEL_TYPE), TIFF_WIDTH, TIFF_HEIGHT, 1,
> COMPONENTS, 1, COMPONENTS);
>
> // set up the writer for OME-TIFF
> TiffWriter writer = new OMETiffWriter();
> writer.setMetadataRetrieve(meta);
> writer.setInterleaved(Boolean.TRUE);
> writer.setBigTiff(false);
> writer.setId(id);
> writer.setCompression(TiffWriter.COMPRESSION_LZW);
>
> // set up IFD for creating a striped OME-TIFF
> IFD ifd = new IFD();
> ifd.put(IFD.ROWS_PER_STRIP, STRIP_HEIGHT);
>
> // create blank image
> byte[] img = new byte[STRIP_WIDTH * STRIP_HEIGHT * COMPONENTS *
> FormatTools.getBytesPerPixel(PIXEL_TYPE)];
>
> // write OME
> -TIFF
> for (int y = 0; y for (int i = 0; i < img.length; i++)
> img[i] = (byte) (256 * Math.random());
>
> // write strip
> writer.saveBytes(0, img, ifd, 0, y * STRIP_HEIGHT,
> STRIP_WIDTH, STRIP_HEIGHT);
> }
>
> writer.close();
> }
>
> }
>
> _______________________________________________
> ome-users mailing list
> ome-users at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
More information about the ome-users
mailing list