[ome-devel] TIFF pyramid support in Bio-Formats - reference files for review

[Roger Leigh]

This should primarily be of interest to Damir Sudar, and anyone else
following the TIFF and OME-TIFF sub-resolution support proposal
(http://openmicroscopy.github.io/design/OME005/).

I have created a set of TIFF and OME-TIFF files containing pyramids
stored as SUBIFDs, which can be downloaded from Box using this link:
https://uod.box.com/s/shoodcr8j1zem3atzcql3c69cc8xrm5x

These files implement strategies "B" and "C" in the above proposal
document.  Taking publicly available sample datasets using existing
pyramid formats (Leica SCN and NDPI), we have tweaked them to store the
pyramid data in SUBIFDS.  Files with a "subifds.tiff" suffix implement
strategy "C" (data in isolated SUBIFDs), while files with a
"subifds-flat.tiff" implement strategy "B" (SUBIFDs linking back to IFDs
in the main directory list).  Files with an "ome.tiff" extension are
exactly the same, but contain OME-XML metadata with TiffData IFD indexes
pointing to the top-level IFDs containing the SUBIFDS.

I will be using these files for validating the Bio-Formats and OME-Files
TIFF and OME-TIFF readers as I add support for pyramids to them.
Likewise for the writers as we add support to bfconvert and can use them
as source material.  If you want to take a look at the sample file
structure and see if this is aligned with your needs, matches the
design, and is generally sensible, that would certainly be appreciated.
If there are any faults either with the design or the reference files,
we can certainly amend them.  You can follow up here on ome-devel, or
open a github issue on https://github.com/openmicroscopy/design

I'd recommend the Leica samples over the NDPI samples, since the tiled
JPEG pixel data is easier to view.  You can view the file structure with
a tool such as libtiff's "tiffinfo", which will show the Directory
entries plus SubIFDs.  You can look at the SubIFD entries with the "-o"
option with a SubIFD offset.  Viewing is possible with the libtiff
"tiffgt" tool, again with "-o" and the appropriate IFD offset (though
you might be constrained by memory and only smaller pyramid levels will
be viewable).

Note that the sample files won't actually work properly with Bio-Formats
or OMERO until we have added pyramid support to the readers.  Some fail
to read properly, or work but are missing the resolution levels.  This
will be rectified soon (I'm working on reader support right now).
Strategy "B" files in particular don't work nicely with the existing
MinimalTiffReader, so Strategy "C" continues to be preferable.


Kind regards,
Roger

--
Dr Roger Leigh -- Open Microscopy Environment
Wellcome Trust Centre for Gene Regulation and Expression,
College of Life Sciences, University of Dundee, Dow Street,
Dundee DD1 5EH Scotland UK   Tel: (01382) 386364

The University of Dundee is a registered Scottish Charity, No: SC015096