[ome-devel] [ome-files-cpp] Cannot Write Multiple ROIs to Image Plane Using SaveBytes

[Roger Leigh]

On 28/06/18 22:55, Dennis Ai wrote:
> Hi Roger,
> 
> If I understand what you are saying, it is that if you only write a ROI of
> the entire plane, it will write that ROI to the directory and won't set all
> the proper tile offsets and sizes in the directory, correct?  I am thinking
> a workaround could be to simply write zeroed data to the entire plane, and
> then during image acquisition, write the first tile to multiple planes, then
> the 2nd tile, and so on.  I've pasted a C# example using BitMiracle's
> LibTIFF
> wrapper, which works fine when I tested it.
> 
> However, when I tried to the same thing using OMETIFFWriter, it demonstrates
> the strange behavior that I described in my previous email.  Doing some more
> debugging into it (see my C# Example with OME Files code example below):
> 
> 1. After Writing Zero to All Planes: I have a TIFF with two directories
> (no_rois.ome.tiff, attached).
> 2. After Writing the First ROI to All Planes: I have a TIFF with three
> directories (first_roi.ome.tiff, attached).  The first directory looks like
> it's all zeroes, and the other two look like they have the first ROI
> written.
> 3. After Writing the Second ROI to All Planes: I have a TIFF with five
> directories (first_second_roi.ome.tiff, attached).  The first directory
> looks like it's all zeros, the second directory and third directories have
> the first ROI written, and the 4th and 5th directories have what I believe
> is the first ROI, except now instead of being the first tiles in the
> directory, they are written at the 7th tile in the directory.
> 4. After Writing the Third ROI to All Planes: I have a TIFF with seven
> directories (first_second_third_roi.ome.tiff, attached).  The first
> directory looks like it's all zeros, the second and third directories have
> the first ROI written, the 4th and 5th directories have the first ROI
> written to the 7th tiles in their respective directories, and the 6th and
> 7th directories have the first ROI written to the 14th tile in the
> directory.
> 
> This seems to be very strange behavior.  I would expect writing ROIs to not
> create new directories, and simply update existing tiles in existing
> directories.

The odd behaviour is because backtracking to previously written planes 
is completely unsupported at present.  [When porting the Java TIFF 
writers, this wan't part of the design and so was not considered a part 
of that work.]  Switching planes results in flushing the IFD and 
creating a new one, irrespective of whether that plane was already 
partially written [see OMETIFFWriter::setPlane].

I have asked on the libtiff mailing list for some clarification about 
what's possible here.  I'll see what the consensus is, and also do some 
testing myself.  If it's possible, we can update the above method to 
revisit the plane written earlier and update it, which should then give 
you the behaviour you require.

The main concern is the directory rewriting.  If it requires appending a 
new copy of the IFD with TIFFRewriteDirectory, it can be expensive:

If you take a 100000×100000 pixel images, with 512×512 tiles and 12 IFDs:

ceil(100000/512) = 196
196^2 = 38416 tiles

38416×16 = 614656 bytes (615 KiB) per plane for tile offsets and sizes 
[bigtiff]

If we stripe tile writing across the 12 planes, and have to rewrite each 
time, that's

614656×391 = 240330496 (240 MiB) wasted space per plane
or 240330496×12 = 2883965952 (2.88 GiB) wasted space for all 12 planes
and that's not including the non-tile metadata.

So long as libtiff can support the addition of new tiles without 
rewriting, we can most likely amend the behaviour without trouble.  If 
not, the above would be a blocker.


Regards,
Roger