[ome-devel] [ome-files-cpp] Cannot Write Multiple ROIs to Image Plane Using SaveBytes

[Roger Leigh]

On 19/06/18 22:15, Dennis Ai wrote:
> Hello OME Team,
> 
> I am working on an image acquisition pipeline that writes to a tiled 
> OME-TIFF.  Let’s say for argument’s sake, my final image is 4096 x 
> 4096.  What I would like to do is:
> 
>  1. Write the first region (2048 x 2048) I acquired to the top-left of
>     the final image.  I call /saveBytes/ with a ROI defined as (0, 0,
>     2048, 2048) [x, y, width, height].  This works okay.
>  2. Write the second region (2048 x 2048) to the right of the first
>     region.  I call /saveBytes/ with a ROI defined as (2048, 0, 2048,
>     2048).  This does not get written.
> 
> I have also tried variations where I write to the bottom of the first 
> region, which doesn’t work either.  Is there a mechanism that allows me 
> to sequentially write regions of interest to the same plane?

Dear Dennis,

What you are describing should work and is explicitly tested for by the 
unit tests.  It's possible there is something which doesn't work as 
expected, but I would consider it unlikely.  I will describe how this 
works internally below.

When writing out a plane of pixel data into a TIFF IFD, it is expected 
that every pixel on the plane be written once only by a saveBytes call. 
The API doesn't mandate any ordering requirements, and it also doesn't 
mandate writing on tile boundaries.  However, libtiff does require the 
tiles be written in a specific linear order.  And writing complete tiles 
in that order will have some performance benefits.  We maintain a tile 
cache to retain tiles which can't yet be written or which are incomplete 
(only partially covered by pixels), plus an R*Tree coverage map to 
detect overlapping writes and completed tiles.

Let's take a simple image, 4096×4096 with 1024×1024 tiles:

  ┌──┬──┐
A│01│23│B
  │45│67│
  ├──┼──┤
C│89│ab│D
  │cd│ef│
  └──┴──┘

The numbers are the order in which tiles will be written to the TIFF 
file. I've blocked it into the 2048×2048 regions you were writing, which 
I've labelled A-D.

If we write in the order A, B, C, D, the tiles will be flushed to disk 
in this order:

A: 01
B: 234567
C: 89
D: abcdef

This is because when we write A, it's only possible to flush two tiles. 
When we write B, it's possible to flush six.  The same pattern repeats 
for C and D.

If we write in reverse order B, C, B, A, we get:

D:
C:
B:
A: 01234567abcdef

This is because it's not possible to flush the tile cache until the very 
last saveBytes call, due to the ordering requirements.

For a different order, B, A, D, C:

B:
A: 01234567
D:
C: 89abcdef

You can see that the tile cache will be flushed to the maximum extent 
possible after each saveBytes call.  But the key point that I want to 
make is that the tile cache won't be flushed if the coverage map isn't 
complete, i.e. some pixels were missing from the full plane.

For your example (2): (2048, 0, 2048, 2048), did you also follow up with
   (0, 0, 2048, 2048)
   (0, 2048, 2048, 2048)
   (2048, 2048, 2048, 2048)
saveBytes calls?  If you did not, then that would possibly explain why 
nothing was written.  It's a requirement that the plane be completely 
written.  TIFF does not support sparse planes.  At least, not for a 
conforming valid TIFF file.


I hope that helps, and if not then it might be useful to see some of 
your code or a more complete description of what you are trying to 
accomplish.


Kind regards,
Roger