[ome-devel] tif import performance

[Richard Beare]

Hi,
I've been thinking about this issue a bit further, so I thought I'd 
better check to see whether I have the order of events correct in my head.

When an "ome import x.tif" command is run, does the following happen:

x.tif is given to OMEIS, which presumably converts it to an internal 
representation.

ome then determines the tif tags by calling the readData() function 
which in turn calls the OMEIS ReadFile function.

-------------------------

I'm not completely sure why the tiff tags need to be read at all - is it 
just so that the dimensions and types are known?

However, ome has direct access to the tif file, so it could presumably 
read the tags using tifflib before entering the file into the image 
server. I guess this could be seen as moving the feature creep to 
somewhere else.

Sorry if this is a dumb question, I'm still trying to learn how 
everything fits together.....

Macura, Tomasz (NIH/NIA/IRP) wrote:
> Dear Richard,
> 
> Tiff tag reading is indeed a bottleneck. I am not sure what you mean by
> perl->libtiff, but our tiff tag reading library is 
> OME/src/perl2/OME/ImportEngine/TIFFUtils.pm .
> 
> Reading tiff tags amounts to doing many reads (readData() calls) on the
> original file. Each call to readData() is much
> less efficient then ideal because readData() has to get the data from OMEIS
> (using the OMEIS method ReadFile()
>  http://www.openmicroscopy.org/api/omeis/files.html#ReadFile) and there is a
> lot of surrounding logic written in Perl 
> that deals with caching. As you can see, due to our requirements with regard
> to OMEIS we can't use Perl's built-in libraries.
> 
> The end result is that the TIFF perl command readTiffIFD() takes alot
> (10-100 times) longer than the libtiff's tifffinfo CLI program.
> 
> The optimizations I did with regard to improving tiff tag reading was (1)
> TIFF caching and (2) increased IFD specificity. So right
> now the tags have to be read only once and its possible to specify quite
> explicitly which tags are required. This means that as
> few as possible readTiffIFD() calls are made per image.
> 
> In my limited imagination, the only way of improving performance is to get
> OMEIS (which is written in C) to use libtiff to
> get the TIFF tags and send them over, in mass, to the ImportEngine.
> Implementing this is surely very feasible but I am not able to
> comment on the larger issues (e.g. OMEIS feature creep).
> 
> Regards,
> 
> Tom
> 
> 


-- 
Richard Beare, CSIRO Mathematical & Information Sciences
Locked Bag 17, North Ryde, NSW 1670, Australia
Phone: +61-2-93253221 (GMT+~10hrs)  Fax: +61-2-93253200

Richard.Beare at csiro.au