[ome-devel] tif import performance

[Richard Beare]

Hi,
I guess I was a bit confused - I assumed that the work was being done in 
a slightly different order - i.e the OME perl code in TIFFUtils reads 
from disk before sending the data to the image server, and that the 
bottle neck was really the use of perl. The fix in this case could be to 
have the perl code interfaced to the tiff library.

As a bottle neck it seems potentially serious. In our case the tiff tags 
aren't particularly important (i.e not proprietary), so are we better 
off converting to something else and using an alternative import 
mechanism if we are interested in speed?

At the moment it is looking as though it could take much longer to get 
the images into OME than it does to capture them.

Thanks

Macura, Tomasz (NIH/NIA/IRP) wrote:

>Dear Richard,
>
>Tiff tag reading is indeed a bottleneck. I am not sure what you mean by
>perl->libtiff, but our tiff tag reading library is 
>OME/src/perl2/OME/ImportEngine/TIFFUtils.pm .
>
>Reading tiff tags amounts to doing many reads (readData() calls) on the
>original file. Each call to readData() is much
>less efficient then ideal because readData() has to get the data from OMEIS
>(using the OMEIS method ReadFile()
> http://www.openmicroscopy.org/api/omeis/files.html#ReadFile) and there is a
>lot of surrounding logic written in Perl 
>that deals with caching. As you can see, due to our requirements with regard
>to OMEIS we can't use Perl's built-in libraries.
>
>The end result is that the TIFF perl command readTiffIFD() takes alot
>(10-100 times) longer than the libtiff's tifffinfo CLI program.
>
>The optimizations I did with regard to improving tiff tag reading was (1)
>TIFF caching and (2) increased IFD specificity. So right
>now the tags have to be read only once and its possible to specify quite
>explicitly which tags are required. This means that as
>few as possible readTiffIFD() calls are made per image.
>
>In my limited imagination, the only way of improving performance is to get
>OMEIS (which is written in C) to use libtiff to
>get the TIFF tags and send them over, in mass, to the ImportEngine.
>Implementing this is surely very feasible but I am not able to
>comment on the larger issues (e.g. OMEIS feature creep).
>
>Regards,
>
>Tom
>
>
>  
>