[ome-devel] smart way to open a time series of one pixel?

[Grabmayr, Heinrich]

Hi Melissa,

> > I have huge movies (20k to 60k frames) in file formats from various
> > camera suppliers that can all be opened using bioformats. I would like
> > to access the time series of indivindual pixels. At the moment, I do
> > this in matlab by using r = bfGetReader and then using the
> loci.formats.ChannelSeparator's function to call
> r.openBytes<http://downloads.openmicroscopy.org/bio-
> formats/5.0.6/api/loci/formats/ChannelSeparator.html#openBytes%28int,%
> 20int,%20int,%20int,%20int%29>(int no, int x, int y, int w, int h) with w=h=1
> and looping through all the planes. This takes ages. I was wondering
> whether there might be a smarter way to access the data I am interested
> in?
> 
> I would suggest starting by allocating a single buffer, and then calling
> openBytes(int, byte[], int, int, int, int) repeatedly instead of openBytes(int,
> int, int, int, int).  The latter will allocate a new byte array to contain the
> pixel(s) each time it is called; I would expect this to have a noticeable
> performance impact when called thousands of times.

Thanks for this suggestion. Are you familiar with MATLAB, and how to implement this? I have used the approach mentioned because I do not know how to pass a reference to an array in matlab.

I investigated the timing just now:
Using one call of openBytes(int, int, int, int, int) takes 0.175 ms. This call, and the necessary type casting to be able to store it into a matlab array takes 0.989 ms (although now I come to think about it, maybe I should try typecasting the whole array after loading)
Opening the file using r=bfGetReader(id) takes 40ms
And opening the file and loading 4000 frames take 16s. In this case (.fits file), an acquisition is stored in 15 bunches of 2GB-files, so the whole data set is loaded after 4 minutes. I do this for several data sets, and that is why I say it takes very long. Broken down, these times do not seem to be bad to me. Maybe I just have incredibly much data to load.
While at the subject, the .cxd files are stored as single 30GB files. When I load these, r=bfGetReader takes 5-10 minutes to call. Shouldn't the metadata information from one plane be enough, so that the time to get the reader wouldn|t  have to depend on the file siye?

> 
> If reusing a single buffer doesn't help noticeably, it would be helpful to know
> more about exactly which formats you are working with, as well as roughly
> how long (in minutes) it takes to open all of the pixels.

I am using .fits files saved by the Andor software Solis, .cxd from Hamamatsu LiveImage, and .pcoraw from PCO's CamWare.

> 
Thanks, 
    Heinrich