[ome-devel] How to define a photon counting format ?

[Roger Leigh]

On 04/07/2012 10:29, TOURNEUR YVES wrote:
> We are developing photon counting cameras.
>
> The data are very sparse and the images are not stored. Every frame of
> the camera is analyzed on-line and the coordinates of a photon is stored
> as a set of values of time and position: t, x and y.
>
> Depending on the technology, t, x and y may either be integer (frame
> number, and position in the frame) or real (i.e. after interpolation).
>
> We need to set-up a standard format to store these data, with a header
> containing the usual parameters (date, comments, optical device
> settings…) and the possibility to incorporate in the future more data
> such as event-markers.
>
> Is there an existing standard we could use or modify ? At least, some
> suggestions ?

It would be useful to know more about how you intend to process the data
post-acquisition, since this will affect how to store it in terms of
being efficient to access.  Will it be used for FLIM or FCS; will you
want to bin the data or look at time correlation?  I've made some
initial suggestions below, but having more information about exactly
what you want to do would help here.  Information about the acquisition
such as sampling rate and resolution would also be helpful, i.e. FWHM
etc.  It would also be useful to know if you can count multiple photons
at each (t,x,y) coordinate.

In terms of standard formats, I'm unaware of existing formats storing
t,x,y only.  The OME model and most of the formats supported by
Bio-Formats are planar image data, whereas single photon counting is
moving a bit outside that scope; it's still representable in this
format, but it might not be the most efficient approach.  One format
which might provide suitable support is the Becker & Hickl SDT/SPC
format.  It supports (t,x,y[,λ]) acquisition in time-tag mode. The
format is proprietary, but there's a reader (no writer) in Bioformats.

The OME-TIFF format could store the data as a planar timeseries, with
events at a single timepoint being represented as 0/1 values on the
pixel grid (or >1 if you can sample multiple photons or bin the data).
However, the timestamp resolution is likely not sufficient to represent
your sampling rate.  High resolution ps timestamps could be added as
custom annotations (though depending upon the sampling rate, this could
result in rather large numbers of annotations).

An alternative to OME-TIFF would be to store your data in your own
format, or a container format such as HDF5.  This would not preclude
using the OME model for storing all the metadata such as data, comments
and device settings, since this is not tied to the OME-TIFF format, and
can be embedded in your format, e.g. as a attribute in HDF5.  One useful
property of HDF5 is that it can handle storage of sparse data, so you
could store your data in a 3D (t,x,y) sparse volume using the data type
of your choice (uint, float, double, etc.).  Or you could store it as an
event list of (t,x,y) as a 2D array, again using the data type of your
choice.  If you need to have (x,y) as real numbers, this makes it
difficult to fit into a 2D array indexed by value, so storing as a 2D
event list might be a better fit.


Regards,
Roger Leigh

The University of Dundee is a registered Scottish Charity, No: SC015096