[ome-devel] Super-Resolution standard format

[Nico Stuurman]

Hi all,

> I agree that the conversion field I suggested is not perfect. My idea 
> was that software could choose to write their data using any units 
> they wanted, so speeding up data writing by not have to convert during 
> output. However they should provide a method to convert them to a 
> standard.
I agree with Chris and others, that speeding up writing the file is not 
an important consideration (unlike speeding up writing image files, 
which is a real problem with the OME-Tiff "standard" as also discussed 
in this thread), but speed of reading is important. Moreover, the simple 
conversions that are required can be executed very fast so are of no 
concern at all.  More important is whether or not the writer has access 
to all the needed data, where the unit for "Intensity" is the most 
contentious:

> For conversion of the Intensity, I left out that the gain would be 
> ADUs/photon. 
What does that mean?  I guess that ADU stands for Analog Digital Unit, 
which I guess stands for the pixel intensities as provided by the 
camera.  Common EM-gain CCD cameras spit out digital numbers whose 
relation to numbers of photons absorbed by the sensor depend on the 
analog gain of the read-out amplifier, the read-out speed, and the 
Electron Multiplication gain.  In many (most?) situations, the 
experimenter has no idea of these things and does not know what the 
final conversion factor (from digital numbers to photons) actually is.  
That was an issue with the EPFL-led localization microscopy challenge in 
which virtually no team provided intensity data in terms of photons.  
So, one can decide that data without that conversion factor are useless, 
and should not be stored, or one can allow two units for this field: 
photons and counts.  My vote is for the latter.

> This is implied by the requirement that the conversion field converts 
> the specified unit to the standard. Thus in my example the conversion 
> for X is nm/Pixel and Frames is seconds/Frame, since the standard 
> units are nm and seconds respectively.
To make this self-evident and human-readable (which are nice features!) 
simply include the actual unit (whether it is the only allowed one or 
one out of a choice of a few - as needed in the Intensity field -).

> If you allow the data to be recorded using any units with the 
> requirement that the units be included then you are  leaving a problem 
> for anyone reading the file, i.e. they must be able to understand the 
> units. The options so far suggested are:
>
> 1. Use standard units
> 2. Use any units you want and include the units in the file
> 3. Use a subset of defined units for each field and include the units 
> in the file
> 4. Use any units you want and include a way to convert them to 
> standard units
>
> For simplicity option 1 would be the best. It transfers responsibility 
> to the writer of the file to record the data correctly. However it 
> does not allow the format to be used in the case where a calibration 
> to convert to standard units is not available.
Surely go with option 1, however, give alternatives where they are 
needed as in the Intensity field.  Have a very short list of allowed 
fields so that it will be straight forward for readers to deal with the 
data.  Do not provide conversion factors, since conversions should be 
executed up front.

In all of this, the most important thing to succeed is to provide a 
reference implementation/library that can be used from most of the 
widely used programming environments, i.e. Java, C, Matlab, and Python.  
I am afraid that job will eclipse much of the discussion here (also note 
that most of this is already available for the Tagged Spot File format: 
https://micro-manager.org/wiki/Tagged_Spot_File_%28tsf%29_format.

Best,

Nico



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20151001/f2944c31/attachment.html>