[ome-devel] Super-Resolution standard format

[mahogny at areta.org]

Hi chris! 

If you ever into working with that format then let me
share an idea I never had time to implement during my phd... 

scanning
through the OMETIFF is currently dirt-slow if there are many planes in
one (I have files that take 5min+ to open, no image plane reading
included). I guess that is due to the linkage structure as well as the
many random reads through the file. it would be great if the format
could be extended with a block having pointers to all the planes in one
single place. this would greatly up the indexing speed. given that
super-res isn't exactly going to spit out planes like crazy this might
not be super-relevant but if you ever get into ometiff, feel free to
give this extension a thought! 

side-note, storing in a single-unit is
a no-brainer. only exception will be if you want to store point clouds
of the universe in the format, in which case you will run out of
floating point range if you only have one unit. I'd save this problem
for the future! 

/Johan 

2015-10-01 17:13 skrev Chris Weisiger: 

> Hi
all,
> 
> Quick background: I'm one of the µManager developers, and
recently had to spend some time going through our TIFF file format. We
may need to be able to output super-resolution localization data
sometime in the future, so I have a potential horse in this race. :)
>

> On Thu, Oct 1, 2015 at 5:32 AM, Alex Herbert <a.herbert at sussex.ac.uk>
wrote:
> 
>> Hi Nico,
>> 
>> Thanks for your comments.
>> 
>> I agree
that the conversion field I suggested is not perfect. My idea was that
software could choose to write their data using any units they wanted,
so speeding up data writing by not have to convert during output.
However they should provide a method to convert them to a standard.
> 
>
I do not believe this would produce a measurable improvement in file
writing speed. Unit conversion is a simple multiplication, maybe with an
addition in some cases; the addition of two operations to your output
code will not make a difference. Meanwhile, allowing the file writer to
use arbitrary units puts a significant onus on whoever must read the
files. 
> 
>> For conversion of the Intensity, I left out that the gain
would be ADUs/photon. This is implied by the requirement that the
conversion field converts the specified unit to the standard. Thus in my
example the conversion for X is nm/Pixel and Frames is seconds/Frame,
since the standard units are nm and seconds respectively.
>> 
>> If you
allow the data to be recorded using any units with the requirement that
the units be included then you are leaving a problem for anyone reading
the file, i.e. they must be able to understand the units. The options so
far suggested are:
>> 
>> 1. Use standard units
>> 2. Use any units you
want and include the units in the file
>> 3. Use a subset of defined
units for each field and include the units in the file
>> 4. Use any
units you want and include a way to convert them to standard units
> 
>
My preference would be for 1, with the ability to mark the units as
"unknown; described by creator as X". In other words, if I, as a file
writer, am *unable* to convert to standard units, then I can say "Look,
best I can tell you is that I got 75 counts from the camera" and then
the number would be 75 with a unit of "counts" in all representations
(i.e. with no attempt at unit conversion). In all other cases, as I *am*
able to convert to standard units, I must do so.
> 
> There should be
one right way to write data. If users want to use different units to
*interpret* the data, well, of course they can do that, but there should
be only one standard set of units in the storage. That vastly simplifies
the lives of whoever has to read/write this data because they don't have
to cope with a bunch of different valid ways things could potentially be
done.
 

Links:
------
[1]
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20151001/5744fc13/attachment.html>