<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<font size="-1">Hi Pedro,<br>
<br>
Indeed, I worked a bit on this a couple of years ago, but since we
do not do much super resolution ourselves I kind of lost steam.
The effort was also driven by Bo Huang, who also got focused on
other thing. Instead of trying to read every format out there, I
tried to develop libraries for the various relevant development
platforms (C++, Java, Matlab) that could read and write the
proposed format. <br>
<br>
We settled on a format based on google protocol buffers
(<a class="moz-txt-link-freetext" href="https://developers.google.com/protocol-buffers/">https://developers.google.com/protocol-buffers/</a>) because it
appeared to be a mature library for fast transmission of
structured data. Since PALM/STORM data sets can become quite
large, speed of transmission and parsing (as well as data size)
become important considerations. To be honest, I never fully
bench-marked our tagged spot format
(<a class="moz-txt-link-freetext" href="https://micro-manager.org/wiki/Tagged_Spot_File_%28tsf%29_format">https://micro-manager.org/wiki/Tagged_Spot_File_%28tsf%29_format</a>).
and it would be good to do such comprisons with other formats.<br>
<br>
Please do have a look at the tagged spot format, but feel free to
to propose something better. in any case, I am convinced that you
need to start out with libraries for reading and writing for all
major computing environments, have import and export functions for
text-based formats, and that you need to enthuse the developers
community to adapt your file format. Convincing Daniel Sage to
adapt it as the format for the next PALM/STORM challenge would go
a long way...<br>
<br>
Hope this helps!<br>
<br>
Best,<br>
<br>
Nico<br>
<br>
</font><br>
<div class="moz-cite-prefix">On 6/25/15 8:15 AM, Pedro Almada wrote:<br>
</div>
<blockquote
cite="mid:CAEtSi3yhY9oB=W5b0TNP9X7fz65hJ4+fSKDoV2bWAnY0=ioBJw@mail.gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<div dir="ltr">Hello all,
<div><br>
</div>
<div>Seamus Holden, involved with PALM-siever and the ISBI
challenge has a few comments to add:</div>
<div><br>
</div>
<div><span style="font-size:12.8000001907349px">A few thoughts -
this is actually something that I think is quite interesting
and I would like to see done correctly! Busy of course with
the new lab, but happy to help where I can.</span><br
style="font-size:12.8000001907349px">
<br style="font-size:12.8000001907349px">
<span style="font-size:12.8000001907349px">Nico Sturman
(Micromanager) was trying to get something similar going, </span><br
style="font-size:12.8000001907349px">
<a moz-do-not-send="true"
href="https://micro-manager.org/wiki/Tagged_Spot_File_%28tsf%29_format"
style="font-size:12.8000001907349px" target="_blank">https://micro-manager.org/wiki/Tagged_Spot_File_%28tsf%29_format</a><br
style="font-size:12.8000001907349px">
<span style="font-size:12.8000001907349px">It seemed promising
last time I checked. Link up the two efforts? I'd hate to
see two different Europe/ US standards emerge!</span><br
style="font-size:12.8000001907349px">
<br style="font-size:12.8000001907349px">
<span style="font-size:12.8000001907349px">Also - import can
be a bit of a challenge, as I learned for PALMsiever - the
BIG.EPFL approach wont work for every case since some
software, eg rapidstorm, dynamically change the number/
order of columns depending on the input settings, and use a
change in the header to reflect this. Then you have to get a
bit cleverer with parsing the file headers. An import/
export program where the import is extendible via a plugin
architecture might be the most flexible solution - we tried
to implement this for PALMsiever, but it turns out rather
ugly if you try and do it in MATLAB. </span><br
style="font-size:12.8000001907349px">
<br style="font-size:12.8000001907349px">
<span style="font-size:12.8000001907349px">We also have
example data for a couple of extra formats which could be
included:</span><br style="font-size:12.8000001907349px">
<span style="font-size:12.8000001907349px">- PeakSelector
(Harold Hess's IDL software)</span><br
style="font-size:12.8000001907349px">
<span style="font-size:12.8000001907349px">- Leica GSD format</span><br
style="font-size:12.8000001907349px">
<span style="font-size:12.8000001907349px">- DAOSTORM</span><br
style="font-size:12.8000001907349px">
<br style="font-size:12.8000001907349px">
<span style="font-size:12.8000001907349px">I'm talking to
Daniel Sage later today about the next ISBI challenge, one
thing to consider is requiring new algorithms in the
challenge to be submitted in the correct format.</span><br
style="font-size:12.8000001907349px">
<br style="font-size:12.8000001907349px">
<span style="font-size:12.8000001907349px">Oh - also - don't
forget about the coordinate system of the different
softwares - does 0,0 define the corner of a pixel or the
middle. Inconsistently chosen between different softwares,
and crucial when you start comparing algorithms.</span><br
style="font-size:12.8000001907349px">
<br style="font-size:12.8000001907349px">
<span style="font-size:12.8000001907349px">Anyway, just a few
initial thoughts.</span><br
style="font-size:12.8000001907349px">
<span style="font-size:12.8000001907349px">Best</span>
<div style="font-size:12.8000001907349px">
<div><img moz-do-not-send="true"
src="https://ssl.gstatic.com/ui/v1/icons/mail/images/cleardot.gif"></div>
</div>
<span style="font-size:12.8000001907349px"><font
color="#888888"><br>
Seamus</font></span><br>
</div>
</div>
<div class="gmail_extra"><br clear="all">
<div>
<div class="gmail_signature">
<div dir="ltr">
<div><span style="font-family:arial;font-size:small">PhD
Student</span></div>
<span style="font-family:arial;font-size:small">
<div><span style="font-family:arial;font-size:small"><br>
</span></div>
MRC Laboratory for Molecular Cell Biology</span><br
style="font-family:arial;font-size:small">
<span style="font-family:arial;font-size:small">University
College London</span><br
style="font-family:arial;font-size:small">
<span style="font-family:arial;font-size:small">Gower
Street</span><br
style="font-family:arial;font-size:small">
<span style="font-family:arial;font-size:small">London</span><br
style="font-family:arial;font-size:small">
<span style="font-family:arial;font-size:small">WC1E 6BT</span><br
style="font-family:arial;font-size:small">
<span style="font-family:arial;font-size:small">Telephone:
+44 (0)20 7679 7806</span><br>
</div>
</div>
</div>
<br>
<div class="gmail_quote">On 25 June 2015 at 09:48, Pedro Almada
<span dir="ltr"><<a moz-do-not-send="true"
href="mailto:pedro.almada.13@ucl.ac.uk" target="_blank">pedro.almada.13@ucl.ac.uk</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div>Dear all,</div>
<div><br>
</div>
<div>Some of the members of the UK Super-Resolution
community have discussed the integration of
single-molecule localization microscopy data
(PALM/STORM) into OMERO and what became completely clear
is the need to have a general way to import the output
of the localization algorithms. We've made a push
towards having a general importer and file-standard by
looking at the most common commercial and free software
available. The early results are available as a public <a
moz-do-not-send="true"
href="https://docs.google.com/spreadsheets/d/1YxC_WBFEvgy5jo__0_DaC6B0kzy9ptOhc2Z284PnRhU/edit?usp=sharing"
target="_blank">google doc</a>.</div>
<div><br>
</div>
<div>Our goals are to follow the BioFormats route and:</div>
<div> - Create an importer capable of reading most
localization data</div>
<div> - Establish a standard, flexible format that the
importer would write into.</div>
<div><br>
</div>
<div>It's our hope future algorithms would then start
using a standard format, since this would finally
decouple localization data from the software used to
create it. A few advantages to the SR field would come
from this:</div>
<div> - Analysis software (eg. clustering) would no longer
need to be custom designed for a specific format</div>
<div> - Rendering could finally be decoupled from the
localization software, as the output of any algorithm
should be compatible with any rendering method.</div>
<div> - Comparison of the output of different algorithms
could be efficiently compared analytically and visual
biases from different rendering methods would be
removed.</div>
<div><br>
</div>
<div>We need to publicly discuss how to implement this,
and we believe the OMERO developers community is the
correct starting point. For now, it seems the minimum
headers common to all datasets are X position, Y
position and the frame number of the localized particle.
Other features are immediately useful such as signal
intensity/photon-count but not all formats include it.</div>
<div><br>
</div>
<div>There are also questions of implementation. The
BIG.EPFL people <a moz-do-not-send="true"
href="http://bigwww.epfl.ch/smlm/methods/index.html?p=metrics"
target="_blank">have tackled this problem </a>in a
fashion, by creating an XML generator. It asks the user
to name the headers, column separation, etc and an XML
is created which is used by their comparison software to
read any text based file such as .csv or .xls. However,
some software packages actually generate binary files
and not text based files, which means an extra exporting
step is necessary. Also, XML has an associated overhead.
I would like to see an importer that can handle the
binary data, but have no clue as to how hard that would
be to implement (I can share example outputs though).</div>
<div><br>
</div>
<div>What do you think?</div>
<div><br>
</div>
<div>All the best,</div>
<div>Pedro</div>
<div><br>
PhD Student<br>
Quantitative Imaging and Nanobiophysics Group<br>
MRC Laboratory for Molecular Cell Biology<br>
University College London<br>
Gower Street<br>
London</div>
<div>
<div>
<div dir="ltr"><span
style="font-family:arial;font-size:small">WC1E 6BT</span><br
style="font-family:arial;font-size:small">
<span style="font-family:arial;font-size:small">Telephone:
<a moz-do-not-send="true"
href="tel:%2B44%20%280%2920%207679%207806"
value="+442076797806" target="_blank">+44 (0)20
7679 7806</a></span><br>
</div>
</div>
</div>
<div dir="ltr"><br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
ome-devel mailing list
<a class="moz-txt-link-abbreviated" href="mailto:ome-devel@lists.openmicroscopy.org.uk">ome-devel@lists.openmicroscopy.org.uk</a>
<a class="moz-txt-link-freetext" href="http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel">http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Nico Stuurman
Vale lab, UCSF/HHMI
Genentech Hall, N316, MC2200
600 - 16th Street
San Francisco, CA 94158-2517
415 514 3927
<a class="moz-txt-link-abbreviated" href="mailto:nico.stuurman@ucsf.edu">nico.stuurman@ucsf.edu</a></pre>
</body>
</html>