[ome-devel] Super-Resolution standard format

[Munro, Ian]

Nice work Simon

Can I draw your attention to a subtlety that I would have missed (from a comment that Seamus made earlier in the thread)

Oh - also - don't forget about the coordinate system of the different softwares - does 0,0 define the corner of a pixel or the middle. Inconsistently chosen between different softwares, and crucial when you start comparing algorithms.

I guess it’s also possible that  some of the systems might use 1-n rather than 0-n-1 numbering for either pixels  or frames.

Ian


On 24 Jul 2015, at 19:25, Seamus Holden <seamus.holden at gmail.com<mailto:seamus.holden at gmail.com>> wrote:

Nice! HD5 makes a lot of sense, great idea! Much more compact than XML.

Next week, I promise I will submit the various multi-format test data from lots of different software that we used in PALMsiever development (if I post this then it will force me to do so!).

Best wishes
Seamus

On Fri, 24 Jul 2015 at 18:24 Simon Li <spli at dundee.ac.uk<mailto:spli at dundee.ac.uk>> wrote:
Hi all

Based on the "Common Headers" section of the spreadsheet that Pedro's shared I've written an example python script for storing some of this data in OMERO as a HDF5 table:

https://github.com/manics/omero-superresolution-tables

This isn't in any way complete, and there's plenty of scope for extending it, and improving it's useability, but I just throught I'd make a start since Ian Munro provided us with some real data.

There are some usage instructions in the README file. The sample file is an extract of a larger file from Imperial- I'm currently trying the script on the full file file with 4 million rows (note some optimisation might be needed on the OMERO side, it's taking a while...). Next step is to consider adding the points as ROIs in OMERO.

Enjoy your weekend!

Simon



On 25 June 2015 at 09:48, Pedro Almada <pedro.almada.13 at ucl.ac.uk<mailto:pedro.almada.13 at ucl.ac.uk>> wrote:
Dear all,

Some of the members of the UK Super-Resolution community have discussed the integration of single-molecule localization microscopy data (PALM/STORM) into OMERO and what became completely clear is the need to have a general way to import the output of the localization algorithms. We've made a push towards having a general importer and file-standard by looking at the most common commercial and free software available. The early results are available as a public google doc<https://docs.google.com/spreadsheets/d/1YxC_WBFEvgy5jo__0_DaC6B0kzy9ptOhc2Z284PnRhU/edit?usp=sharing>.

Our goals are to follow the BioFormats route and:
 - Create an importer capable of reading most localization data
 - Establish a standard, flexible format that the importer would write into.

It's our hope future algorithms would then start using a standard format, since this would finally decouple localization data from the software used to create it. A few advantages to the SR field would come from this:
 - Analysis software (eg. clustering) would no longer need to be custom designed for a specific format
 - Rendering could finally be decoupled from the localization software, as the output of any algorithm should be compatible with any rendering method.
 - Comparison of the output of different algorithms could be efficiently compared analytically and visual biases from different rendering methods would be removed.

We need to publicly discuss how to implement this, and we believe the OMERO developers community is the correct starting point. For now, it seems the minimum headers common to all datasets are X position, Y position and the frame number of the localized particle. Other features are immediately useful such as signal intensity/photon-count but not all formats include it.

There are also questions of implementation. The BIG.EPFL people have tackled this problem <http://bigwww.epfl.ch/smlm/methods/index.html?p=metrics> in a fashion, by creating an XML generator. It asks the user to name the headers, column separation, etc and an XML is created which is used by their comparison software to read any text based file such as .csv or .xls. However, some software packages actually generate binary files and not text based files, which means an extra exporting step is necessary. Also, XML has an associated overhead. I would like to see an importer that can handle the binary data, but have no clue as to how hard that would be to implement (I can share example outputs though).

What do you think?

All the best,
Pedro

PhD Student
Quantitative Imaging and Nanobiophysics Group
MRC Laboratory for Molecular Cell Biology
University College London
Gower Street
London
WC1E 6BT
Telephone: +44 (0)20 7679 7806<tel:%2B44%20%280%2920%207679%207806>


The University of Dundee is a registered Scottish Charity, No: SC015096

_______________________________________________
ome-devel mailing list
ome-devel at lists.openmicroscopy.org.uk<mailto:ome-devel at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel


The University of Dundee is a registered Scottish Charity, No: SC015096
_______________________________________________
ome-devel mailing list
ome-devel at lists.openmicroscopy.org.uk<mailto:ome-devel at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20150727/2e0ab1c3/attachment.html>