[ome-devel] Super-Resolution standard format

Fri Jul 31 11:17:44 BST 2015

Hi Ian

It sounds like Ann has a lot of data, so if it requires you to do extra work let's wait.

Though here's a crazy idea. Have you comes across http://nucleotid.es/ or http://bioboxes.org/ (running bioinformatics algorithms in Docker)? You could create something similar for testing SR reconstruction algorithms...

Simon

On 30 July 2015 at 18:24, Munro, Ian <i.munro at imperial.ac.uk<mailto:i.munro at imperial.ac.uk>> wrote:
Hi All

As I understand it the raw data (ome-tiff) that I uploaded last week could, potentially, be processed with packages other than Thunderstorm to produce localisation data
in other formats.

Simon: let me know if you think that would be useful and I’ll lean on Kwasi to try.

Ian

On 30 Jul 2015, at 15:53, Seamus Holden <seamus.holden at gmail.com<mailto:seamus.holden at gmail.com>> wrote:

Unfortunately no, not to hand.

Frame offsets: you can pretty reliably just look at what the frame number in the first line of the dataset is - it is usually unlikely there will be a completely empty first frame. Not bulletproof obviously.

Pixel offsets: I will get back to you on this. It was a required part of the LM challenge so Daniel Sage should have the data; I will ask him.

On Thu, 30 Jul 2015 at 11:09 Simon Li <spli at dundee.ac.uk<mailto:spli at dundee.ac.uk>> wrote:
Seamus: Do you have the image files used to generate those particle localisations? It's not essential, but it'd be useful.

Ann: it'd be useful to have as many different formats and types of superresolution data as possible, ideally the input image and processed output, and if one format has multiple variants it'd be useful to have them all. You can upload them to https://www.openmicroscopy.org/qa2/qa/upload/ or if they're very large contact us directly and we can arrange something else.

Everyone: Reiterating Seamus and Ian's points about frame and distance offsets, does anyone have this information for different formats? For example where is the origin relative to the image (top-left, centre), are measurements from the centre or corner of a pixel, 0 or 1 based indexing, etc.

Cheers

Simon

On 29 July 2015 at 17:26, Seamus Holden <seamus.holden at gmail.com<mailto:seamus.holden at gmail.com>> wrote:
I have just uploaded output examples of :

PeakSelector - Harold Hess NIH software
QuickPALM
RapidSTORM (v 2 & v3)
uTrack - http://lccb.hms.harvard.edu/software.html
Vutara
DAOSTORM

I have also uploaded a Leica GSD file specification but I don't seem to have an example file any more.

Mixture of 2D and 3D data. All single colour. RapidSTORM supports a multicolour output format too, but I don't have any examples.

Hope that helps.

Best
Seamus

On Mon, 27 Jul 2015 at 10:19 Munro, Ian <i.munro at imperial.ac.uk<mailto:i.munro at imperial.ac.uk>> wrote:
Nice work Simon

Can I draw your attention to a subtlety that I would have missed (from a comment that Seamus made earlier in the thread)

Oh - also - don't forget about the coordinate system of the different softwares - does 0,0 define the corner of a pixel or the middle. Inconsistently chosen between different softwares, and crucial when you start comparing algorithms.

I guess it’s also possible that  some of the systems might use 1-n rather than 0-n-1 numbering for either pixels  or frames.

Ian

On 24 Jul 2015, at 19:25, Seamus Holden <seamus.holden at gmail.com<mailto:seamus.holden at gmail.com>> wrote:

Nice! HD5 makes a lot of sense, great idea! Much more compact than XML.

Next week, I promise I will submit the various multi-format test data from lots of different software that we used in PALMsiever development (if I post this then it will force me to do so!).

Best wishes
Seamus

On Fri, 24 Jul 2015 at 18:24 Simon Li <spli at dundee.ac.uk<mailto:spli at dundee.ac.uk>> wrote:
Hi all

Based on the "Common Headers" section of the spreadsheet that Pedro's shared I've written an example python script for storing some of this data in OMERO as a HDF5 table:

https://github.com/manics/omero-superresolution-tables

This isn't in any way complete, and there's plenty of scope for extending it, and improving it's useability, but I just throught I'd make a start since Ian Munro provided us with some real data.

There are some usage instructions in the README file. The sample file is an extract of a larger file from Imperial- I'm currently trying the script on the full file file with 4 million rows (note some optimisation might be needed on the OMERO side, it's taking a while...). Next step is to consider adding the points as ROIs in OMERO.

Enjoy your weekend!

Simon

On 25 June 2015 at 09:48, Pedro Almada <pedro.almada.13 at ucl.ac.uk<mailto:pedro.almada.13 at ucl.ac.uk>> wrote:
Dear all,

Some of the members of the UK Super-Resolution community have discussed the integration of single-molecule localization microscopy data (PALM/STORM) into OMERO and what became completely clear is the need to have a general way to import the output of the localization algorithms. We've made a push towards having a general importer and file-standard by looking at the most common commercial and free software available. The early results are available as a public google doc<https://docs.google.com/spreadsheets/d/1YxC_WBFEvgy5jo__0_DaC6B0kzy9ptOhc2Z284PnRhU/edit?usp=sharing>.

Our goals are to follow the BioFormats route and:
 - Create an importer capable of reading most localization data
 - Establish a standard, flexible format that the importer would write into.

It's our hope future algorithms would then start using a standard format, since this would finally decouple localization data from the software used to create it. A few advantages to the SR field would come from this:
 - Analysis software (eg. clustering) would no longer need to be custom designed for a specific format
 - Rendering could finally be decoupled from the localization software, as the output of any algorithm should be compatible with any rendering method.
 - Comparison of the output of different algorithms could be efficiently compared analytically and visual biases from different rendering methods would be removed.

We need to publicly discuss how to implement this, and we believe the OMERO developers community is the correct starting point. For now, it seems the minimum headers common to all datasets are X position, Y position and the frame number of the localized particle. Other features are immediately useful such as signal intensity/photon-count but not all formats include it.

There are also questions of implementation. The BIG.EPFL people have tackled this problem <http://bigwww.epfl.ch/smlm/methods/index.html?p=metrics> in a fashion, by creating an XML generator. It asks the user to name the headers, column separation, etc and an XML is created which is used by their comparison software to read any text based file such as .csv or .xls. However, some software packages actually generate binary files and not text based files, which means an extra exporting step is necessary. Also, XML has an associated overhead. I would like to see an importer that can handle the binary data, but have no clue as to how hard that would be to implement (I can share example outputs though).

What do you think?

All the best,
Pedro

PhD Student
Quantitative Imaging and Nanobiophysics Group
MRC Laboratory for Molecular Cell Biology
University College London
Gower Street
London
WC1E 6BT
Telephone: +44 (0)20 7679 7806<tel:%2B44%20%280%2920%207679%207806>

The University of Dundee is a registered Scottish Charity, No: SC015096

_______________________________________________
ome-devel mailing list
ome-devel at lists.openmicroscopy.org.uk<mailto:ome-devel at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel

The University of Dundee is a registered Scottish Charity, No: SC015096
_______________________________________________
ome-devel mailing list
ome-devel at lists.openmicroscopy.org.uk<mailto:ome-devel at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel

_______________________________________________
ome-devel mailing list
ome-devel at lists.openmicroscopy.org.uk<mailto:ome-devel at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel

The University of Dundee is a registered Scottish Charity, No: SC015096

_______________________________________________
ome-devel mailing list
ome-devel at lists.openmicroscopy.org.uk<mailto:ome-devel at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel

The University of Dundee is a registered Scottish Charity, No: SC015096
_______________________________________________
ome-devel mailing list
ome-devel at lists.openmicroscopy.org.uk<mailto:ome-devel at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel

The University of Dundee is a registered Scottish Charity, No: SC015096

_______________________________________________
ome-devel mailing list
ome-devel at lists.openmicroscopy.org.uk<mailto:ome-devel at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel

The University of Dundee is a registered Scottish Charity, No: SC015096
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20150731/595eaa83/attachment-0001.html>