Hi Sebastian,<br><br>As Jason said, there are several steps in the import process. OMERO.insight is currently limited in which types of
metadata it can show, but that doesn't necessarily mean your metadata has been
dropped. To troubleshoot a problem with a specific piece of metadata from a particular file, the first thing is to identify how far your favorite metadata field got along...<br><br> <b>The Bio-Formats/OMERO Metadata Import Process<br>
</b><br>A. Bio-Formats parses the file's metadata in its original structure.<br><br>B. Bio-Formats converts as much metadata as it can into the OME data model.<br><br>C. The OMERO.importer feeds the standardized OME metadata to the OMERO database.<br>
<br>D. Once import is complete, client applications (e.g., OMERO.insight) can query the metadata field from the database, display it to the user, and otherwise make use of it.<br><br>The good news is: it is very easy to verify whether steps A and B are happening correctly for your metadata of interest, using the Bio-Formats plugins for ImageJ. Here is how you do it!<br>
<br>Let's take Bio-Rad PIC as an example, though this procedure will work for any supported file format.<br><br>1. If you haven't already done so, download loci_tools.jar from the Bio-Formats web site (<a href="http://www.loci.wisc.edu/ome/formats-download.html">http://www.loci.wisc.edu/ome/formats-download.html</a>) and drop it into your ImageJ plugins folder. Make sure you are using the latest trunk build, as it contains the latest bug fixes as well as the "Display OME-XML metadata" feature we will be using.<br>
<br>2. Start up ImageJ.<br><br>3. Choose Plugins > LOCI > Bio-Formats Importer from the menu.<br><br>4. Select the file you wish to test and click OK.<br><br>5. For "View stack with" choose "Metadata only." Uncheck all the options except for "Display metadata in results window" and "Display OME-XML metadata." Click OK and two windows should pop up fairly quickly.<br>
<br>6. To see whether step A is happening, take a look at the "Original Metadata" window.<br><br>For illustration, suppose I am interested in the physical width of my image in microns, and the magnification factor of the lens used to acquire the data. Browsing through the list, I find a key labeled "AXIS_4 distance (X) in microns" that looks pretty promising for the physical width, and another couple of keys labeled "LENS_MAGNIFICATION" and "lens" equal to "10" which matches the 10X lens used during the experiment. So we have determined that Bio-Formats is successfully parsing the metadata I'm interested in.<br>
<br>One important note: please remember that the names of these keys will be different for each file format.<br><br>7. Next we want to check whether step B is happening, so we look at the "OME Metadata" window. Notice that the display is in an XML tree structure.<br>
<br>According to the OME data model, the physical width in microns is saved as an attribute of Pixels beneath Image. Looking at my tree, I see that indeed, the Pixels has a PhysicalSizeX attribute that has been populated.<br>
<br>Next is the lens magnification, which is saved as a property of the instrument's objective. (There are actually two types of lens magnifications that OME can represent: a "nominal" magnification, which is the listed magnification factor of the lens, and a "calibrated" magnification, which is the actual measured magnification, since it can deviate slightly from the nominal factor. In our case, we are looking for the nominal magnification, since the Bio-Rad software merely recorded that I have a 10X lens.) The specific path we want is Instrument -> Objective -> NominalMagnification. But looking at my tree, I see that there is no Instrument node at all, much less an Objective with a NominalMagnification.<br>
<br>So we have verified that Bio-Formats successfully standardized the physical image width, but we also found a limitation: Bio-Formats does not yet standardize lens magnification factor for Bio-Rad PIC files!<br><br>At this point, you have sufficient information to complain to us ;-) about exactly what you need added in order to meet your metadata requirements. Though we are doing our best, it can be very tedious going through each metadata field one by one, which is why we highly value reports from users like you about specifically desired metadata fields. And it helps us to prioritize which file formats and metadata fields to focus on first.<br>
<br>If you prefer the command line to GUIs, it is also possible to follow these same steps using Bio-Formats from the console instead of ImageJ. To do so, download the Bio-Formats command line tools from the web site, and use "showinf -nopix -omexml myFile.pic" to see a dump of both the original metadata key/value pairs, and the standardized OME-XML tree.<br>
<br>Another question you might have is: assuming my metadata field passes steps A and B, how can I tell whether step C was successful?<br><br>Unfortunately I am not an OMERO expert so I don't know an easy way -- but if you are a programmer you can take a look at the source code responsible for it:<br>
<br><a href="https://trac.openmicroscopy.org.uk/omero/browser/trunk/components/importer/src/ome/formats/OMEROMetadataStore.java">https://trac.openmicroscopy.org.uk/omero/browser/trunk/components/importer/src/ome/formats/OMEROMetadataStore.java</a><br>
<br>Doing a text search for "NominalMagnification" I can find a method called "setObjectiveNominalMagnification" which sounds like it might do the right thing, and the code does look like it's actually doing something. In contrast, the method "<span class="nf">setOTFOpticalAxisAveraged</span><span class="o">" has a debug statement that says "TODO" and the </span>"<span class="nf">setExperimentDescription</span><span class="o">" method has a "TODO" comment, so it is probably a safe bet that the OTF optical axis averaging and experiment description </span>fields are not currently getting past step C.<br>
<br>In conclusion, metadata standardization is a massive undertaking and we are hugely appreciative of your feedback. I hope this guide has given you a sense of how to isolate specific things you would like to see added or improved to Bio-Formats and OMERO, so that we can provide you with a suite of software tools that meets your needs.<br>
<br>Cheers,<br>Curtis<br><br><div class="gmail_quote">On Thu, Oct 30, 2008 at 11:54 AM, Jason Swedlow <span dir="ltr"><<a href="mailto:jason@lifesci.dundee.ac.uk">jason@lifesci.dundee.ac.uk</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div style="">
Hi Sebastian-<div><br></div><div>Ok, again; a few problems.</div><div><br></div><div>Many people have noted the limited exposure of metadata in OMERO.insight. The good news is that this is a subset of what is in the database-- so much of what you want is in the DB. But it is not all there-- some metadata is not supported by Bio-formats, some doesn't have a home in OMERO. </div>
<div><br></div><div>This is why we are (as I type) are in the middle of a "metadata completeness" iteration-- from Bio-Formats through to OMERO.insight and OMERO.editor, we are working hard to address a complete metadata workflow. This is a massive undertaking, and much of the team is focussed on it. As an example, it means going through each file format and ensuring we have a home for all metadata somewhere in OMERO. Given the huge disparity in metadata support across different vendor file formats, this takes some work. It also means finding ways to define what people want to see and how, and raises the question of what metadata is editable. We've had long discussions about this for many years, and we also have survey data coming from our partners at Usable Image. We'll provide updates on release of these facilities over the next few days.</div>
<div><br></div><div>It's good to hear the 510 and 710 file formats are behaving similarly-- this is helpful.</div><div><br></div><div>I'll ask Brian to follow-up on your question about Bio-Rad .PIC. </div><div><br>
</div><div>Cheers,</div><div><br></div><div>Jason</div><div><br></div><div><div><div>Sebastian Cachero wrote:</div><br><blockquote type="cite">Hi Jason,<div><br></div><div>We are having the same problem with both types of files.</div>
<div><br></div><div>If I open a 710 or 510 file in imageJ using bio-formats importer and the ask for the file info I get a whole bunch of metadata but after importing them into OMERO I can only see a basic set of data. </div>
<div><br></div><div>A second problem seems to be that the LSM software from Zeiss gives a different voxel size than the metadata retrieved by bio-formats importer/OMERO (the LSM toolbox gives the same voxel size as the LSM soft, I guess that bio-formats might be calculating voxel size differently) .</div>
<div><br></div><div>Lastly, is there anything special about importing the biorad PIC files? I usually have a folder with two pic files (one for each channel) and one data.raw, lse.xml and one experiment.lse files.</div><div>
<br></div><div>Best,</div><div><br></div><div>Sebastian </div><div><div></div><div class="Wj3C7c"><div><br></div><div><br><div><div>On 30 Oct 2008, at 12:23, Jason Swedlow wrote:</div><br><blockquote type="cite"><div style="">
Hi Sebastian-<div><br></div><div>These are problems we are dealing with right now (you have a number of issues, from complete metadata support to number of channels) -- the bad news is the vendors are doing various things in their file formats that we have to deal with. They tailor their software to handle the problem for their file format-- we get to do it for all of them!</div>
<div><br></div><div>A more detailed response will follow, but can you confirm whether the problem you are having with Zeiss is a 510 or 710 file? </div><div><br></div><div>We are having a meeting to discuss all these problems tomorrow and will report back to the community over the weekend.</div>
<div><br></div><div>Thanks!</div><div><br></div><div>Cheers,</div><div><br></div><div>Jason<br><div><div><br></div><div>On 30 Oct 2008, at 12:03, Sebastian Cachero wrote:</div><br><blockquote type="cite">Hi,<div><br></div>
<div>We have just installed omero in the lab and are having a couple of problems while trying to import some confocal stacks from a number of different microscopes (leica sp5, zeiss 510-710, biorad). </div><div><br></div>
<div><div><span style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;"><div style="">
<div>1. For some reason importing of leica .lif files fails (although bioformats-importer has no trouble importing them to imageJ).</div><div><br></div><div>2. Zeiss files are imported correctly but we get only a limited amount of the metadata from the original file.</div>
<div><br></div><div>3. Biorad .pic files get imported but have little metadata (same as zeiss files) and the channel information is incorrect (a two channel file gets imported as 6 channels, each one looking as being one slice offset the others)</div>
<div><br></div><div>Any help with this will be greatly appreciated!</div><div><br></div><div>Thanks!</div><div><br></div><div>Sebastian</div><div><br></div><div>MRC Laboratory of Molecular Biology,<br>Hills Road,<br>Cambridge,<br>
CB2 2QH.<br>United Kingdom.</div><div><br><span style="font-size: 13px;">Office: +44 (0)1223 252944</span></div><div><span style="font-size: 13px;"></span></div><div><span style="font-size: 13px;">Lab<span style="white-space: pre;"> </span>+44 (0)1223 252943</span><br>
Fax: <span style="font-size: 13px;">+44 (0)1223 402310</span><br>E-mail: <a href="mailto:scachero@mrc-lmb.cam.ac.uk" target="_blank">scachero@mrc-lmb.cam.ac.uk</a></div><div><br></div></div></span><br> </div><br></div><div style="margin: 0px;">
_______________________________________________</div><div style="margin: 0px;">ome-devel mailing list</div><div style="margin: 0px;"><a href="mailto:ome-devel@lists.openmicroscopy.org.uk" target="_blank">ome-devel@lists.openmicroscopy.org.uk</a></div>
<div style="margin: 0px;"><a href="http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel" target="_blank">http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel</a></div> </blockquote></div><br><div> <span style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;"><div style="font-family: Helvetica;">
<br><br></div><div style="font-family: Helvetica;"><span style="font-family: Helvetica;">**************************</span></div><div style="font-family: Helvetica;"><font face="Arial">Wellcome Trust Centre for Gene Regulation & Expression</font></div>
<div style="font-family: Helvetica;"><font face="Arial">College of Life Sciences</font></div><div style="font-family: Helvetica;"><font face="Arial">MSI/WTB/JBC Complex</font></div><div style="font-family: Helvetica;"><span style="font-family: Helvetica;">University of Dundee</span></div>
<div style="font-family: Helvetica;"><span style="font-family: Helvetica;">Dow Street</span></div><div style="font-family: Helvetica;"><span style="font-family: Helvetica;">Dundee DD1 5EH</span></div><div style="font-family: Helvetica;">
<span style="font-family: Helvetica;">United Kingdom</span></div><div style="font-family: Helvetica;"><br style="font-family: Helvetica;"></div><div style="font-family: Helvetica;"><span style="font-family: Helvetica;">phone (01382) 385819</span></div>
<div style="font-family: Helvetica;"><span style="font-family: Helvetica;">Intl phone: 44 1382 385819 </span></div><div style="font-family: Helvetica;"><span style="font-family: Helvetica;">FAX (01382) 388072 </span></div>
<div style="font-family: Helvetica;"><span style="font-family: Helvetica;">email: <a href="mailto:jason@lifesci.dundee.ac.uk" target="_blank">jason@lifesci.dundee.ac.uk</a></span></div><div style="font-family: Helvetica;">
<br style="font-family: Helvetica;"></div><div style="font-family: Helvetica;"><span style="font-family: Helvetica;">Lab Page: <a href="http://www.dundee.ac.uk/lifesciences/swedlow/" target="_blank">http://www.dundee.ac.uk/lifesciences/swedlow/</a></span></div>
<div style="font-family: Helvetica;"><span style="font-family: Helvetica;">Open Microscopy Environment: <a href="http://openmicroscopy.org" target="_blank">http://openmicroscopy.org</a></span></div><div style="font-family: Helvetica;">
<span style="font-family: Helvetica;">**************************</span></div><div style="font-family: Helvetica;"><br></div><div style="font-family: Helvetica;"><div>The University of Dundee is a Scottish Registered Charity, No. SC015096.</div>
</div><br></span><br> </div><br></div></div></blockquote></div><br><div> <span style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;"><div style="">
<div>MRC Laboratory of Molecular Biology,<br>Hills Road,<br>Cambridge,<br>CB2 2QH.<br>United Kingdom.</div><div><br><span style="font-size: 13px;">Office: +44 (0)1223 252944</span></div><div><span style="font-size: 13px;"></span></div>
<div><span style="font-size: 13px;">Lab<span style="white-space: pre;"> </span>+44 (0)1223 252943</span><br>Fax: <span style="font-size: 13px;">+44 (0)1223 402310</span><br>E-mail: <a href="mailto:scachero@mrc-lmb.cam.ac.uk" target="_blank">scachero@mrc-lmb.cam.ac.uk</a></div>
<div><br></div></div></span><br> </div><br></div></div></div></blockquote></div><br><div> <span style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;"><div style="font-family: Helvetica;">
<br><br></div><div class="Ih2E3d"><div style="font-family: Helvetica;"><span style="font-family: Helvetica;">**************************</span></div><div style="font-family: Helvetica;"><font face="Arial">Wellcome Trust Centre for Gene Regulation & Expression</font></div>
<div style="font-family: Helvetica;"><font face="Arial">College of Life Sciences</font></div><div style="font-family: Helvetica;"><font face="Arial">MSI/WTB/JBC Complex</font></div><div style="font-family: Helvetica;"><span style="font-family: Helvetica;">University of Dundee</span></div>
<div style="font-family: Helvetica;"><span style="font-family: Helvetica;">Dow Street</span></div><div style="font-family: Helvetica;"><span style="font-family: Helvetica;">Dundee DD1 5EH</span></div><div style="font-family: Helvetica;">
<span style="font-family: Helvetica;">United Kingdom</span></div><div style="font-family: Helvetica;"><br style="font-family: Helvetica;"></div><div style="font-family: Helvetica;"><span style="font-family: Helvetica;">phone (01382) 385819</span></div>
<div style="font-family: Helvetica;"><span style="font-family: Helvetica;">Intl phone: 44 1382 385819 </span></div><div style="font-family: Helvetica;"><span style="font-family: Helvetica;">FAX (01382) 388072 </span></div>
<div style="font-family: Helvetica;"><span style="font-family: Helvetica;">email: <a href="mailto:jason@lifesci.dundee.ac.uk" target="_blank">jason@lifesci.dundee.ac.uk</a></span></div><div style="font-family: Helvetica;">
<br style="font-family: Helvetica;"></div><div style="font-family: Helvetica;"><span style="font-family: Helvetica;">Lab Page: <a href="http://www.dundee.ac.uk/lifesciences/swedlow/" target="_blank">http://www.dundee.ac.uk/lifesciences/swedlow/</a></span></div>
<div style="font-family: Helvetica;"><span style="font-family: Helvetica;">Open Microscopy Environment: <a href="http://openmicroscopy.org" target="_blank">http://openmicroscopy.org</a></span></div><div style="font-family: Helvetica;">
<span style="font-family: Helvetica;">**************************</span></div><div style="font-family: Helvetica;"><br></div><div style="font-family: Helvetica;"><div>The University of Dundee is a Scottish Registered Charity, No. SC015096.</div>
</div><br></div></span><br> </div><br></div></div><br>_______________________________________________<br>
ome-users mailing list<br>
<a href="mailto:ome-users@lists.openmicroscopy.org.uk">ome-users@lists.openmicroscopy.org.uk</a><br>
<a href="http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users" target="_blank">http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users</a><br>
<br></blockquote></div><br>