[ome-users] ND2 meta data with bioformats 5.0.4

[Enno R. Oldewurtel]

Hi Melissa,

On Fri, Sep 12, 2014 at 3:44 AM, Melissa Linkert <
melissa at glencoesoftware.com> wrote:

> Hi Enno,
>
> > What I need to do is to retrieve the positions of the multipoints in an
> nd2
> > file in order to know how to stitch them together.
> > I have nd2 files with 120 multipoints, several time-points and 2-3
> channels.
> > I usually only retrieve the positions of the first time-point, since each
> > multipoint should be at the same position at following times. In my first
> > e-mail I outlined how to retrieve positions using the entries from
> > r.getGlobalMetadata().
> > Now I tried the same using the commands you gave me:
> >
> > r.setMetadataStore(loci.formats.MetadataTools.createOMEXMLMetadata());
> > r.setID();
> > metadata = r.getMetadataStore();
> > planeIndex=0;
> > for positionIndex=1:120
> >    xPosition(i) = metadata.getPlanePositionX(positionIndex, planeIndex);
> >    yPosition(i) = metadata.getPlanePositionY(positionIndex, planeIndex);
> > end
> >
> > However, I do not get the expected positions. For clarity I attach plots
> of
> > the x and y positions retrieved with versions 5.0.3 and 5.0.4 and also
> the
> > expected plot retrieved via my old method with version 5.0.3 and another
> > plot with wrong positions retrieved with my old method and version 5.0.4.
> >
> > I know that I should get a pattern like in the lower left plot that I
> > attached. Can you help me once more how to call the correct positions?
>
> The only thing that you might try here is calling:
>
>   xPosition(i) = metadata.getPlanePositionX(positionIndex - 1, planeIndex);
>   yPosition(i) = metadata.getPlanePositionY(positionIndex - 1, planeIndex);
>

Sorry for that. This was only a typo in the e-mail. In my test I started
the index with 0.

>
> as the 'metadata' object is assuming that the index values start at 0.
> Otherwise, that all seems correct for getting the positions for the
> first timepoint and channel.  If changing that index doesn't fully solve
> the problem, then this is most likely a bug in the metadata population.
> In that case, it would be very much appreciated if you could upload one
> of the files to http://qa.openmicroscopy.org/qa/upload.
>

I tested different nd2 files. With bioformats 5.0.4 I could actually
retrieve the right positions, but only if there was only one z and
time-point. For higher t, I got unexpected positions. I uploaded an example
nd2 http://qa.openmicroscopy.org.uk/qa/feedback/9529/ that was small enough
(I was a bit too quick in clicking buttons and created an empty QA Bug 9528
that should be ignored). The error seems to depend on the actual number of
time-points, too, since the pattern seemed to change depending on how many
time-points I had exactly.
I attached a plot of the correct x,y positions of the uploaded nd2 file.

>
> > I understand that the positionIndex should be the index for differnt
> > multipoints, but how does the planeIndex look like for different
> > time-points and channels?
>
> For subsequent timepoints and channels, planeIndex can be calculated:
>
>   timepoint = 2; % the timepoint to select, starting from 0
>   channel = 1; % the channel to select, starting from 0
>   zSection = 0; % the Z section to select (probably not applicable)
>   planeIndex = FormatTools.getIndex(r, zSection, channel, timepoint);
>
> > The next bit is not as important to me, but still with regards to the PFS
> > status: PFS offset and state are not quite sufficient for me. During my
> > experiments the PFS often fails to correct to the proper position and
> > resultingly there can be different status returns indicating if the PFS
> was
> > switched off, still searching, not finding, or actually finding the
> correct
> > z-position (maybe there are even more status possibilities, from the top
> of
> > my head I know that 0,1,5 and 7 are possible values). With NIS elements I
> > can take a look at those status values for each image to see what went
> > wrong. It would be nice if this information was to be retrieved by
> > bioformats, too. Also, I often have differnt PFS offsets, e.g. at
> different
> > multipoints. Hence, a single PFS offset value for the entire experiment
> > does not tell me too much.
>
> OK, thanks for clarifying.  There is now a ticket on our issue tracking
> system for fully parsing PFS values:
>
> http://trac.openmicroscopy.org.uk/ome/ticket/12560
>
> If you would like to be automatically notified of updates to the ticket,
> please let us know.
>
It would be nice to get notified. Also, I want to emphasize that Nikon
seems to distinguish between the PFS state and status. I guess the former
can be just true or false, but the latter carries more information (see my
last e-mail).

Many thanks,
Enno

>
> Regards,
> -Melissa
>
> On Tue, Sep 09, 2014 at 01:46:31PM +0200, Enno R. Oldewurtel wrote:
> > Hi Melissa,
> >
> > thank you for your reply. I managed to use the commands you recommended.
> > It, also, helped me to look at an m-file by Sebastien Besson (
> > https://gist.github.com/sbesson/6529406) to learn how to use the
> commands.
> > However, I still have some problems. Maybe I still do not understand how
> to
> > call the right plane with the right index.
> >
> >
> > What I need to do is to retrieve the positions of the multipoints in an
> nd2
> > file in order to know how to stitch them together.
> > I have nd2 files with 120 multipoints, several time-points and 2-3
> channels.
> > I usually only retrieve the positions of the first time-point, since each
> > multipoint should be at the same position at following times. In my first
> > e-mail I outlined how to retrieve positions using the entries from
> > r.getGlobalMetadata().
> > Now I tried the same using the commands you gave me:
> >
> > r.setMetadataStore(loci.formats.MetadataTools.createOMEXMLMetadata());
> > r.setID();
> > metadata = r.getMetadataStore();
> > planeIndex=0;
> > for positionIndex=1:120
> >    xPosition(i) = metadata.getPlanePositionX(positionIndex, planeIndex);
> >    yPosition(i) = metadata.getPlanePositionY(positionIndex, planeIndex);
> > end
> >
> > However, I do not get the expected positions. For clarity I attach plots
> of
> > the x and y positions retrieved with versions 5.0.3 and 5.0.4 and also
> the
> > expected plot retrieved via my old method with version 5.0.3 and another
> > plot with wrong positions retrieved with my old method and version 5.0.4.
> >
> > I know that I should get a pattern like in the lower left plot that I
> > attached. Can you help me once more how to call the correct positions? I
> > understand that the positionIndex should be the index for differnt
> > multipoints, but how does the planeIndex look like for different
> > time-points and channels?
> >
> > The next bit is not as important to me, but still with regards to the PFS
> > status: PFS offset and state are not quite sufficient for me. During my
> > experiments the PFS often fails to correct to the proper position and
> > resultingly there can be different status returns indicating if the PFS
> was
> > switched off, still searching, not finding, or actually finding the
> correct
> > z-position (maybe there are even more status possibilities, from the top
> of
> > my head I know that 0,1,5 and 7 are possible values). With NIS elements I
> > can take a look at those status values for each image to see what went
> > wrong. It would be nice if this information was to be retrieved by
> > bioformats, too. Also, I often have differnt PFS offsets, e.g. at
> different
> > multipoints. Hence, a single PFS offset value for the entire experiment
> > does not tell me too much.
> >
> >
> > Many thanks,
> > Enno
> >
> >
> > On Tue, Sep 9, 2014 at 12:23 AM, Melissa Linkert <
> > melissa at glencoesoftware.com> wrote:
> >
> > > Hi Enno,
> > >
> > > > I use bioformats.jar with Matlab to read in ND2 files. Previously I
> used
> > > a
> > > > 5.0.0-DEV version of loci_tools.
> > > > With bioformats I have some problems to retrieve meta data from ND2
> > > files.
> > > > With 5.0.3 I can successfully read x,y,z position data, but with
> 5.0.4 I
> > > > get some discrepancies.
> > > > With both bioformat versions I fail to get the time values of each
> image.
> > > > However, with loci_tools I was able to do this.
> > > >
> > > > Below is my current approach to read meta data from an ND2 file.
> Maybe
> > > > someone can point out to me how to do this differently, or how to
> fix the
> > > > code I use?
> > >
> > > The best way to retrieve the stage position and timestamp data is as
> > > follows:
> > >
> > > %%%
> > > % positionIndex selects which multipoint to use (starting from 0)
> > > % planeIndex selects the image in the chosen multipoint (starting from
> 0)
> > > metadata = r.getMetadataStore();
> > > xPosition = r.getPlanePositionX(positionIndex, planeIndex);
> > > timestamp = r.getPlaneDeltaT(positionIndex, planeIndex);
> > > acquisitionStart = r.getImageAcquisitionDate(positionIndex);
> > > %%%
> > >
> > > 'xPosition' will be the stage position along X; the Y and Z positions
> > > can be retrieved using similar calls to getPlanePositionY and
> > > getPlanePositionZ.  'timestamp' is the time in seconds from the
> > > beginning of acquisition ('acquisitionStart').  See also:
> > >
> > >
> > >
> http://downloads.openmicroscopy.org/bio-formats/5.0.4/api/ome/xml/meta/MetadataRetrieve.html
> > >
> > >
> http://www.openmicroscopy.org/Schemas/Documentation/Generated/OME-2013-06/ome_xsd.html#Plane
> > >
> > > getGlobalMetadata and getSeriesMetadata are provided for informational
> > > purposes, but are not necessarily guaranteed to have the same key/value
> > > mappings between releases.
> > >
> > > > Also, is it possible to retrieve the PFS status of each image plane
> from
> > > an
> > > > ND2 file?
> > >
> > > PFS status is not included in the OME data model; from the files we
> > > have, it looks like the "PFS, State" and "PFS, Offset" keys in the
> > > global metadata table will show the state and offset.  If that doesn't
> > > work,
> > > though, please let us know.
> > >
> > > Regards,
> > > -Melissa
> > >
> > > On Thu, Sep 04, 2014 at 06:09:16PM +0200, Enno R. Oldewurtel wrote:
> > > > Dear all,
> > > >
> > > > I use bioformats.jar with Matlab to read in ND2 files. Previously I
> used
> > > a
> > > > 5.0.0-DEV version of loci_tools.
> > > > With bioformats I have some problems to retrieve meta data from ND2
> > > files.
> > > > With 5.0.3 I can successfully read x,y,z position data, but with
> 5.0.4 I
> > > > get some discrepancies.
> > > > With both bioformat versions I fail to get the time values of each
> image.
> > > > However, with loci_tools I was able to do this.
> > > >
> > > > Below is my current approach to read meta data from an ND2 file.
> Maybe
> > > > someone can point out to me how to do this differently, or how to
> fix the
> > > > code I use?
> > > > Also, is it possible to retrieve the PFS status of each image plane
> from
> > > an
> > > > ND2 file?
> > > >
> > > >
> > > > I used to retrieve e.g the x-position of a specific plane from an ND2
> > > file
> > > > in the following way:
> > > >
> > > > raw=r.getGlobalMetadata();
> > > > x=raw.get('X position for position #00083');
> > > > This is for an ND2 file with 82 timepoints, 2 channels and 120
> > > multipoints,
> > > > i.e. the highest index was #09840 (for some reason I had to assume
> > > > 82*120*2channels as the highest index for zero padding, starting from
> > > > #09841 to #19680 all values are identical)
> > > >
> > > > However, with bioformats 5.0.4 this gives me the x-value of the first
> > > > time-point instead of the 83rd. Similarly, in another experiment
> with 65
> > > > timepoints, the string 'X position for position #00066' gives me
> again
> > > the
> > > > value of the first time-point instead of the 66th.
> > > >
> > > > Is this a bug in the new 5.0.4 version? I think this happened to me
> > > before
> > > > with some loci_tools version, but in a later version my approach
> worked
> > > > again. For bioformats 5.0.3 I can retrieve the mentioned x positions
> just
> > > > fine.
> > > >
> > > > I also need to retrieve the time data for each image. I used to work
> with
> > > > loci_tools (daily built from 6th September 2013) with
> > > > rawSeries=r.getSeriesMetadata();
> > > > t=rawSeries.get('timestamp #0001')
> > > > (less zero-padding was needed, as this time the highest index was
> > > > 82*120=9840, also getSeriesMetadata is used since getMetadata takes
> very
> > > > long to load)
> > > >
> > > > This worked fine, but since there were some bug fixes for ND2 files
> and
> > > > r.setID() runs significantly faster, I would like to stay with
> bioformats
> > > > instead of loci_tools.
> > > >
> > > >
> > > > Best regards,
> > > > Enno
> > > >
> > > >
> > > > --
> > > > Enno R. Oldewurtel
> > > > AG Prof. B. Maier / Biophysics Group
> > > > Biozentrum
> > > > Universitaet zu Koeln
> > > > Zuelpicher Str. 47b
> > > > 50674 Koeln
> > > >
> > > > enno.oldewurtel at uni-koeln.de
> > > > Tel: +49-221-470-8048
> > > > Fax: +49-221-470-6230
> > > > http://www.biophysics.uni-koeln.de/
> > >
> > > > _______________________________________________
> > > > ome-users mailing list
> > > > ome-users at lists.openmicroscopy.org.uk
> > > > http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
> > >
> > >
> >
> >
> > --
> > Enno R. Oldewurtel
> > AG Prof. B. Maier / Biophysics Group
> > Biozentrum
> > Universitaet zu Koeln
> > Zuelpicher Str. 47b
> > 50674 Koeln
> >
> > enno.oldewurtel at uni-koeln.de
> > Tel: +49-221-470-8048
> > Fax: +49-221-470-6230
> > http://www.biophysics.uni-koeln.de/
>
>
>


-- 
Enno R. Oldewurtel
AG Prof. B. Maier / Biophysics Group
Biozentrum
Universitaet zu Koeln
Zuelpicher Str. 47b
50674 Koeln

enno.oldewurtel at uni-koeln.de
Tel: +49-221-470-8048
Fax: +49-221-470-6230
http://www.biophysics.uni-koeln.de/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20140912/2c75311d/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot_xyPositionPlot.png
Type: image/png
Size: 4701 bytes
Desc: not available
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20140912/2c75311d/attachment.png>