[ome-users] ND2 position meta data with bioformats 5.1.1
Enno R. Oldewurtel
enno.oldewurtel at uni-koeln.de
Tue May 5 14:11:04 BST 2015
Dear Balaji and Melissa,
firstly thanks for your great effort regarding bioformats! Following is a
problem, which I believe could be another bug:
last year I submitted a problem about wrong indexing of x,y data of nd2
files (#12569: BUG: ND2 files - x,y position of the different multi points
returned in the wrong order).
I still use matlab for loading nd2 files and retrieve metadata with
commands like:
r = loci.formats.ChannelFiller();
r = loci.formats.ChannelSeparator(r);
r.setMetadataStore(loci.formats.MetadataTools.createOMEXMLMetadata());
r.setId( FileName);
metadata=r.getMetadataStore();
i=positionIndex;
j=loci.formats.FormatTools.getIndex(r, z, channel, time)
x=double(metadata.getPlanePositionX(i,j).value)
t=double(metadata.getPlaneDeltaT(i,j).value)
Position data for x,y,z retrieved with 5.1.1 for the nd2 file I submitted
with my previous problem, are correct now. However, time-points retrieved
with metadata.getPlaneDeltaT(i,j) in the same manner are in the wrong order.
Furthermore the position data for nd2 files acquired with several z-slices
is wrongly indexed. I have an example nd2 files with T(1) x XY(8) x Z(4)
dimensions. Version 5.0.3 shows the correct order of 4 z-steps at the first
XY position, which is then repeated at the remaining 7 positions. Version
5.1.1 shows the x,y,z values jumping about in the wrong order.
As an aside: I noted that with 5.1.1 the number of the dimensions of nd2
files with all four T, XY, Z and lambda dimensions are correctly
identified. Previously I could only use bioformats with 3 different types
of dimensions. However, as soon as an nd2 files is modified and saved in
NIS elements, the numbers of the dimensions are not correct anymore.
I can supply example nd2 files if needed, but will need to upload them via
the qa system.
Hope this helps, and again many thanks for maintaining and developing such
a great product!
Best,
Enno
On Thu, Sep 18, 2014 at 4:59 PM, Balaji Ramalingam <
b.ramalingam at dundee.ac.uk> wrote:
> Hi,
>
> Thank you for submitting your file to our QA system.
>
> We were able to reproduce the bug locally,
> and a ticket has been filed on this regard.
> https://trac.openmicroscopy.org.uk/ome/ticket/12569
>
> You have been added to the CC list, and will get notified when there is
> an update.
>
> Best,
> Balaji
>
> __________________
>
> Mr Balaji Ramalingam
>
> Software Developer
>
> OME Team
>
> College of Life Sciences
>
> University of Dundee
>
> From: "Enno R. Oldewurtel" <enno.oldewurtel at uni-koeln.de>
> Date: Friday, 12 September 2014 10:28
> To: Melissa Linkert <melissa at glencoesoftware.com>
> Cc: "OME-users ome-users at lists.openmicroscopy.org.uk" <
> ome-users at lists.openmicroscopy.org.uk>
> Subject: Re: [ome-users] ND2 meta data with bioformats 5.0.4
>
> Hi Melissa,
>
> On Fri, Sep 12, 2014 at 3:44 AM, Melissa Linkert <
> melissa at glencoesoftware.com> wrote:
>
>> Hi Enno,
>>
>> > What I need to do is to retrieve the positions of the multipoints in an
>> nd2
>> > file in order to know how to stitch them together.
>> > I have nd2 files with 120 multipoints, several time-points and 2-3
>> channels.
>> > I usually only retrieve the positions of the first time-point, since
>> each
>> > multipoint should be at the same position at following times. In my
>> first
>> > e-mail I outlined how to retrieve positions using the entries from
>> > r.getGlobalMetadata().
>> > Now I tried the same using the commands you gave me:
>> >
>> > r.setMetadataStore(loci.formats.MetadataTools.createOMEXMLMetadata());
>> > r.setID();
>> > metadata = r.getMetadataStore();
>> > planeIndex=0;
>> > for positionIndex=1:120
>> > xPosition(i) = metadata.getPlanePositionX(positionIndex, planeIndex);
>> > yPosition(i) = metadata.getPlanePositionY(positionIndex, planeIndex);
>> > end
>> >
>> > However, I do not get the expected positions. For clarity I attach
>> plots of
>> > the x and y positions retrieved with versions 5.0.3 and 5.0.4 and also
>> the
>> > expected plot retrieved via my old method with version 5.0.3 and another
>> > plot with wrong positions retrieved with my old method and version
>> 5.0.4.
>> >
>> > I know that I should get a pattern like in the lower left plot that I
>> > attached. Can you help me once more how to call the correct positions?
>>
>> The only thing that you might try here is calling:
>>
>> xPosition(i) = metadata.getPlanePositionX(positionIndex - 1,
>> planeIndex);
>> yPosition(i) = metadata.getPlanePositionY(positionIndex - 1,
>> planeIndex);
>>
>
> Sorry for that. This was only a typo in the e-mail. In my test I started
> the index with 0.
>
>>
>> as the 'metadata' object is assuming that the index values start at 0.
>> Otherwise, that all seems correct for getting the positions for the
>> first timepoint and channel. If changing that index doesn't fully solve
>> the problem, then this is most likely a bug in the metadata population.
>> In that case, it would be very much appreciated if you could upload one
>> of the files to http://qa.openmicroscopy.org/qa/upload.
>>
>
> I tested different nd2 files. With bioformats 5.0.4 I could actually
> retrieve the right positions, but only if there was only one z and
> time-point. For higher t, I got unexpected positions. I uploaded an example
> nd2 http://qa.openmicroscopy.org.uk/qa/feedback/9529/ that was small
> enough (I was a bit too quick in clicking buttons and created an empty QA
> Bug 9528 that should be ignored). The error seems to depend on the actual
> number of time-points, too, since the pattern seemed to change depending on
> how many time-points I had exactly.
> I attached a plot of the correct x,y positions of the uploaded nd2 file.
>
>>
>> > I understand that the positionIndex should be the index for differnt
>> > multipoints, but how does the planeIndex look like for different
>> > time-points and channels?
>>
>> For subsequent timepoints and channels, planeIndex can be calculated:
>>
>> timepoint = 2; % the timepoint to select, starting from 0
>> channel = 1; % the channel to select, starting from 0
>> zSection = 0; % the Z section to select (probably not applicable)
>> planeIndex = FormatTools.getIndex(r, zSection, channel, timepoint);
>>
>> > The next bit is not as important to me, but still with regards to the
>> PFS
>> > status: PFS offset and state are not quite sufficient for me. During my
>> > experiments the PFS often fails to correct to the proper position and
>> > resultingly there can be different status returns indicating if the PFS
>> was
>> > switched off, still searching, not finding, or actually finding the
>> correct
>> > z-position (maybe there are even more status possibilities, from the
>> top of
>> > my head I know that 0,1,5 and 7 are possible values). With NIS elements
>> I
>> > can take a look at those status values for each image to see what went
>> > wrong. It would be nice if this information was to be retrieved by
>> > bioformats, too. Also, I often have differnt PFS offsets, e.g. at
>> different
>> > multipoints. Hence, a single PFS offset value for the entire experiment
>> > does not tell me too much.
>>
>> OK, thanks for clarifying. There is now a ticket on our issue tracking
>> system for fully parsing PFS values:
>>
>> http://trac.openmicroscopy.org.uk/ome/ticket/12560
>>
>> If you would like to be automatically notified of updates to the ticket,
>> please let us know.
>>
> It would be nice to get notified. Also, I want to emphasize that Nikon
> seems to distinguish between the PFS state and status. I guess the former
> can be just true or false, but the latter carries more information (see my
> last e-mail).
>
> Many thanks,
> Enno
>
>>
>> Regards,
>> -Melissa
>>
>> On Tue, Sep 09, 2014 at 01:46:31PM +0200, Enno R. Oldewurtel wrote:
>> > Hi Melissa,
>> >
>> > thank you for your reply. I managed to use the commands you recommended.
>> > It, also, helped me to look at an m-file by Sebastien Besson (
>> > https://gist.github.com/sbesson/6529406) to learn how to use the
>> commands.
>> > However, I still have some problems. Maybe I still do not understand
>> how to
>> > call the right plane with the right index.
>> >
>> >
>> > What I need to do is to retrieve the positions of the multipoints in an
>> nd2
>> > file in order to know how to stitch them together.
>> > I have nd2 files with 120 multipoints, several time-points and 2-3
>> channels.
>> > I usually only retrieve the positions of the first time-point, since
>> each
>> > multipoint should be at the same position at following times. In my
>> first
>> > e-mail I outlined how to retrieve positions using the entries from
>> > r.getGlobalMetadata().
>> > Now I tried the same using the commands you gave me:
>> >
>> > r.setMetadataStore(loci.formats.MetadataTools.createOMEXMLMetadata());
>> > r.setID();
>> > metadata = r.getMetadataStore();
>> > planeIndex=0;
>> > for positionIndex=1:120
>> > xPosition(i) = metadata.getPlanePositionX(positionIndex, planeIndex);
>> > yPosition(i) = metadata.getPlanePositionY(positionIndex, planeIndex);
>> > end
>> >
>> > However, I do not get the expected positions. For clarity I attach
>> plots of
>> > the x and y positions retrieved with versions 5.0.3 and 5.0.4 and also
>> the
>> > expected plot retrieved via my old method with version 5.0.3 and another
>> > plot with wrong positions retrieved with my old method and version
>> 5.0.4.
>> >
>> > I know that I should get a pattern like in the lower left plot that I
>> > attached. Can you help me once more how to call the correct positions? I
>> > understand that the positionIndex should be the index for differnt
>> > multipoints, but how does the planeIndex look like for different
>> > time-points and channels?
>> >
>> > The next bit is not as important to me, but still with regards to the
>> PFS
>> > status: PFS offset and state are not quite sufficient for me. During my
>> > experiments the PFS often fails to correct to the proper position and
>> > resultingly there can be different status returns indicating if the PFS
>> was
>> > switched off, still searching, not finding, or actually finding the
>> correct
>> > z-position (maybe there are even more status possibilities, from the
>> top of
>> > my head I know that 0,1,5 and 7 are possible values). With NIS elements
>> I
>> > can take a look at those status values for each image to see what went
>> > wrong. It would be nice if this information was to be retrieved by
>> > bioformats, too. Also, I often have differnt PFS offsets, e.g. at
>> different
>> > multipoints. Hence, a single PFS offset value for the entire experiment
>> > does not tell me too much.
>> >
>> >
>> > Many thanks,
>> > Enno
>> >
>> >
>> > On Tue, Sep 9, 2014 at 12:23 AM, Melissa Linkert <
>> > melissa at glencoesoftware.com> wrote:
>> >
>> > > Hi Enno,
>> > >
>> > > > I use bioformats.jar with Matlab to read in ND2 files. Previously I
>> used
>> > > a
>> > > > 5.0.0-DEV version of loci_tools.
>> > > > With bioformats I have some problems to retrieve meta data from ND2
>> > > files.
>> > > > With 5.0.3 I can successfully read x,y,z position data, but with
>> 5.0.4 I
>> > > > get some discrepancies.
>> > > > With both bioformat versions I fail to get the time values of each
>> image.
>> > > > However, with loci_tools I was able to do this.
>> > > >
>> > > > Below is my current approach to read meta data from an ND2 file.
>> Maybe
>> > > > someone can point out to me how to do this differently, or how to
>> fix the
>> > > > code I use?
>> > >
>> > > The best way to retrieve the stage position and timestamp data is as
>> > > follows:
>> > >
>> > > %%%
>> > > % positionIndex selects which multipoint to use (starting from 0)
>> > > % planeIndex selects the image in the chosen multipoint (starting
>> from 0)
>> > > metadata = r.getMetadataStore();
>> > > xPosition = r.getPlanePositionX(positionIndex, planeIndex);
>> > > timestamp = r.getPlaneDeltaT(positionIndex, planeIndex);
>> > > acquisitionStart = r.getImageAcquisitionDate(positionIndex);
>> > > %%%
>> > >
>> > > 'xPosition' will be the stage position along X; the Y and Z positions
>> > > can be retrieved using similar calls to getPlanePositionY and
>> > > getPlanePositionZ. 'timestamp' is the time in seconds from the
>> > > beginning of acquisition ('acquisitionStart'). See also:
>> > >
>> > >
>> > >
>> http://downloads.openmicroscopy.org/bio-formats/5.0.4/api/ome/xml/meta/MetadataRetrieve.html
>> > >
>> > >
>> http://www.openmicroscopy.org/Schemas/Documentation/Generated/OME-2013-06/ome_xsd.html#Plane
>> > >
>> > > getGlobalMetadata and getSeriesMetadata are provided for informational
>> > > purposes, but are not necessarily guaranteed to have the same
>> key/value
>> > > mappings between releases.
>> > >
>> > > > Also, is it possible to retrieve the PFS status of each image plane
>> from
>> > > an
>> > > > ND2 file?
>> > >
>> > > PFS status is not included in the OME data model; from the files we
>> > > have, it looks like the "PFS, State" and "PFS, Offset" keys in the
>> > > global metadata table will show the state and offset. If that doesn't
>> > > work,
>> > > though, please let us know.
>> > >
>> > > Regards,
>> > > -Melissa
>> > >
>> > > On Thu, Sep 04, 2014 at 06:09:16PM +0200, Enno R. Oldewurtel wrote:
>> > > > Dear all,
>> > > >
>> > > > I use bioformats.jar with Matlab to read in ND2 files. Previously I
>> used
>> > > a
>> > > > 5.0.0-DEV version of loci_tools.
>> > > > With bioformats I have some problems to retrieve meta data from ND2
>> > > files.
>> > > > With 5.0.3 I can successfully read x,y,z position data, but with
>> 5.0.4 I
>> > > > get some discrepancies.
>> > > > With both bioformat versions I fail to get the time values of each
>> image.
>> > > > However, with loci_tools I was able to do this.
>> > > >
>> > > > Below is my current approach to read meta data from an ND2 file.
>> Maybe
>> > > > someone can point out to me how to do this differently, or how to
>> fix the
>> > > > code I use?
>> > > > Also, is it possible to retrieve the PFS status of each image plane
>> from
>> > > an
>> > > > ND2 file?
>> > > >
>> > > >
>> > > > I used to retrieve e.g the x-position of a specific plane from an
>> ND2
>> > > file
>> > > > in the following way:
>> > > >
>> > > > raw=r.getGlobalMetadata();
>> > > > x=raw.get('X position for position #00083');
>> > > > This is for an ND2 file with 82 timepoints, 2 channels and 120
>> > > multipoints,
>> > > > i.e. the highest index was #09840 (for some reason I had to assume
>> > > > 82*120*2channels as the highest index for zero padding, starting
>> from
>> > > > #09841 to #19680 all values are identical)
>> > > >
>> > > > However, with bioformats 5.0.4 this gives me the x-value of the
>> first
>> > > > time-point instead of the 83rd. Similarly, in another experiment
>> with 65
>> > > > timepoints, the string 'X position for position #00066' gives me
>> again
>> > > the
>> > > > value of the first time-point instead of the 66th.
>> > > >
>> > > > Is this a bug in the new 5.0.4 version? I think this happened to me
>> > > before
>> > > > with some loci_tools version, but in a later version my approach
>> worked
>> > > > again. For bioformats 5.0.3 I can retrieve the mentioned x
>> positions just
>> > > > fine.
>> > > >
>> > > > I also need to retrieve the time data for each image. I used to
>> work with
>> > > > loci_tools (daily built from 6th September 2013) with
>> > > > rawSeries=r.getSeriesMetadata();
>> > > > t=rawSeries.get('timestamp #0001')
>> > > > (less zero-padding was needed, as this time the highest index was
>> > > > 82*120=9840, also getSeriesMetadata is used since getMetadata takes
>> very
>> > > > long to load)
>> > > >
>> > > > This worked fine, but since there were some bug fixes for ND2 files
>> and
>> > > > r.setID() runs significantly faster, I would like to stay with
>> bioformats
>> > > > instead of loci_tools.
>> > > >
>> > > >
>> > > > Best regards,
>> > > > Enno
>> > > >
>> > > >
>> > > > --
>> > > > Enno R. Oldewurtel
>> > > > AG Prof. B. Maier / Biophysics Group
>> > > > Biozentrum
>> > > > Universitaet zu Koeln
>> > > > Zuelpicher Str. 47b
>> > > > 50674 Koeln
>> > > >
>> > > > enno.oldewurtel at uni-koeln.de
>> > > > Tel: +49-221-470-8048
>> > > > Fax: +49-221-470-6230
>> > > > http://www.biophysics.uni-koeln.de/
>> > >
>> > > > _______________________________________________
>> > > > ome-users mailing list
>> > > > ome-users at lists.openmicroscopy.org.uk
>> > > > http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>> > >
>> > >
>> >
>> >
>> > --
>> > Enno R. Oldewurtel
>> > AG Prof. B. Maier / Biophysics Group
>> > Biozentrum
>> > Universitaet zu Koeln
>> > Zuelpicher Str. 47b
>> > 50674 Koeln
>> >
>> > enno.oldewurtel at uni-koeln.de
>> > Tel: +49-221-470-8048
>> > Fax: +49-221-470-6230
>> > http://www.biophysics.uni-koeln.de/
>>
>>
>>
>
>
> --
> Enno R. Oldewurtel
> AG Prof. B. Maier / Biophysics Group
> Biozentrum
> Universitaet zu Koeln
> Zuelpicher Str. 47b
> 50674 Koeln
>
> enno.oldewurtel at uni-koeln.de
> Tel: +49-221-470-8048
> Fax: +49-221-470-6230
> http://www.biophysics.uni-koeln.de/
>
> The University of Dundee is a registered Scottish Charity, No: SC015096
>
--
Enno R. Oldewurtel
AG Prof. B. Maier / Biophysics Group
Biozentrum
Universitaet zu Koeln
Zuelpicher Str. 47b
50674 Koeln
enno.oldewurtel at uni-koeln.de
Tel: +49-221-470-8048
Fax: +49-221-470-6230
http://www.biophysics.uni-koeln.de/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20150505/8d05bb02/attachment.html>
More information about the ome-users
mailing list