[ome-users] loci_tools with Nikon ND2 files - Excessive metdadata entries for time + multipoint stack

Melissa Linkert melissa at glencoesoftware.com
Mon Jul 1 20:57:10 BST 2013 

Hi Enno,

> thanks for quick reply! getSeriesMetadata() pointed me into the right
> direction. In combination with getGlobalMetadata() I can now get all
> relevant info.

Glad to hear it!

> As far as I understand getMetadata() simply gives
> GlobalMetadata plus the SeriesMetadata for each position (number of entries
> in GlobalMetadata + SeriesCount * number of entries in SeriesMetadata =
> number of entries in Metadata; at least in my case).

Yes, that is correct.

> This certainly helps me for my image processing in Matlab. However, is this
> still a problem for people opening large time+XY nd2 images from imageJ
> using the loci_tools plugin? Does imageJ always call getMetadata() when
> opening nd2 files, or only if metadata is marked for display?

No, the Bio-Formats plugin for ImageJ calls getSeriesMetadata(), so this
would not be an issue for ImageJ users.

Regards,
-Melissa

On Fri, Jun 28, 2013 at 08:44:51AM +0200, Enno R. Oldewurtel wrote:
> Hi Melissa,
> 
> thanks for quick reply! getSeriesMetadata() pointed me into the right
> direction. In combination with getGlobalMetadata() I can now get all
> relevant info. As far as I understand getMetadata() simply gives
> GlobalMetadata plus the SeriesMetadata for each position (number of entries
> in GlobalMetadata + SeriesCount * number of entries in SeriesMetadata =
> number of entries in Metadata; at least in my case).
> 
> This certainly helps me for my image processing in Matlab. However, is this
> still a problem for people opening large time+XY nd2 images from imageJ
> using the loci_tools plugin? Does imageJ always call getMetadata() when
> opening nd2 files, or only if metadata is marked for display?
> 
> Best wishes,
> Enno
> 
> 
> On Thu, Jun 27, 2013 at 5:41 PM, Melissa Linkert <
> melissa at glencoesoftware.com> wrote:
> 
> > Hi Enno,
> >
> > > I am using loci_tools.jar to read out Nikon ND2 files obtained with
> > > NIS-Elements AR 4.00.03 (Build 775).
> > >
> > > I record nd2 files with several timepoints and multipoints (XY). Without
> > > time dimension assigned the metadata shows entries for each x,y,z and
> > time
> > > position only once, but whenever there is also a time dimension, each XY
> > > point is treated as one series point and for each series point the x,y,z
> > > and time information is repeated.
> > > This repeat causes a large size increase of the metadata. I read in the
> > > metadataList using the getMetadata() in Matlab and frequently get a
> > > metadataList with a few million entries, causing a large increase in
> > memory
> > > usage. For one special experiment I had to obtain a few thousand
> > > multipoints and neither Matlab nor imageJ managed to open this.
> > >
> > > Is it possible to change the metadata structure obtained from nd2 files
> > in
> > > such a way that there are no repeats?
> > >
> > > In case this is not possible, is there a way to extract the information
> > > within the usual Metadata List in a different way. That way I might be
> > able
> > > to leave out the .getMetadata() and create the metadataList manually,
> > > although I'd prefer to directly access a complete list of metadata.
> > >
> > > At the bottom I give some of the metadata of a nd2 file one obtained with
> > > time dimension turned on, causing the increase of size for a small nd2
> > file
> > > with only 6 XY points.
> >
> > Thank you for the report.  If you are using bfopen.m, we now have a fix
> > for this under review:
> >
> > https://github.com/openmicroscopy/bioformats/pull/573
> >
> > If you are using your own .m file that has a call to getMetadata(), then
> > the solution would be to call getSeriesMetadata() instead; that will
> > give you only the metadata for the current position.
> >
> > Regards,
> > -Melissa
> >
> >
> > On Wed, Jun 26, 2013 at 03:12:52PM +0200, Enno R. Oldewurtel wrote:
> > > Dear all,
> > >
> > > I am using loci_tools.jar to read out Nikon ND2 files obtained with
> > > NIS-Elements AR 4.00.03 (Build 775).
> > >
> > > I record nd2 files with several timepoints and multipoints (XY). Without
> > > time dimension assigned the metadata shows entries for each x,y,z and
> > time
> > > position only once, but whenever there is also a time dimension, each XY
> > > point is treated as one series point and for each series point the x,y,z
> > > and time information is repeated.
> > > This repeat causes a large size increase of the metadata. I read in the
> > > metadataList using the getMetadata() in Matlab and frequently get a
> > > metadataList with a few million entries, causing a large increase in
> > memory
> > > usage. For one special experiment I had to obtain a few thousand
> > > multipoints and neither Matlab nor imageJ managed to open this.
> > >
> > > Is it possible to change the metadata structure obtained from nd2 files
> > in
> > > such a way that there are no repeats?
> > >
> > > In case this is not possible, is there a way to extract the information
> > > within the usual Metadata List in a different way. That way I might be
> > able
> > > to leave out the .getMetadata() and create the metadataList manually,
> > > although I'd prefer to directly access a complete list of metadata.
> > >
> > > At the bottom I give some of the metadata of a nd2 file one obtained with
> > > time dimension turned on, causing the increase of size for a small nd2
> > file
> > > with only 6 XY points.
> > >
> > > I have two example nd2. One without the repeats of metadata, since I
> > turned
> > > the time dimension off, and one with it being turned on during
> > acquisition.
> > > I will submit those to the OME's QA system.
> > >
> > >
> > > Many thanks for your great work and support. I hope someone can help me
> > out
> > > on this one.
> > > Enno
> > >
> > >
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 1) X position    -71158.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 1) Y position    59268.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 1) Z position for position, plane #1
> > > 499.975
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 1) Z position for position, plane #2
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 1) Z position for position, plane #3
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 1) Z position for position, plane #4
> > > 499.975
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 1) Z position for position, plane #5
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 1) Z position for position, plane #6
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 1) timestamp #1    1.5470483098831027
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 1) timestamp #2    3.1381038113413378
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 1) timestamp #3    3.808205685772933
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 1) timestamp #4    4.541229801683687
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 1) timestamp #5    6.136187115310691
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 1) timestamp #6    6.8340883906930685
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 2) X position    -71158.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 2) Y position    59268.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 2) Z position for position, plane #1
> > > 499.975
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 2) Z position for position, plane #2
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 2) Z position for position, plane #3
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 2) Z position for position, plane #4
> > > 499.975
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 2) Z position for position, plane #5
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 2) Z position for position, plane #6
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 2) timestamp #1    1.5470483098831027
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 2) timestamp #2    3.1381038113413378
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 2) timestamp #3    3.808205685772933
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 2) timestamp #4    4.541229801683687
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 2) timestamp #5    6.136187115310691
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 2) timestamp #6    6.8340883906930685
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 3) X position    -71158.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 3) Y position    59268.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 3) Z position for position, plane #1
> > > 499.975
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 3) Z position for position, plane #2
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 3) Z position for position, plane #3
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 3) Z position for position, plane #4
> > > 499.975
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 3) Z position for position, plane #5
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 3) Z position for position, plane #6
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 3) timestamp #1    1.5470483098831027
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 3) timestamp #2    3.1381038113413378
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 3) timestamp #3    3.808205685772933
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 3) timestamp #4    4.541229801683687
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 3) timestamp #5    6.136187115310691
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 3) timestamp #6    6.8340883906930685
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 4) X position    -71158.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 4) Y position    59268.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 4) Z position for position, plane #1
> > > 499.975
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 4) Z position for position, plane #2
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 4) Z position for position, plane #3
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 4) Z position for position, plane #4
> > > 499.975
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 4) Z position for position, plane #5
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 4) Z position for position, plane #6
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 4) timestamp #1    1.5470483098831027
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 4) timestamp #2    3.1381038113413378
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 4) timestamp #3    3.808205685772933
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 4) timestamp #4    4.541229801683687
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 4) timestamp #5    6.136187115310691
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 4) timestamp #6    6.8340883906930685
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 5) X position    -71158.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 5) Y position    59268.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 5) Z position for position, plane #1
> > > 499.975
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 5) Z position for position, plane #2
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 5) Z position for position, plane #3
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 5) Z position for position, plane #4
> > > 499.975
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 5) Z position for position, plane #5
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 5) Z position for position, plane #6
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 5) timestamp #1    1.5470483098831027
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 5) timestamp #2    3.1381038113413378
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 5) timestamp #3    3.808205685772933
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 5) timestamp #4    4.541229801683687
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 5) timestamp #5    6.136187115310691
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 5) timestamp #6    6.8340883906930685
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 6) X position    -71158.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 6) Y position    59268.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 6) Z position for position, plane #1
> > > 499.975
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 6) Z position for position, plane #2
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 6) Z position for position, plane #3
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 6) Z position for position, plane #4
> > > 499.975
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 6) Z position for position, plane #5
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 6) Z position for position, plane #6
> > > 500.0
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 6) timestamp #1    1.5470483098831027
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 6) timestamp #2    3.1381038113413378
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 6) timestamp #3    3.808205685772933
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 6) timestamp #4    4.541229801683687
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 6) timestamp #5    6.136187115310691
> > > MP2x3_Ch1_T1_timeOn.nd2 (series 6) timestamp #6    6.8340883906930685
> > >
> > >
> > > --
> > > Enno R. Oldewurtel
> > > AG Prof. B. Maier / Biophysics Group
> > > Biozentrum
> > > Universitaet zu Koeln
> > > Zuelpicher Str. 47b
> > > 50674 Koeln
> > >
> > > enno.oldewurtel at uni-koeln.de
> > > Tel: +49-221-470-8048
> > > Fax: +49-221-470-6230
> > > http://www.biophysics.uni-koeln.de/
> >
> > > _______________________________________________
> > > ome-users mailing list
> > > ome-users at lists.openmicroscopy.org.uk
> > > http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
> >
> >
> 
> 
> -- 
> Enno R. Oldewurtel
> AG Prof. B. Maier / Biophysics Group
> Biozentrum
> Universitaet zu Koeln
> Zuelpicher Str. 47b
> 50674 Koeln
> 
> enno.oldewurtel at uni-koeln.de
> Tel: +49-221-470-8048
> Fax: +49-221-470-6230
> http://www.biophysics.uni-koeln.de/

More information about the ome-users mailing list