[ome-users] loci_tools with Nikon ND2 files - Excessive metdadata entries for time + multipoint stack

Melissa Linkert melissa at glencoesoftware.com
Thu Jun 27 16:41:52 BST 2013 

Hi Enno,

> I am using loci_tools.jar to read out Nikon ND2 files obtained with
> NIS-Elements AR 4.00.03 (Build 775).
> 
> I record nd2 files with several timepoints and multipoints (XY). Without
> time dimension assigned the metadata shows entries for each x,y,z and time
> position only once, but whenever there is also a time dimension, each XY
> point is treated as one series point and for each series point the x,y,z
> and time information is repeated.
> This repeat causes a large size increase of the metadata. I read in the
> metadataList using the getMetadata() in Matlab and frequently get a
> metadataList with a few million entries, causing a large increase in memory
> usage. For one special experiment I had to obtain a few thousand
> multipoints and neither Matlab nor imageJ managed to open this.
> 
> Is it possible to change the metadata structure obtained from nd2 files in
> such a way that there are no repeats?
> 
> In case this is not possible, is there a way to extract the information
> within the usual Metadata List in a different way. That way I might be able
> to leave out the .getMetadata() and create the metadataList manually,
> although I'd prefer to directly access a complete list of metadata.
> 
> At the bottom I give some of the metadata of a nd2 file one obtained with
> time dimension turned on, causing the increase of size for a small nd2 file
> with only 6 XY points.

Thank you for the report.  If you are using bfopen.m, we now have a fix
for this under review:

https://github.com/openmicroscopy/bioformats/pull/573

If you are using your own .m file that has a call to getMetadata(), then
the solution would be to call getSeriesMetadata() instead; that will
give you only the metadata for the current position.

Regards,
-Melissa


On Wed, Jun 26, 2013 at 03:12:52PM +0200, Enno R. Oldewurtel wrote:
> Dear all,
> 
> I am using loci_tools.jar to read out Nikon ND2 files obtained with
> NIS-Elements AR 4.00.03 (Build 775).
> 
> I record nd2 files with several timepoints and multipoints (XY). Without
> time dimension assigned the metadata shows entries for each x,y,z and time
> position only once, but whenever there is also a time dimension, each XY
> point is treated as one series point and for each series point the x,y,z
> and time information is repeated.
> This repeat causes a large size increase of the metadata. I read in the
> metadataList using the getMetadata() in Matlab and frequently get a
> metadataList with a few million entries, causing a large increase in memory
> usage. For one special experiment I had to obtain a few thousand
> multipoints and neither Matlab nor imageJ managed to open this.
> 
> Is it possible to change the metadata structure obtained from nd2 files in
> such a way that there are no repeats?
> 
> In case this is not possible, is there a way to extract the information
> within the usual Metadata List in a different way. That way I might be able
> to leave out the .getMetadata() and create the metadataList manually,
> although I'd prefer to directly access a complete list of metadata.
> 
> At the bottom I give some of the metadata of a nd2 file one obtained with
> time dimension turned on, causing the increase of size for a small nd2 file
> with only 6 XY points.
> 
> I have two example nd2. One without the repeats of metadata, since I turned
> the time dimension off, and one with it being turned on during acquisition.
> I will submit those to the OME's QA system.
> 
> 
> Many thanks for your great work and support. I hope someone can help me out
> on this one.
> Enno
> 
> 
> MP2x3_Ch1_T1_timeOn.nd2 (series 1) X position    -71158.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 1) Y position    59268.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 1) Z position for position, plane #1
> 499.975
> MP2x3_Ch1_T1_timeOn.nd2 (series 1) Z position for position, plane #2
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 1) Z position for position, plane #3
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 1) Z position for position, plane #4
> 499.975
> MP2x3_Ch1_T1_timeOn.nd2 (series 1) Z position for position, plane #5
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 1) Z position for position, plane #6
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 1) timestamp #1    1.5470483098831027
> MP2x3_Ch1_T1_timeOn.nd2 (series 1) timestamp #2    3.1381038113413378
> MP2x3_Ch1_T1_timeOn.nd2 (series 1) timestamp #3    3.808205685772933
> MP2x3_Ch1_T1_timeOn.nd2 (series 1) timestamp #4    4.541229801683687
> MP2x3_Ch1_T1_timeOn.nd2 (series 1) timestamp #5    6.136187115310691
> MP2x3_Ch1_T1_timeOn.nd2 (series 1) timestamp #6    6.8340883906930685
> MP2x3_Ch1_T1_timeOn.nd2 (series 2) X position    -71158.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 2) Y position    59268.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 2) Z position for position, plane #1
> 499.975
> MP2x3_Ch1_T1_timeOn.nd2 (series 2) Z position for position, plane #2
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 2) Z position for position, plane #3
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 2) Z position for position, plane #4
> 499.975
> MP2x3_Ch1_T1_timeOn.nd2 (series 2) Z position for position, plane #5
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 2) Z position for position, plane #6
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 2) timestamp #1    1.5470483098831027
> MP2x3_Ch1_T1_timeOn.nd2 (series 2) timestamp #2    3.1381038113413378
> MP2x3_Ch1_T1_timeOn.nd2 (series 2) timestamp #3    3.808205685772933
> MP2x3_Ch1_T1_timeOn.nd2 (series 2) timestamp #4    4.541229801683687
> MP2x3_Ch1_T1_timeOn.nd2 (series 2) timestamp #5    6.136187115310691
> MP2x3_Ch1_T1_timeOn.nd2 (series 2) timestamp #6    6.8340883906930685
> MP2x3_Ch1_T1_timeOn.nd2 (series 3) X position    -71158.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 3) Y position    59268.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 3) Z position for position, plane #1
> 499.975
> MP2x3_Ch1_T1_timeOn.nd2 (series 3) Z position for position, plane #2
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 3) Z position for position, plane #3
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 3) Z position for position, plane #4
> 499.975
> MP2x3_Ch1_T1_timeOn.nd2 (series 3) Z position for position, plane #5
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 3) Z position for position, plane #6
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 3) timestamp #1    1.5470483098831027
> MP2x3_Ch1_T1_timeOn.nd2 (series 3) timestamp #2    3.1381038113413378
> MP2x3_Ch1_T1_timeOn.nd2 (series 3) timestamp #3    3.808205685772933
> MP2x3_Ch1_T1_timeOn.nd2 (series 3) timestamp #4    4.541229801683687
> MP2x3_Ch1_T1_timeOn.nd2 (series 3) timestamp #5    6.136187115310691
> MP2x3_Ch1_T1_timeOn.nd2 (series 3) timestamp #6    6.8340883906930685
> MP2x3_Ch1_T1_timeOn.nd2 (series 4) X position    -71158.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 4) Y position    59268.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 4) Z position for position, plane #1
> 499.975
> MP2x3_Ch1_T1_timeOn.nd2 (series 4) Z position for position, plane #2
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 4) Z position for position, plane #3
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 4) Z position for position, plane #4
> 499.975
> MP2x3_Ch1_T1_timeOn.nd2 (series 4) Z position for position, plane #5
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 4) Z position for position, plane #6
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 4) timestamp #1    1.5470483098831027
> MP2x3_Ch1_T1_timeOn.nd2 (series 4) timestamp #2    3.1381038113413378
> MP2x3_Ch1_T1_timeOn.nd2 (series 4) timestamp #3    3.808205685772933
> MP2x3_Ch1_T1_timeOn.nd2 (series 4) timestamp #4    4.541229801683687
> MP2x3_Ch1_T1_timeOn.nd2 (series 4) timestamp #5    6.136187115310691
> MP2x3_Ch1_T1_timeOn.nd2 (series 4) timestamp #6    6.8340883906930685
> MP2x3_Ch1_T1_timeOn.nd2 (series 5) X position    -71158.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 5) Y position    59268.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 5) Z position for position, plane #1
> 499.975
> MP2x3_Ch1_T1_timeOn.nd2 (series 5) Z position for position, plane #2
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 5) Z position for position, plane #3
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 5) Z position for position, plane #4
> 499.975
> MP2x3_Ch1_T1_timeOn.nd2 (series 5) Z position for position, plane #5
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 5) Z position for position, plane #6
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 5) timestamp #1    1.5470483098831027
> MP2x3_Ch1_T1_timeOn.nd2 (series 5) timestamp #2    3.1381038113413378
> MP2x3_Ch1_T1_timeOn.nd2 (series 5) timestamp #3    3.808205685772933
> MP2x3_Ch1_T1_timeOn.nd2 (series 5) timestamp #4    4.541229801683687
> MP2x3_Ch1_T1_timeOn.nd2 (series 5) timestamp #5    6.136187115310691
> MP2x3_Ch1_T1_timeOn.nd2 (series 5) timestamp #6    6.8340883906930685
> MP2x3_Ch1_T1_timeOn.nd2 (series 6) X position    -71158.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 6) Y position    59268.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 6) Z position for position, plane #1
> 499.975
> MP2x3_Ch1_T1_timeOn.nd2 (series 6) Z position for position, plane #2
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 6) Z position for position, plane #3
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 6) Z position for position, plane #4
> 499.975
> MP2x3_Ch1_T1_timeOn.nd2 (series 6) Z position for position, plane #5
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 6) Z position for position, plane #6
> 500.0
> MP2x3_Ch1_T1_timeOn.nd2 (series 6) timestamp #1    1.5470483098831027
> MP2x3_Ch1_T1_timeOn.nd2 (series 6) timestamp #2    3.1381038113413378
> MP2x3_Ch1_T1_timeOn.nd2 (series 6) timestamp #3    3.808205685772933
> MP2x3_Ch1_T1_timeOn.nd2 (series 6) timestamp #4    4.541229801683687
> MP2x3_Ch1_T1_timeOn.nd2 (series 6) timestamp #5    6.136187115310691
> MP2x3_Ch1_T1_timeOn.nd2 (series 6) timestamp #6    6.8340883906930685
> 
> 
> -- 
> Enno R. Oldewurtel
> AG Prof. B. Maier / Biophysics Group
> Biozentrum
> Universitaet zu Koeln
> Zuelpicher Str. 47b
> 50674 Koeln
> 
> enno.oldewurtel at uni-koeln.de
> Tel: +49-221-470-8048
> Fax: +49-221-470-6230
> http://www.biophysics.uni-koeln.de/

> _______________________________________________
> ome-users mailing list
> ome-users at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users

More information about the ome-users mailing list