[ome-users] Loading a subset of a stack

[Melissa Linkert]

Hi Thomas,

> I tried to read the first 10 frames of the first stack (≈11gb; split into 3 files by micro-manager) using ImageReader() as described early in this thread (see attached file).
> with default options: « 494.016s to create a reader. »
> with no grouping: « 489.204s to create a reader. » 
> with no grouping, minimum metadata: « 474.095s to create a reader. » 
> 
> I guess the little speed up is due to repeated file access: if I use default options once again at the end I get « 460.052s to create a reader. »
> 
> Whether I disable grouping or not, I always get 5 messages «  Reading IFDs; Populating metadata »; so I guess that grouping is actually not disabled. correct?
> I suspect a function disabling grouping of « series » might be what I’m after here, though I don’t know which one exactly…

Thank you for the additional information.  We are reviewing a fix for
this now:

https://github.com/openmicroscopy/bioformats/pull/1687

I would expect that to be included in the upcoming 5.1.0 release.  This
will mean that disabling file grouping works as intended - the
initialization time will be faster (~20x in local tests), but the only
planes available will be those in the selected TIFF file.

Regards,
-Melissa

On Wed, Mar 18, 2015 at 04:07:44PM +0100, Thomas Julou wrote:
> Hello,
> 
> > I heard a lot of good in the past about the legendary support for ome. I’m glad to experience it for real :)
> 
> I’m happy I gave you a good week end start ;)
> 
> >> It looks like the performance of this reader is unlikely to be improved
> >> further. (…)
> >> 
> >> You said that it was taking a very long time to initialise.  Could you
> >> possibly let us know any quantitative measurements of that time?  And do
> >> you know how many files/directories are in the dataset in question?
> > 
> > 
> > I’ll try next week to disable file grouping, and also to benchmark import with different options next week.
> 
> Here you go:
> I did the tests with a directory that contains the following files (time lapse at 5 positions acquired with micro-manager)
> 
>  595 20150304_exp_settings.txt
> 4.0G 20150304_glu_multiexp_1_MMStack_Pos0_1.ome.tif
> 2.7G 20150304_glu_multiexp_1_MMStack_Pos0_2.ome.tif
>  20M 20150304_glu_multiexp_1_MMStack_Pos0_metadata.txt
> 4.0G 20150304_glu_multiexp_1_MMStack_Pos0.ome.tif
> 4.0G 20150304_glu_multiexp_1_MMStack_Pos1_1.ome.tif
> 2.7G 20150304_glu_multiexp_1_MMStack_Pos1_2.ome.tif
>  20M 20150304_glu_multiexp_1_MMStack_Pos1_metadata.txt
> 4.0G 20150304_glu_multiexp_1_MMStack_Pos1.ome.tif
> 4.0G 20150304_glu_multiexp_1_MMStack_Pos2_1.ome.tif
> 2.7G 20150304_glu_multiexp_1_MMStack_Pos2_2.ome.tif
>  20M 20150304_glu_multiexp_1_MMStack_Pos2_metadata.txt
> 4.0G 20150304_glu_multiexp_1_MMStack_Pos2.ome.tif
> 4.0G 20150304_glu_multiexp_1_MMStack_Pos3_1.ome.tif
> 2.7G 20150304_glu_multiexp_1_MMStack_Pos3_2.ome.tif
>  20M 20150304_glu_multiexp_1_MMStack_Pos3_metadata.txt
> 4.0G 20150304_glu_multiexp_1_MMStack_Pos3.ome.tif
> 4.0G 20150304_glu_multiexp_1_MMStack_Pos4_1.ome.tif
> 2.7G 20150304_glu_multiexp_1_MMStack_Pos4_2.ome.tif
>  20M 20150304_glu_multiexp_1_MMStack_Pos4_metadata.txt
> 4.0G 20150304_glu_multiexp_1_MMStack_Pos4.ome.tif
> 
> I tried to read the first 10 frames of the first stack (≈11gb; split into 3 files by micro-manager) using ImageReader() as described early in this thread (see attached file).
> with default options: « 494.016s to create a reader. »
> with no grouping: « 489.204s to create a reader. » 
> with no grouping, minimum metadata: « 474.095s to create a reader. » 
> 
> I guess the little speed up is due to repeated file access: if I use default options once again at the end I get « 460.052s to create a reader. »
> 
> Whether I disable grouping or not, I always get 5 messages «  Reading IFDs; Populating metadata »; so I guess that grouping is actually not disabled. correct?
> I suspect a function disabling grouping of « series » might be what I’m after here, though I don’t know which one exactly…
> 
> This is maybe not so slow if you consider that it’s parsing all metadata; but in my case, what I would like would be to simply access the data of the first frames, don’t care about what happens later.
> 
> Any hint to help us handle these datasets in a faster way will be very appreciated.
> Best,
> 
> Thomas
> 
> --
> Thomas Julou  |  Computational & Systems Biology  |  Biozentrum – University of Basel  |  Klingelbergstrasse 50/70 CH-4056 Basel  |  +41 (0)61 267 16 21
> 
> 
> 

> _______________________________________________
> ome-users mailing list
> ome-users at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users