[ome-users] Loading a subset of a stack

[Thomas Julou]

Hello,

> I heard a lot of good in the past about the legendary support for ome. I’m glad to experience it for real :)

I’m happy I gave you a good week end start ;)

>> It looks like the performance of this reader is unlikely to be improved
>> further. (…)
>> 
>> You said that it was taking a very long time to initialise.  Could you
>> possibly let us know any quantitative measurements of that time?  And do
>> you know how many files/directories are in the dataset in question?
> 
> 
> I’ll try next week to disable file grouping, and also to benchmark import with different options next week.

Here you go:
I did the tests with a directory that contains the following files (time lapse at 5 positions acquired with micro-manager)

 595 20150304_exp_settings.txt
4.0G 20150304_glu_multiexp_1_MMStack_Pos0_1.ome.tif
2.7G 20150304_glu_multiexp_1_MMStack_Pos0_2.ome.tif
 20M 20150304_glu_multiexp_1_MMStack_Pos0_metadata.txt
4.0G 20150304_glu_multiexp_1_MMStack_Pos0.ome.tif
4.0G 20150304_glu_multiexp_1_MMStack_Pos1_1.ome.tif
2.7G 20150304_glu_multiexp_1_MMStack_Pos1_2.ome.tif
 20M 20150304_glu_multiexp_1_MMStack_Pos1_metadata.txt
4.0G 20150304_glu_multiexp_1_MMStack_Pos1.ome.tif
4.0G 20150304_glu_multiexp_1_MMStack_Pos2_1.ome.tif
2.7G 20150304_glu_multiexp_1_MMStack_Pos2_2.ome.tif
 20M 20150304_glu_multiexp_1_MMStack_Pos2_metadata.txt
4.0G 20150304_glu_multiexp_1_MMStack_Pos2.ome.tif
4.0G 20150304_glu_multiexp_1_MMStack_Pos3_1.ome.tif
2.7G 20150304_glu_multiexp_1_MMStack_Pos3_2.ome.tif
 20M 20150304_glu_multiexp_1_MMStack_Pos3_metadata.txt
4.0G 20150304_glu_multiexp_1_MMStack_Pos3.ome.tif
4.0G 20150304_glu_multiexp_1_MMStack_Pos4_1.ome.tif
2.7G 20150304_glu_multiexp_1_MMStack_Pos4_2.ome.tif
 20M 20150304_glu_multiexp_1_MMStack_Pos4_metadata.txt
4.0G 20150304_glu_multiexp_1_MMStack_Pos4.ome.tif

I tried to read the first 10 frames of the first stack (≈11gb; split into 3 files by micro-manager) using ImageReader() as described early in this thread (see attached file).
with default options: « 494.016s to create a reader. »
with no grouping: « 489.204s to create a reader. » 
with no grouping, minimum metadata: « 474.095s to create a reader. » 

I guess the little speed up is due to repeated file access: if I use default options once again at the end I get « 460.052s to create a reader. »

Whether I disable grouping or not, I always get 5 messages «  Reading IFDs; Populating metadata »; so I guess that grouping is actually not disabled. correct?
I suspect a function disabling grouping of « series » might be what I’m after here, though I don’t know which one exactly…

This is maybe not so slow if you consider that it’s parsing all metadata; but in my case, what I would like would be to simply access the data of the first frames, don’t care about what happens later.

Any hint to help us handle these datasets in a faster way will be very appreciated.
Best,

Thomas

--
Thomas Julou  |  Computational & Systems Biology  |  Biozentrum – University of Basel  |  Klingelbergstrasse 50/70 CH-4056 Basel  |  +41 (0)61 267 16 21



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20150318/7d5fbb10/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test_ome_user.bsh
Type: application/octet-stream
Size: 1170 bytes
Desc: not available
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20150318/7d5fbb10/attachment.obj>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20150318/7d5fbb10/attachment-0001.html>