[ome-users] Loading a subset of a stack

[Thomas Julou]

Hi Robert,

>> The Micro-Manager reader is in
>> components/formats-bsd/src/loci/formats/in/MicromanagerReader.java
>> 
>> To view the changes with git (the github interface won't show it--there
>> are too many other changes):
>> 
>>   git diff origin/dev_5_0..origin/develop --
>> components/formats-bsd/src/loci/formats/in/MicromanagerReader.java
>> 
>> Summary: there are no significant changes other than some annotations
>> and support for units in the data model; nothing that would obviously
>> improve performance.  Any other changes have already been backported to
>> 5.0.x (dev_5_0).
>> 
>> Looking at the reader, it does appear that the slowness is likely due to
>> the nature of the file format.  In the initFile method, it's required to
>> trawl through the directory tree to find additional positions, and this
>> may be quite slow on networked storage.  It's possible that this could
>> be updated to improve performance, but that's making the assumption that
>> this hasn't already been done which I'm not certain of (I'll
>> double-check this and open a ticket for it if not).
> 
> It looks like the performance of this reader is unlikely to be improved
> further.
> 
> One additional suggestion would be to try disabling file grouping.  This
> is as simple as doing:
> 
>   reader.setGroupFiles(false);
> 
> before calling reader.setId(file);
> 
> You said that it was taking a very long time to initialise.  Could you
> possibly let us know any quantitative measurements of that time?  And do
> you know how many files/directories are in the dataset in question?
> 
> If you wish to try out a newer version of Bio-Formats to see if it
> improves performance, you could try the current merge build from:
> 
> https://ci.openmicroscopy.org/view/5.1/job/BIOFORMATS-5.1-merge-build/lastSuccessfulBuild/artifact/artifacts/
> 
> (e.g. bioformats_package.jar or loci_tools.jar or bftools.zip to try it
> out standalone with tools/showinf, for example)

I heard a lot of good in the past about the legendary support for ome. I’m glad to experience it for real :)
Thanks a lot for the detailed explanation!

I’ll try next week to disable file grouping, and also to benchmark import with different options next week.
Best,

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20150313/354b0e83/attachment.html>