[ome-users] Loading a subset of a stack

[Roger Leigh]

On 12/03/15 10:21, Roger Leigh wrote:
> On 11/03/15 12:49, Thomas Julou wrote:
>> Dear Roger,
>>
>> Thank you very much for your answer and help.
>>
>>>> 2. or to create a reader and then load the plane I want only.
>>>> but the metadata of all frames are still loaded during the reader
>>>> creation?
>>>>
>>>>> IFormatReader reader = new ImageReader();
>>>
>>> Try adding here:
>>>
>>> MetadataOptions options = reader.getMetadataOptions();
>>> options.setMetadataLevel(MetadataLevel.MINIMUM);
>>>
>>>>> reader.setId(id);
>>>>> //seriesCount = reader.getSeriesCount();
>>>>> sizeZ = reader.getSizeZ();
>>>>> sizeC = reader.getSizeC();
>>>>> sizeT = reader.getSizeT();
>>>>> imageCount = reader.getImageCount();
>>>>> IJ.log(sizeT + " frames.");
>>
>> This looks like exactly what I was after. Unfortunately, when I tried it
>> with a 5gb dataset produced by micromanager (supposably ome.tif stacks),
>> I don’t get any significant change in performance. I guess it is due to
>> how this specific reader is implemented :(
>>
>>> The default behaviour is to read all metadata.  The above two lines
>>> should set it to read a minimum amount of metadata.  However, please
>>> note that this is dependent upon the implementation details of the
>>> individual readers and file formats; you may or many not see any
>>> difference in speed.  It will also mean if you need the extra metadata
>>> you'll need to reopen the image since it's parsed once when you
>>> initialise the reader.
>>
>> Thanks for the detailed explanation.
>>
>>> I should also point out that recent work has been done to improve the
>>> performance of Bio-Formats with files on network storage which should be
>>> in the upcoming 5.1 release.  You may find that this will provide
>>> significant improvements, though it will depend upon the file
>>> format/reader in question.
>>
>> As mentionned above, I’m mostly using micro-manager dataset. Do you know
>> whether any work has been done on this?
>> Or can you explain me where to find the corresponding files in the
>> source code? (then I could track to check the changes on github).
>
> The Micro-Manager reader is in
> components/formats-bsd/src/loci/formats/in/MicromanagerReader.java
>
> To view the changes with git (the github interface won't show it--there
> are too many other changes):
>
>    git diff origin/dev_5_0..origin/develop --
> components/formats-bsd/src/loci/formats/in/MicromanagerReader.java
>
> Summary: there are no significant changes other than some annotations
> and support for units in the data model; nothing that would obviously
> improve performance.  Any other changes have already been backported to
> 5.0.x (dev_5_0).
>
> Looking at the reader, it does appear that the slowness is likely due to
> the nature of the file format.  In the initFile method, it's required to
> trawl through the directory tree to find additional positions, and this
> may be quite slow on networked storage.  It's possible that this could
> be updated to improve performance, but that's making the assumption that
> this hasn't already been done which I'm not certain of (I'll
> double-check this and open a ticket for it if not).

It looks like the performance of this reader is unlikely to be improved
further.

One additional suggestion would be to try disabling file grouping.  This
is as simple as doing:

   reader.setGroupFiles(false);

before calling reader.setId(file);

You said that it was taking a very long time to initialise.  Could you
possibly let us know any quantitative measurements of that time?  And do
you know how many files/directories are in the dataset in question?

If you wish to try out a newer version of Bio-Formats to see if it
improves performance, you could try the current merge build from:


https://ci.openmicroscopy.org/view/5.1/job/BIOFORMATS-5.1-merge-build/lastSuccessfulBuild/artifact/artifacts/

(e.g. bioformats_package.jar or loci_tools.jar or bftools.zip to try it
out standalone with tools/showinf, for example)


Kind regards,
Roger

--
Dr Roger Leigh -- Open Microscopy Environment
Wellcome Trust Centre for Gene Regulation and Expression,
College of Life Sciences, University of Dundee, Dow Street,
Dundee DD1 5EH Scotland UK   Tel: (01382) 386364

The University of Dundee is a registered Scottish Charity, No: SC015096