[ome-users] Loading a subset of a stack

[Roger Leigh]

On 09/03/15 13:12, Thomas Julou wrote:

> I’m writing about performance issues when handling large files with bioformats. I realize that this is a rather common issue a since I could already read the following threads/issues:
> http://lists.openmicroscopy.org.uk/pipermail/ome-users/2014-December/004958.html
> http://trac.openmicroscopy.org/ome/ticket/6586
> http://imagej.1557.x6.nabble.com/Fiji-Bio-Formats-is-not-opening-large-czi-td5011842.html
>
> Since we’re typically working with 10-15gb stacks stored on network storage, loading stacks with bioformats (even virtual stacks) is very long. It looks like this is due to the fact that the metadata of the entire stack are always parsed extensively at first. So my questions are two-fold:
> - is it possible to get the overall stack metadata (number of timepoints, channels, slices, etc) without parsing the metadata of all frames?
> - is it possible to load a few frames (e.g. all channels and slices of the first time point) without parsing the metadata of all frames?
>
> So far, it tried to use
> 1. either the high level importer:
> but this is reading the metadata of all frames at first, and then opens a flat stack without time/channel/slice info…
>
>> ImporterOptions options = new ImporterOptions();
>> options.setId(id);
>> options.setVirtual(true);
>> options.setTBegin(0, 0);
>> options.setTEnd(0, 1);
>> options.setTStep(0, 1);
>> ImagePlus[] imps = BF.openImagePlus(options);
>> imps[0].show();

I don't think this higher-level interface will give you the control
needed, but see below.

> 2. or to create a reader and then load the plane I want only.
> but the metadata of all frames are still loaded during the reader creation…
>
>> IFormatReader reader = new ImageReader();

Try adding here:

MetadataOptions options = reader.getMetadataOptions();
options.setMetadataLevel(MetadataLevel.MINIMUM);

>> reader.setId(id);
>> //seriesCount = reader.getSeriesCount();
>> sizeZ = reader.getSizeZ();
>> sizeC = reader.getSizeC();
>> sizeT = reader.getSizeT();
>> imageCount = reader.getImageCount();
>> IJ.log(sizeT + " frames.");
>>

> I would really appreciate to know whether there are better ways to read a subset of a stack only using bioformats and/or to get overall info without reading the metadata of all frames.
> I thank you in advance for your help. Best,

The default behaviour is to read all metadata.  The above two lines
should set it to read a minimum amount of metadata.  However, please
note that this is dependent upon the implementation details of the
individual readers and file formats; you may or many not see any
difference in speed.  It will also mean if you need the extra metadata
you'll need to reopen the image since it's parsed once when you
initialise the reader.

I should also point out that recent work has been done to improve the
performance of Bio-Formats with files on network storage which should be
in the upcoming 5.1 release.  You may find that this will provide
significant improvements, though it will depend upon the file
format/reader in question.


Regards,
Roger

--
Dr Roger Leigh -- Open Microscopy Environment
Wellcome Trust Centre for Gene Regulation and Expression,
College of Life Sciences, University of Dundee, Dow Street,
Dundee DD1 5EH Scotland UK   Tel: (01382) 386364

The University of Dundee is a registered Scottish Charity, No: SC015096