[ome-devel] Adding min/max information to PyramidTiffReader

[Melissa Linkert]

Hi Paul,

> > PyramidTiffReader was intended for a very specific case of pyramided
> > BigTIFF files, and won't accept all pyramided BigTIFFs by default - see
> > the last couple of lines in isThisType(RandomAccessInputStream).  Can
> > you confirm that PyramidTiffReader is being used for these files, and
> > not the "plain" TiffReader?
> 
> Yes, I'm sure the PyramidTiffReader is being used. I'm a colleague of
> Raimond Ravelli. He is one of the authors of the zebra fish em map,
> for which this reader was originally developed.

OK, great.

> > Whether store is an instance of IMinMaxStore will depend on the context;
> > during import into OMERO that should be the case, but if you're testing
> > with e.g. the showinf command line tool then that won't happen by
> > default.  I'd certainly expect that importing into OMERO with the above
> > addition would create the StatsInfo correctly, assuming that StatsInfo
> > hasn't been disabled during a command line import.
> 
> It's good to know that this indeed the case, as I was starting to
> suspect this. Is this even the case when you use the -minmax flag of
> showinf?

showinf's -minmax flag always calculates the actual minimum and maximum
values.

> > If PyramidTiffReader is definitely being used and the StatsInfo still
> > doesn't show up in OMERO, then I'd suggest opening a Bio-Formats pull
> > request with the changes you've made so far, so that we can better see
> > what the problem might be.
> 
> I've pushed my changes to a branch here:
> 
> https://github.com/M4I-nanoscopy/bioformats/commit/c8575e6ef17b29b63db88d52fa6b094b23b013eb
> 
> When running this I can see this in the log:
> 
> 2016-01-08 16:44:01,535 INFO  [
> loci.formats.ImageReader] (l.Server-3) PyramidTiffReader initializing
> 2016-01-08 16:44:01,536 INFO  [
> loci.formats.in.MinimalTiffReader] (l.Server-3) Reading IFDs
> 2016-01-08 16:44:01,536 INFO  [
> loci.formats.in.MinimalTiffReader] (l.Server-3) Populating metadata
> 2016-01-08 16:44:01,539 INFO  [
> loci.formats.in.BaseTiffReader] (l.Server-3) Populating OME metadata
> 2016-01-08 16:44:01,539 INFO  [
> loci.formats.in.PyramidTiffReader] (l.Server-3) Before minmax
> 
> The "Setting minmax" which should be logged does not show up.

Thank you for pushing the commit - I can reproduce the same behavior now.
The details of the problem and what we need to fix are here:

https://trello.com/c/QdCI9tHU/79-memoizer-does-not-pass-along-min-max-values

> You mention that StatsInfo should not be disabled during import.
> Hasn't StatsInfo been disabled automatically for any big files since
> OMERO 5.1?

As far as I understand, min/max calculation is always disabled for big
images, but StatsInfo creation can still occur (in theory, if min/max
values are available without calculation) unless the "--no-stats-info" option is used
(per https://www.openmicroscopy.org/site/support/omero5.2/sysadmins/in-place-import.html).
Based on the above, that doesn't seem to be the issue here though.

Regards,
-Melissa

On Fri, Jan 08, 2016 at 04:55:23PM +0100, Paul van Schayck wrote:
> Hi Melissa,
> 
> On Fri, Jan 8, 2016 at 4:22 PM, Melissa Linkert
> <melissa at glencoesoftware.com> wrote:
> > PyramidTiffReader was intended for a very specific case of pyramided
> > BigTIFF files, and won't accept all pyramided BigTIFFs by default - see
> > the last couple of lines in isThisType(RandomAccessInputStream).  Can
> > you confirm that PyramidTiffReader is being used for these files, and
> > not the "plain" TiffReader?
> 
> Yes, I'm sure the PyramidTiffReader is being used. I'm a colleague of
> Raimond Ravelli. He is one of the authors of the zebra fish em map,
> for which this reader was originally developed.
> 
> > Whether store is an instance of IMinMaxStore will depend on the context;
> > during import into OMERO that should be the case, but if you're testing
> > with e.g. the showinf command line tool then that won't happen by
> > default.  I'd certainly expect that importing into OMERO with the above
> > addition would create the StatsInfo correctly, assuming that StatsInfo
> > hasn't been disabled during a command line import.
> 
> It's good to know that this indeed the case, as I was starting to
> suspect this. Is this even the case when you use the -minmax flag of
> showinf?
> 
> > If PyramidTiffReader is definitely being used and the StatsInfo still
> > doesn't show up in OMERO, then I'd suggest opening a Bio-Formats pull
> > request with the changes you've made so far, so that we can better see
> > what the problem might be.
> 
> I've pushed my changes to a branch here:
> 
> https://github.com/M4I-nanoscopy/bioformats/commit/c8575e6ef17b29b63db88d52fa6b094b23b013eb
> 
> When running this I can see this in the log:
> 
> 2016-01-08 16:44:01,535 INFO  [
> loci.formats.ImageReader] (l.Server-3) PyramidTiffReader initializing
> 2016-01-08 16:44:01,536 INFO  [
> loci.formats.in.MinimalTiffReader] (l.Server-3) Reading IFDs
> 2016-01-08 16:44:01,536 INFO  [
> loci.formats.in.MinimalTiffReader] (l.Server-3) Populating metadata
> 2016-01-08 16:44:01,539 INFO  [
> loci.formats.in.BaseTiffReader] (l.Server-3) Populating OME metadata
> 2016-01-08 16:44:01,539 INFO  [
> loci.formats.in.PyramidTiffReader] (l.Server-3) Before minmax
> 
> The "Setting minmax" which should be logged does not show up.
> 
> You mention that StatsInfo should not be disabled during import.
> Hasn't StatsInfo been disabled automatically for any big files since
> OMERO 5.1?
> 
> Thanks,
> 
> Paul