[ome-users] Fwd: .ND2 reader in BioFormats

[Melissa Linkert]

Hi Christophe,

> I am using Bio-Formats 5.0.4 indeed. Have tried from a vanilla IJ1 install
> (as suggested on the IJ mailing list) but I see the same memory usage
> pattern : some saw-teeth usage around 100MB, then using ~3 times the stack
> size (still reading the header), then if I don't max out memory at this
> stage it starts reading planes up to ~1x stack size memory usage and
> opening.
> 
> These are 1-channel streamed stacks from an N-STORM acquisition under NIS
> Elements 4.2, with between 50,000 and 120,000 256x256 16-bit images. As
> regards Bio-Formats import options, everything is unchecked and I ask for a
> regular ImageJ stack as output.

Thank you for the additional information.  I strongly suspect that the
issue then is the number of planes (and the metadata associated with
each plane), and not so much the raw size of the file.  Unfortunately,
I still haven't been able to reproduce the problem, but admittedly we
don't have any .nd2 files with more than ~12,000 images.

I hate to ask given the size of the files, but would you be willing to
upload one of the smaller ones for us to take a look at?  I am sending
FTP information privately in case it's needed.

Regards,
-Melissa

On Tue, Sep 09, 2014 at 11:48:03AM +0200, Christophe Leterrier wrote:
> Hi Melissa,
> 
> I am using Bio-Formats 5.0.4 indeed. Have tried from a vanilla IJ1 install
> (as suggested on the IJ mailing list) but I see the same memory usage
> pattern : some saw-teeth usage around 100MB, then using ~3 times the stack
> size (still reading the header), then if I don't max out memory at this
> stage it starts reading planes up to ~1x stack size memory usage and
> opening.
> 
> These are 1-channel streamed stacks from an N-STORM acquisition under NIS
> Elements 4.2, with between 50,000 and 120,000 256x256 16-bit images. As
> regards Bio-Formats import options, everything is unchecked and I ask for a
> regular ImageJ stack as output.
> 
> Thanks for your help,
> 
> Christophe
> 
> 2014-09-08 23:44 GMT+02:00 Melissa Linkert <melissa at glencoesoftware.com>:
> 
> > Hi Christophe,
> >
> > > I'm trying to open a big Nikon .nd2 files in Fiji using Bio-Formats (not
> > > the new SciFiIO) in Fiji, Windows 7 64 bits, 64GB RAM. The "Reading file
> > > header part" usually peaks well above the actual file size with 17.6 GB
> > (on
> > > ImageJ memory monitor) for a 6.3 GB nd2 file. For a 15.9 GB file, it
> > maxes
> > > out the 60 GB allocated to Fiji. As it is the header reading part, it
> > does
> > > not make a difference to try to open as a virtual stack or not.
> >
> > Thank you for the bug report.  Could you please confirm which version of
> > Bio-Formats you're using?  Using 5.0.4, we've been able to initialize 8.5
> > GB
> > files with <1GB of RAM, so if you're using an older version then
> > upgrading may solve the problem.
> >
> > If you see the same behavior with 5.0.4, which options are selected in
> > the "Bio-Formats Import Options" window?
> >
> > Regards,
> > -Melissa
> >
> > On Mon, Sep 08, 2014 at 12:00:48PM +0200, Christophe Leterrier wrote:
> > > I have a question regarding Bio-Formats regarding memory use of Nikon
> > .nd2
> > > files, and Curtis Rueden advised me to post it here rather than on the
> > > ImageJ mailing list.
> > >
> > > thanks for your help,
> > >
> > > --
> > > Christophe Leterrier
> > > Chercheur
> > > Equipe Architecture des Domaines Axonaux
> > > CRN2M CNRS UMR 7286 - Aix Marseille Université
> > >
> > > ---------- Forwarded message ----------
> > > From: Christophe Leterrier <christophe.leterrier at gmail.com>
> > > Date: 2014-09-05 16:42 GMT+02:00
> > > Subject: .ND2 reader in BioFormats
> > > To: ImageJ Mailing List <imagej at list.nih.gov>
> > >
> > >
> > > Hi,
> > >
> > > This is primarily a Bio-Formats question.
> > >
> > > I'm trying to open a big Nikon .nd2 files in Fiji using Bio-Formats (not
> > > the new SciFiIO) in Fiji, Windows 7 64 bits, 64GB RAM. The "Reading file
> > > header part" usually peaks well above the actual file size with 17.6 GB
> > (on
> > > ImageJ memory monitor) for a 6.3 GB nd2 file. For a 15.9 GB file, it
> > maxes
> > > out the 60 GB allocated to Fiji. As it is the header reading part, it
> > does
> > > not make a difference to try to open as a virtual stack or not.
> > >
> > > Is this a known situation? What are my options for opening these files?
> > >
> > > Thanks for your help,
> > >
> > > Christophe
> >
> > > _______________________________________________
> > > ome-users mailing list
> > > ome-users at lists.openmicroscopy.org.uk
> > > http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
> >
> >