[ome-users] -|EXT|- Re: Ticket ID: 001-00-019862__Issue importing large Multi-point scans into Fiji___Doesn't occur on smaller datasets

[WAINWRIGHT James]

Hi David,

I have now repeated the problem, which is independent of using a Virtual Stack.

The problem happens somewhere between 57GB and 95GB – for example, a 57GB file opens fine, but the 95GB file generates the following error message:

[cid:image001.png at 01D423FD.78CB8990]

This is using the latest netCDF from https://www.unidata.ucar.edu/downloads/netcdf/netcdf-java-4/index.jsp

How can I provide more information on how to troubleshoot this?

Many thanks!

James

James Wainwright
Global Applications Specialist - Microscopy Systems

Tel:        +44 (0) 2890 237 126 ext. 2130
Skype:  andor.j.wainwright
Mob:     +44 (0) 7834 710 834
Web:     https://www.andor.com/microscopy-systems

[logo]

From: David Gault (Staff) [mailto:d.gault at dundee.ac.uk]
Sent: 23 July 2018 18:54
To: OME User Support List <ome-users at lists.openmicroscopy.org.uk>
Cc: row_microscopy_support <row_microscopy_support at andor.com>; WILDE Geraint <g.wilde at andor.com>; WAINWRIGHT James <J.Wainwright at andor.com>
Subject: -|EXT|- Re: [ome-users] Ticket ID: 001-00-019862__Issue importing large Multi-point scans into Fiji___Doesn't occur on smaller datasets

Hi James,

In the Bio-Formats importer options there is a section labelled "Memory management”, from this if you select the option “Use Virtual Stack” then it will load each frame as required rather than reading the entire file at once.

With Thanks,
David Gault

On 23 Jul 2018, at 15:15, WAINWRIGHT James <J.Wainwright at ANDOR.COM<mailto:J.Wainwright at ANDOR.COM>> wrote:

Just to add that these are IMS files and that we have “patched” FIJI with the latest netCDF file with the jar from here:

https://www.unidata.ucar.edu/downloads/netcdf/netcdf-java-4/index.jsp

James

James Wainwright
Global Applications Specialist - Microscopy Systems


Tel:        +44 (0) 2890 237 126 ext. 2130
Skype:  andor.j.wainwright
Mob:     +44 (0) 7834 710 834
Web:     https://www.andor.com/microscopy-systems

<image002.jpg>

From: row_microscopy_support
Sent: 23 July 2018 15:05
To: ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>
Cc: row_microscopy_support <row_microscopy_support at andor.com<mailto:row_microscopy_support at andor.com>>; WILDE Geraint <g.wilde at andor.com<mailto:g.wilde at andor.com>>; WAINWRIGHT James <J.Wainwright at andor.com<mailto:J.Wainwright at andor.com>>
Subject: Ticket ID: 001-00-019862__Issue importing large Multi-point scans into Fiji___Doesn't occur on smaller datasets

Hi,

We have a customer in Australia that is acquiring large Multi-point datasets. Can be upwards of 500GB data sets each time.

They capture full Frame, no Binning (1024x1024 pixels), dual channel ( 2 simultaneous cameras acquiring) and 60-100 Z planes on multiple X/Y positions in an overnight (timelapse) protocol. These files can be upwards of 80Gb per XY position.

The problem they have is when they try and import these large data sets to FIJI it fails, however if they perform a similar experiment, only smaller in size there is no issue.

After asking the customer to perform a couple of tests for us, this is what they discovered:

“Definitely looks to be a file size issue.  Tried the smaller 1.5Gb files yesterday and it worked fine.  Did a longer experiment (same experimental setup just with more z-slices and more time points so the file was around 60Gb and these are failing.   The only other difference was that the small files that worked had been captured using two channel sequential while the large files that don’t work were captured with dual channel simultaneous capture.”

Any advice/workaround you can supply for handling these larger datasets would be appreciated.

Best,

Phil Mullan
Product Support Engineer (Microscopy Systems)
Europe, Middle East & Africa
Tel: +44 (0) 28 9023 7126 Ext. 2732
Fax: +44 (0) 28 9031 0792
Web:  andor.com/support<http://www.andor.com/>

<image003.jpg>

___________________________________________________________________________
This e-mail is confidential and is for the addressee only.   Please refer to
www.oxinst.com/email-statement<http://www.oxinst.com/email-statement> for regulatory information. _______________________________________________
ome-users mailing list
ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users


The University of Dundee is a registered Scottish Charity, No: SC015096
___________________________________________________________________________
This e-mail is confidential and is for the addressee only.   Please refer to
www.oxinst.com/email-statement<http://www.oxinst.com/email-statement> for regulatory information.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20180725/dbb6bb3a/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 15256 bytes
Desc: image001.png
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20180725/dbb6bb3a/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 8737 bytes
Desc: image002.jpg
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20180725/dbb6bb3a/attachment.jpg>