[ome-users] OMERO.qa - new comment

[Sebastien Besson (Staff)]

Hi Martin,

Looking at the content of the server logs, there are lots of "Too many open files"
exceptions in it which are likely related to the import failures including pyramid
generation failures.

What is your `ulimit -a` output? Could you try increasing the maximum number of
open files per process and see if that fixes your issues?

See also
https://www.openmicroscopy.org/site/support/omero5.0/sysadmins/limitations.html#limitations-openfiles

Best,
Sebastien


On 24 Feb 2015, at 20:14, Martin Schorb <martin.schorb at embl.de<mailto:martin.schorb at embl.de>> wrote:

Hi Sebastien,

the only log file that I found something related to the import s
blitz.log which is raw 293MB. I have uploaded it here:
* snip *

let me know if I can provide more useful files.

Martin

Am 2/24/15 um 5:44 PM schrieb Sebastien Besson:
Hi Martin,

I tried to reproduce your failures with a 5.0.8 server while performing an in-place
import of 6000 fake files but could not reproduce. I will give it another go with tiff
files tomorrow.

In the meantime, do you have access to the server logs corresponding to the failing
import? Would it be possible to upload them for our examination?

Best,
Sebastien

On 23 Feb 2015, at 20:48, Josh Moore <josh at glencoesoftware.com<mailto:josh at glencoesoftware.com>> wrote:

Forwarding conversation to the list.


---------- Forwarded message ----------
From: Martin Schorb <martin.schorb at embl.de<mailto:martin.schorb at embl.de>>
Date: Mon, Feb 23, 2015 at 9:45 PM
Subject: Re: OMERO.qa - new comment
To: Josh Moore <josh at glencoesoftware.com<mailto:josh at glencoesoftware.com>>


I have a set of ~6000 tif files from a FIB-SEM. 15 MB each that I try
to import using
OMERO.server/bin/omero -s localhost -u $user import -- -d 1
--transfer=ln_s $imagedir

It manages to do so for part of the files but randomly exits with the
following complaint (first line shows last successful import):
2015-02-20 17:41:53,539 3097900 [l.Client-0] INFO
ormats.importer.cli.LoggingImportMonitor - IMPORT_DONE Imported file:
/home/emcf/images/MB1-ROI9-FIBSEM-AliCropAli-InvCLAHE/MB1-ROI9-FIBSEM_s1000-1999_InvCLAHE0781.tif
Imported pixels:
3197
Other imported objects:
Fileset:3198
Image:3197

...

2015-02-20 17:41:53,539 3097900 [l.Client-0] INFO
ome.formats.importer.cli.ErrorHandler - Number of errors: 0
2015-02-20 17:41:53,558 3097919 [ main] INFO
ome.formats.OMEROMetadataStoreClient - Call context: {omero.group:3}
2015-02-20 17:41:53,565 3097926 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - FILESET_UPLOAD_PREPARATION
2015-02-20 17:41:53,689 3098050 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - FILESET_UPLOAD_START
2015-02-20 17:41:53,690 3098051 [ main] INFO
s.importer.transfers.SymlinkFileTransfer - Transferring
/home/emcf/images/MB1-ROI9-FIBSEM-AliCropAli-InvCLAHE/MB1-ROI9-FIBSEM_s1000-1999_InvCLAHE0764.tif...
2015-02-20 17:41:53,707 3098068 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED:
/home/emcf/images/MB1-ROI9-FIBSEM-AliCropAli-InvCLAHE/MB1-ROI9-FIBSEM_s1000-1999_InvCLAHE0764.tif
2015-02-20 17:41:53,813 3098174 [ main] ERROR
ome.formats.importer.cli.ErrorHandler - FILE_EXCEPTION:
/home/emcf/images/MB1-ROI9-FIBSEM-AliCropAli-InvCLAHE/MB1-ROI9-FIBSEM_s1000-1999_InvCLAHE0764.tif
omero.ResourceError: null
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method) ~[na:1.7.0_75]

....

I run the import command as the omero system user on the actual server
(thru ssh). It succeeds with 3197 files and then causes the problem. The
target dir in ManagedRepositories is empty and writable, there is no
quota and plenty of free disk space.

Martin


https://www.openmicroscopy.org/qa2/qa/feedback/10523/?token=515b9ee71de697b8f25684d282d61532
_______________________________________________
ome-users mailing list
ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users





Dr Sébastien Besson
Open Microscopy Environment / Harvard Medical School
Wellcome Trust Centre for Gene Regulation and Expression,
College of Life Sciences, University of Dundee, Dow Street,
Dundee DD1 5EH Scotland UK   Tel: (01382) 386364


The University of Dundee is a registered Scottish Charity, No: SC015096
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20150225/e49692a7/attachment.html>