[ome-users] Core dump attempting to perfrom bulk upload to Omero

[Josh Moore]

On May 23, 2014, at 10:33 AM, John Webber (NBI) wrote:

> Hi Josh,

Hi John,

> Thanks for the quick response - as per normal! :-)

Gladly. Thanks for all the log data!

> Please see attached a zip file containing the files in my Omero log folder (/opt/Omero/var/log).  As well as the log files that normally reside in this location, I have also added the following files:
> 
> 	bulk-upload-script.log		The logfile for my bulk upload script - so you can see what that is doing.
> 	var-log-messages-file		The main server log (/var/log/messages) file.
> 	emails				The emails which were sent for these core dumps.
> 
> Any pointers you can give as to where things are going wrong would be gratefully received!  Do you also need to know the software versions that I have running, (for example Java and Python)?

Can you check that you still have the file:

May 22 17:20:11 v0246 abrt[2613]: Saved core dump of pid 1561 (/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.55.x86_64/jre/bin/java) to /var/spool/abrt/ccpp-2014-05-22-17:20:04-1561 (741801984 bytes)

If so, do you have gdb installed? It would be good to try to get a stack trace of what core dumped.

Here are the restarts in your server:

2014-05-22 10:17:23,158 INFO  [    ome.services.util.ServerVersionCheck] (      main) OMERO Version: 5.0.0-ice35-b19 Ready.
2014-05-22 10:58:26,515 INFO  [    ome.services.util.ServerVersionCheck] (      main) OMERO Version: 5.0.0-ice35-b19 Ready.
2014-05-22 11:58:43,658 INFO  [    ome.services.util.ServerVersionCheck] (      main) OMERO Version: 5.0.0-ice35-b19 Ready.
2014-05-22 14:52:47,780 INFO  [    ome.services.util.ServerVersionCheck] (      main) OMERO Version: 5.0.0-ice35-b19 Ready.
2014-05-22 14:55:34,882 INFO  [    ome.services.util.ServerVersionCheck] (      main) OMERO Version: 5.0.0-ice35-b19 Ready.
2014-05-22 15:15:26,549 INFO  [    ome.services.util.ServerVersionCheck] (      main) OMERO Version: 5.0.0-ice35-b19 Ready.
2014-05-22 16:38:54,761 INFO  [    ome.services.util.ServerVersionCheck] (      main) OMERO Version: 5.0.0-ice35-b19 Ready.
2014-05-22 17:11:40,885 INFO  [    ome.services.util.ServerVersionCheck] (      main) OMERO Version: 5.0.0-ice35-b19 Ready.
2014-05-22 17:20:38,436 INFO  [    ome.services.util.ServerVersionCheck] (      main) OMERO Version: 5.0.0-ice35-b19 Ready.
2014-05-22 17:23:23,171 INFO  [    ome.services.util.ServerVersionCheck] (      main) OMERO Version: 5.0.0-ice35-b19 Ready.


If they match any pattern you can detect, please let me know.

Also, has there been any system maintenance recently? New Java versions, etc?
Cheers,
~Josh.


> Thanks
> John
> 
> 
> 
> 
> -----Original Message-----
> From: Josh Moore [mailto:josh at glencoesoftware.com] 
> Sent: 22 May 2014 21:23
> To: John Webber (NBI)
> Subject: Re: [ome-users] Core dump attempting to perfrom bulk upload to Omero
> 
> 
> On May 22, 2014, at 6:17 PM, John Webber (NBI) wrote:
> 
>> Hi,
> 
> Hi John,
> 
>> I am currently testing the same process on a new server which is built with Omero 5.0 and Ice 3.5.  I am seeing the following error, but not after a consistent time:
>> 
>> Using session feccfbae-b08e-4298-9f3c-5f31139af486 
>> (webberj at localhost:4064). Idle timeout: 10.0 min. Current group: 
>> system Previous session expired for webberj on localhost:4064
>> InternalException: Failed to connect: exception 
>> ::Glacier2::CannotCreateSessionException
>> {
>>   reason = Server not fully initialized }
> 
> This message is coming from the Java process and means that it's just restarted. Likely your server is bouncing, up/down up/down. Could you send us the OMERO log files under var/log zipped up, incl master.out and master.err?
> 
> 
>> A core dump is then generated.  The following are the log entries (/var/log/messages) when this occurs:
>> ...
>> The email that is sent as a result of this crash  is very long, but I have attached the first part of an example to this email.
>> 
>> Does anyone have any ideas why Omero / Ice is crashing in this way?  Does this look like an Omero Issue, an Ice issue or a Java issue?
> 
> It's the Java process which is aborting. More of the sos email could be useful if you could zip that up as well. For anyone who's wondering about OS & versions for a comparison to their own system:
> 
> executable:     /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.55.x86_64/jre/bin/java
> kernel:         3.10.34-11.el6.centos.alt.x86_64
> 
> 
>> Thanks
>> John
> 
> Cheers,
> ~Josh
> <logfile.tar.gz>