[ome-users] Server error (500). Why is this displayed
Simon Li
s.p.li at dundee.ac.uk
Tue Jun 17 17:02:28 BST 2014
Hi John
I’m glad you’ve got it working! Your diagnostics look fine, so unless the problem re-appears I don’t think there’s anything else we can do.
Cheers
Simon
On 17 Jun 2014, at 16:35, John Webber (NBI) <John.Webber at nbi.ac.uk<mailto:John.Webber at nbi.ac.uk>> wrote:
Hi Simon,
Thanks again for your assistance with this issue.
I have made the first of the two changes that you suggested below – to fix the error handler to work with Python 2.4.
As well as making this change, I have also rebooted the server and cleared my browser cache. I’m not sure which of these was finally resolved the issue, but I don’t seem to be getting the “Server error[500]” any longer.
I have attached the output from “omero admin diagnostics` and also my apache config files, in case there is still another issue lurking!
Thanks
John
From: Simon Li [mailto:s.p.li at dundee.ac.uk]
Sent: 13 June 2014 11:59
To: John Webber (NBI)
Cc: ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>
Subject: Re: [ome-users] Server error (500). Why is this displayed
Hi John
I can’t see anything obviously wrong, so I can think of a couple of options:
1. Fix the error handler to work with Python 2.4:
Edit /opt/OMERO.server-4.4.8p1-ice33-b304/lib/python/omeroweb/feedback/views.py
and change the logging statement at line 159 from:
158 def handler404(request):
159 logger.warning('Not Found: %s' % request.path,
160 extra={
161 'status_code': 404,
162 'request': request
163 })
164 if request.is<http://request.is/>_ajax():
165 msg = traceback.format_exception(*sys.exc_info())[-1]
166 return HttpResponseNotFound(msg)
167
168 return page_not_found(request, "404.html”)
to:
158 def handler404(request):
159 logger.warning('Not Found: %s' % request.path)
160 if request.is<http://request.is/>_ajax():
161 msg = traceback.format_exception(*sys.exc_info())[-1]
162 return HttpResponseNotFound(msg)
163
164 return page_not_found(request, "404.html”)
If this works the server error 500 message should be more informative, and you’ll hopefully have a warning message in OMEROweb.log.
2. Go back to basics. Are you able to setup OMERO.web in development mode instead of running via Apache?
omero config set omero.web.application_server development
omero config set omero.web.debug True
Restart, and see if the problems still occur using http://server:4080/
If you need to keep the main instance running one thing you could try is running a second OMERO.web in development mode (but connecting to the same OMERO.server). This isn’t ideal, since the OMERO sessions will be shared between both OMERO.web instances:
cp -a OMERO.server-XXX OMERO.server-webdev
cd OMERO.server-webdev
rm -rf var
bin/omero config set omero.web.application_server development
bin/omero config set omero.web.debug True
# The main OMERO.web will be using port 4080 internally to communicate with Apache
bin/omero config set omero.web.application_server.port 14080
bin/omero web start
and go to http://server:14080/
Could you also send us the output from `omero admin diagnostics` and your apache config files, e;.g.
/etc/httpd/conf.d/<OMERO>.conf
/etc/httpd/conf.d/fastcgi.conf
Cheers
Simon
On 12 Jun 2014, at 15:53, John Webber (NBI) <John.Webber at nbi.ac.uk<mailto:John.Webber at nbi.ac.uk>> wrote:
Simon (et al)
Sorry for the delay in responding to your email below – it’s been a busy week.
I have done some more investigation into this issue.
I’ve found that,
• If I log off of the Omero server correctly, I do not encounter any issues when I attempt to log on again.
• Similarly, on different Omero server, I can close (crash) the browser window without logging off, and when I go back on to that site, I am automatically logged straight back in. I do not encounter the Server error (500).
• On this Omero server, if I close (crash) the browser window without logging off, when I go back on to that site I am NOT automatically logged in – instead the server displays the normal OMERO.web login (enter username and password) screen.
• When I enter my credentials on this OMERO.web login screen I then see the “Server error (500)” error. Once this screen has displayed, I can click “Webadmin” or “Webclient” and use the site as normal.
Please see attached two different log files, one from a successful log into Omero, and one from a login that encountered a “Server error (500)”.
The logs are the same, down to the following lines:
2014-06-12 14:02:21,978 INFO [ org.perf4j.TimingLogger] (l.Server-9) start[1402578141975] time[3] tag[omero.call.success.ome.logic.AdminImpl.getSecurityRoles]
2014-06-12 14:02:21,985 INFO [ ome.services.util.ServiceHandler] (l.Server-9) Rslt: ome.system.Roles at 67d1de8b<mailto:ome.system.Roles at 67d1de8b>
2014-06-12 14:02:22,019 INFO [ ome.services.blitz.fire.SessionManagerI] (l.Server-3) Found session locally: 5a1dc603-b2de-4dfc-99a6-6fe724e2521d
On the session that displays the “Server error (500)” error, the last of the lines above is not shown.
I have checked the Apache logs, and am not able to find any messages in either the /var/log/httpd/access_log or the /var/log/httpd/error_log at the time I experience the issue. The only logs being written to when these issues occur are the /var/log/httpd/ssl_request_log, which has the following entries:
[12/Jun/2014:15:49:42 +0100] 149.155.213.117 TLSv1 DHE-RSA-AES128-SHA "POST /webclient/login/?url=FwebclientF HTTP/1.1" -
[12/Jun/2014:15:49:43 +0100] 149.155.213.117 TLSv1 DHE-RSA-AES128-SHA "GET /webclient/login/FwebclientF HTTP/1.1" 6032
[12/Jun/2014:15:49:43 +0100] 149.155.213.117 TLSv1 DHE-RSA-AES128-SHA "GET /static/feedback/css/layout.css HTTP/1.1" -
and the /var/log/httpd/ssl_access_log, which has the following entries:
149.155.213.117 - - [12/Jun/2014:15:47:34 +0100] "POST /webclient/login/?url=FwebclientF HTTP/1.1" 302 -
149.155.213.117 - - [12/Jun/2014:15:47:34 +0100] "GET /webclient/login/FwebclientF HTTP/1.1" 200 6032
149.155.213.117 - - [12/Jun/2014:15:47:35 +0100] "GET /static/feedback/css/layout.css HTTP/1.1" 304 -
Does anyone have any ideas on what is going wrong with this server?
Thanks
John
From: Simon Li [mailto:s.p.li at dundee.ac.uk]
Sent: 09 June 2014 10:53
To: John Webber (NBI)
Cc: ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>
Subject: Re: [ome-users] Server error (500). Why is this displayed
Hi John
It sounds like you stopped OMERO.web successfully, I can't think of anything else that would be running other than django. The traceback error is occurring in the OMERO.web 404 error handler- it looks like the logging module in Python 2.4 doesn't support the additional 'extra' parameter used in omeroweb/feedback/views.py:160. Unfortunately this makes it difficult to figure out what the exact error is!
Could you check your Apache logs for clues?
/var/log/httpd/access_log and particularly /var/log/httpd/error_log
One other thing you could try is enabling debug mode in OMERO.web to see if that gives any additional information:
omero config set omero.web.debug True
but remember to disable it once the problem is fixed
Cheers
Simon
On 6 Jun 2014, at 16:20, John Webber (NBI) <John.Webber at nbi.ac.uk<mailto:John.Webber at nbi.ac.uk>> wrote:
Hi Simon,
Thanks for your quick response (as normal!) ;-)
I have stopped and restarted OMERO.web, as requested, using “<omero>/bin/omero web stop” and “<omero>/bin/omero web start”. Once I had stopped OMERO.web, all of the “django” processes did stop correctly. Are there any other processes that I should check as well?
OMERO.web stopped and restarted successfully, but the “/opt/Omero/var/log/OMEROweb.log” log file is STILL not being written to. There are also no other OMEROweb.log files on the system.
The “Server Error (500)” message that I am seeing includes the following error message:
Traceback (most recent call last):
File "/opt/OMERO.server-4.4.8p1-ice33-b304/lib/python/django/core/handlers/base.py", line 150, in get_response
response = callback(request, **param_dict)
File "/opt/OMERO.server-4.4.8p1-ice33-b304/lib/python/omeroweb/feedback/views.py", line 160, in handler404
extra={
File "/usr/lib64/python2.4/logging/__init__.py", line 983, in warning
apply(self._log, (WARNING, msg, args), kwargs)
TypeError: _log() got an unexpected keyword argument 'extra'
<WSGIRequest
GET:<QueryDict: {}>,
POST:<QueryDict: {}>,
COOKIES:{'sessionid': '249cf076455eda32b05b4212a84d86eb'},
META:{'DOCUMENT_ROOT': '/var/www/html',
'GATEWAY_INTERFACE': 'CGI/1.1',
'HTTPS': 'on',
'HTTP_ACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
'HTTP_ACCEPT_ENCODING': 'gzip,deflate,sdch',
'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.8',
'HTTP_CACHE_CONTROL': 'max-age=0',
'HTTP_CONNECTION': 'keep-alive',
'HTTP_COOKIE': 'sessionid=249cf076455eda32b05b4212a84d86eb',
'HTTP_HOST': 'open-omero.nbi.ac.uk<http://open-omero.nbi.ac.uk/>',
'HTTP_REFERER': 'https://open-omero.nbi.ac.uk/webclient/login/?url=FwebclientF',
'HTTP_USER_AGENT': 'Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36',
'PATH': '/sbin:/usr/sbin:/bin:/usr/bin',
'PATH_INFO': u'/webclient/login/FwebclientF',
'PATH_TRANSLATED': '/opt/OMERO.server-4.4.8p1-ice33-b304/var/omero.fcgi/webclient/login/FwebclientF',
'QUERY_STRING': '',
'REMOTE_ADDR': '149.155.213.117',
'REMOTE_PORT': '61082',
'REQUEST_METHOD': 'GET',
'REQUEST_URI': '/webclient/login/FwebclientF',
'SCRIPT_FILENAME': '/opt/OMERO.server-4.4.8p1-ice33-b304/var/omero.fcgi',
'SCRIPT_NAME': u'',
'SERVER_ADDR': '149.155.214.217',
'SERVER_ADMIN': 'root at localhost',
'SERVER_NAME': 'open-omero.nbi.ac.uk<http://open-omero.nbi.ac.uk/>',
'SERVER_PORT': '443',
'SERVER_PROTOCOL': 'HTTP/1.1',
'SERVER_SIGNATURE': '<address>Apache/2.2.3 (CentOS) Server at open-omero.nbi.ac.uk<http://open-omero.nbi.ac.uk/> Port 443</address>\n',
'SERVER_SOFTWARE': 'Apache/2.2.3 (CentOS)',
'wsgi.errors': <flup.server.fcgi_base.TeeOutputStream object at 0x10548090>,
'wsgi.input': <flup.server.fcgi_base.InputStream object at 0x10548510>,
'wsgi.multiprocess': True,
'wsgi.multithread': False,
'wsgi.run_once': False,
'wsgi.url_scheme': 'https',
'wsgi.version': (1, 0)}>”
Please can you advise how it is best to proceed with this issue?
Thanks
John
From: Simon Li [mailto:s.p.li at dundee.ac.uk<http://dundee.ac.uk/>]
Sent: 06 June 2014 15:06
To: John Webber (NBI)
Subject: Re: [ome-users] Server error (500). Why is this displayed
Hi John
You're right, I should've said OMEROweb.log. The fact that it's empty is a bit strange though. Is it possible you've got an old OMERO.web running from a previous server upgrade? Could you run `omero web stop`, then verify all relevant processes are killed (you can use something like `ps -ef | grep django` to check). If there are any still present you can try killing them manually, then restart OMERO.web?
If that doesn't work then could you copy and paste the error message from the dialog- if there's no private information it would be helpful if you could CC the ome-users list, so that people don't see a hanging conversation. If we still can't figure out the problem then server credentials could be useful.
Hope this helps,
Simon
On 6 Jun 2014, at 11:45, "John Webber (NBI)" <John.Webber at nbi.ac.uk<mailto:John.Webber at nbi.ac.uk>>
wrote:
Hi Simon,
I’m afraid I am not able to find a log file called OMERO_web.log! Where should this be located?
There is a logfile in <omero>/var/log called “OMEROweb.log”, but the date on this file is September, and the file is empty! There is also a file called “OMEROweb_request.log”, which the same date which is also empty!
I have looked through the blitz log, but this does not seem to contain any errors or warnings at the time when I see this message. Would you like me to send you a copy of that?
The server is actually available through our firewall, so I can give you credentials on the server for you to see the error occur, if you would like!
Thanks
John
From: Simon Li [mailto:s.p.li at dundee.ac.uk<http://dundee.ac.uk/>]
Sent: 06 June 2014 10:21
To: John Webber (NBI)
Cc: ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>
Subject: Re: [ome-users] Server error (500). Why is this displayed
Hi John
You should be able to login from multiple browsers/locations without errors. Could you send us your OMERO_web.log?
Thanks
Simon
On 6 Jun 2014, at 10:34, John Webber (NBI) <John.Webber at nbi.ac.uk<mailto:John.Webber at nbi.ac.uk>> wrote:
Hi,
I have built a new omero server, but during testing I sometimes see the following “Server Error (500)” screen when I log in. I am using Omero 4.4.8 with ice 3.3.
I notice that this seems to occur when I access the same user account from different computers or browsers. Is this because I have not logged out correctly from another sessions? Is there a way to remove the likelihood of users seeing this screen?
Thanks
John
<image001.png>
--------------------------------------------------------------------------------------------------------------------------------------------
John Webber
Computing and Information Services
NBI Partnership Ltd
Norwich Research park
Colney Lane
Norwich
NR4 7UH
E-Mail: john.webber at nbi.ac.uk<mailto:john.webber at nbi.ac.uk>
Web: http://www.nbi.ac.uk<http://www.nbi.ac.uk/>
Tel: +44(0)1603 450 818
Helpdesk (Tel): 1234 HPC Support (Web): http://hpc.nbi.ac.uk<http://hpc.nbi.ac.uk/> (NBI internal)
HPC Support (Tel): 2003 (NBI internal)
The NBI Partnership Ltd provides non-scientific services to the Institute of Food Research, the John Innes Centre, The Genome Analysis Centre and The Sainsbury Laboratory
_______________________________________________
ome-users mailing list
ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
The University of Dundee is a registered Scottish Charity, No: SC015096
The University of Dundee is a registered Scottish Charity, No: SC015096
The University of Dundee is a registered Scottish Charity, No: SC015096
<successful.login><500error.login>
The University of Dundee is a registered Scottish Charity, No: SC015096
<fastcgi.conf><httpd.conf><omero-admin-diagnostics.txt>
The University of Dundee is a registered Scottish Charity, No: SC015096
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20140617/ce1fc72b/attachment.html>
More information about the ome-users
mailing list