[ome-users] Problem with omero *downloads* from web - still having problems...

[Aleksandra Tarkowska (Staff)]

Hi Jake,

Could you give us more details what is exactly not working? Did you try gevent as well?
Which OS, Python, Django, gunicorn, gevent/eventlet?

Please provide us with 'ps aux | grep django.pid' output and matching log files showing failure.
Ideally if you run gunicorn in debug mode to give more accurate details.


Ola
Software Engineer
Open Microscopy Environment
University of Dundee

On 22 Jan 2016, at 10:44, Jake Carroll <jake.carroll at uq.edu.au<mailto:jake.carroll at uq.edu.au>> wrote:

Hi again,



It looks like the suggestions below do not solve the problem. I have tried with many different combinations of the eventlet string and nothing would appear to fix the issue.

Do you have any other suggestions?

Thanks!

-jc





Today’ Topics:

 1. Format for arguments for OMERO model objects (Paul Kibet Korir)
 2. Re: Problem with omero *downloads* from web

Message: 2
Date: Thu, 14 Jan 2016 13:38:48 +0000
From: "Aleksandra Tarkowska (Staff)" <A.Tarkowska at dundee.ac.uk<mailto:A.Tarkowska at dundee.ac.uk>>
To: OME User Support List <ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>>
Subject: Re: [ome-users] Problem with omero *downloads* from web
Message-ID: <BD2B5F53-7A4B-4960-9E15-F816321F3895 at dundee.ac.uk<mailto:BD2B5F53-7A4B-4960-9E15-F816321F3895 at dundee.ac.uk>>
Content-Type: text/plain; charset="utf-8"

Hi Jace and Dave

Sorry for a slow response.

Thanks again for reporting download issue. Failing download is not related to the size of 1GB but Gunicorn worker timeout 30 sec. Your nginx log should show error that upstream prematurely closed connection while reading...

and gunicorn log will say: [CRITICAL] WORKER TIMEOUT (pid:2701)

You can test it by starting web with custom timeout http://docs.gunicorn.org/en/stable/settings.html#timeout
bin/omero web start --wsgi-args ' -t 5 --error-logfile=/home/omero/omero_dir/var/log/g_error.log '


By default OMERO.web runs with gunicorn 5 sync workers with single thread and default timeout 30 sec. Sync workers are preferred for applications that perform complex calculations and async for one that bound (I/O). So you basically need to decide your users will use web for.

You can resolve this problem in two ways depends on download activity and scale by choosing appropriate parameters:

1. async workers:

$ pip install (gevent|eventlet)
$ bin/omero web start ?-workers INT --wsgi-args ' ?-worker-class (gevent|eventlet) '


2. sync workers with multiple threads (this option is available since gunicorn 19):
(this is less fabulous as request will take up CPU core once is waiting)

$ bin/omero web start ?-workers INT --wsgi-args ? --timeout INT --threads INT '
depends on python version you may have to pip install futures


For more details how to calculate number of workers, threads, etc please refer to the documentation http://docs.gunicorn.org/en/stable/settings.html

You can also run benchmarks tool such as weighttp or Apache ab and do e.g 1000 requests with 100 parallel requests, changing between different kind of workers and see what happens performance-wise.

We will definitely update our documentation to make it clear.


Ola
Software Engineer
Open Microscopy Environment
University of Dundee
_______________________________________________
ome-users mailing list
ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users


The University of Dundee is a registered Scottish Charity, No: SC015096
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20160124/2c02e158/attachment.html>