[ome-users] Timeout for server-side scripts

[Woodbridge, Mark R]

I've looked into this a bit more closely and I was wrong about the script timing-out. Obviously this occurs when the script is run on the client, but it's not a problem on the server. The real problem appears to be the server running out of file handles. We're running 4.2.1 and it looks like a relevant fix is included in 4.2.2:

https://trac.openmicroscopy.org.uk/omero/ticket/2764

I won't be able to upgrade for a little while but I'll try again when I can.

One thing I did discover in the meantime was that if you don't set OMERO_TEMPDIR when running the server then it defaults to using your home directory:

http://trac.openmicroscopy.org.uk/omero/wiki/TempFileManager

In our case home directories are NFS mounted. Exports (when successful) are a lot faster now I've switched to using /tmp on the local disk. I think this should at least be documented in the installation notes, or the default order should be /tmp and then the home directory.

Thanks,

Mark.
________________________________________
From: Josh Moore [josh at glencoesoftware.com]
Sent: 17 January 2011 19:03
To: Woodbridge, Mark R
Cc: ome-users at lists.openmicroscopy.org.uk
Subject: Re: [ome-users] Timeout for server-side scripts

Hi Mark,

On Jan 17, 2011, at 12:25 PM, Woodbridge, Mark R wrote:
> Script attached.

Great, much appreciated.

> It's launched from Insight and performs a batch export of a dataset to OME-TIFF files. The bottleneck seems to be the conversion (rather than writing the output to disk) so if this can be achieved more efficiently then perhaps I can avoid the timeout. Obviously a dataset can be arbitrarily big, so one solution might be to create a new session for each image.

It's true that the OME-TIFF generation can be the most time consuming, but it shouldn't cause your script to crash. Could you send an example of the out and err files that the script produces as well as your var/log/Processor-0.log?

> There is another potentially related issue - I sometimes see 'Too many open files' on the server when running the script, at which point I have to restart the server.

That's a known issue that should have been fixed. What version is your server?

> Obviously the script doesn't yet do things that it should, like checking to see if the files have already been exported.

No worries, it's good to see any examples of scripts that are floating around.

Cheers,
~Josh

> Mark.
> ________________________________________
> From: Josh Moore [josh.moore at gmx.de]
> Sent: 14 January 2011 22:10
> To: Woodbridge, Mark R
> Cc: ome-users at lists.openmicroscopy.org.uk
> Subject: Re: [ome-users] Timeout for server-side scripts
>
> Hi Mark,
>
>
> On Jan 14, 2011, at 6:28 PM, Woodbridge, Mark R wrote:
>> I have a server-side Python script which seems to be timing-out after 10 minutes.
>
> Would it possible for you to share the script or describe what it's doing? How is it being launched? From Insight? Are you repeatedly getting the same stacktrace in the logs which you could send along?
>
>> Is there a way to either specify an increased timeout when calling client.getSession(),
>
> It shouldn't be necessary, but we need to figure out what's going on.
>
>> or to use more than one non-concurrent session in a script?
>
> That's not necessary nor really possible.
>
>> Many thanks,
>> Mark.
>
> Cheers,
> ~Josh
> <Definiens_Export.py>