[ome-users] Timeout for server-side scripts

[Josh Moore]

Hey Mark,

On Jan 20, 2011, at 12:01 AM, Woodbridge, Mark R wrote:

> I've looked into this a bit more closely and I was wrong about the script timing-out. Obviously this occurs when the script is run on the client,

How are you running the script on the client?

> but it's not a problem on the server.

Good to heart.

> The real problem appears to be the server running out of file handles. We're running 4.2.1 and it looks like a relevant fix is included in 4.2.2:
> 
> https://trac.openmicroscopy.org.uk/omero/ticket/2764
> 
> I won't be able to upgrade for a little while but I'll try again when I can.

Great. Let us know how it goes.

> One thing I did discover in the meantime was that if you don't set OMERO_TEMPDIR when running the server then it defaults to using your home directory:
> 
> http://trac.openmicroscopy.org.uk/omero/wiki/TempFileManager
> 
> In our case home directories are NFS mounted. Exports (when successful) are a lot faster now I've switched to using /tmp on the local disk. I think this should at least be documented in the installation notes, or the default order should be /tmp and then the home directory.

Understood. I added http://trac.openmicroscopy.org.uk/omero/ticket/3931 to more clearly document this. The thinking is that the omero user would typically be a local user and not an NFS mounted one, but that obviously doesn't always hold. Perhaps we can detect mounted directories on startup: http://trac.openmicroscopy.org.uk/omero/ticket/3932

Cheers,
~Josh

> Thanks,
> 
> Mark.
> ________________________________________
> From: Josh Moore [josh at glencoesoftware.com]
> Sent: 17 January 2011 19:03
> To: Woodbridge, Mark R
> Cc: ome-users at lists.openmicroscopy.org.uk
> Subject: Re: [ome-users] Timeout for server-side scripts
> 
> Hi Mark,
> 
> On Jan 17, 2011, at 12:25 PM, Woodbridge, Mark R wrote:
>> Script attached.
> 
> Great, much appreciated.
> 
>> It's launched from Insight and performs a batch export of a dataset to OME-TIFF files. The bottleneck seems to be the conversion (rather than writing the output to disk) so if this can be achieved more efficiently then perhaps I can avoid the timeout. Obviously a dataset can be arbitrarily big, so one solution might be to create a new session for each image.
> 
> It's true that the OME-TIFF generation can be the most time consuming, but it shouldn't cause your script to crash. Could you send an example of the out and err files that the script produces as well as your var/log/Processor-0.log?
> 
>> There is another potentially related issue - I sometimes see 'Too many open files' on the server when running the script, at which point I have to restart the server.
> 
> That's a known issue that should have been fixed. What version is your server?
> 
>> Obviously the script doesn't yet do things that it should, like checking to see if the files have already been exported.
> 
> No worries, it's good to see any examples of scripts that are floating around.
> 
> Cheers,
> ~Josh
> 
>> Mark.
>> ________________________________________
>> From: Josh Moore [josh.moore at gmx.de]
>> Sent: 14 January 2011 22:10
>> To: Woodbridge, Mark R
>> Cc: ome-users at lists.openmicroscopy.org.uk
>> Subject: Re: [ome-users] Timeout for server-side scripts
>> 
>> Hi Mark,
>> 
>> 
>> On Jan 14, 2011, at 6:28 PM, Woodbridge, Mark R wrote:
>>> I have a server-side Python script which seems to be timing-out after 10 minutes.
>> 
>> Would it possible for you to share the script or describe what it's doing? How is it being launched? From Insight? Are you repeatedly getting the same stacktrace in the logs which you could send along?
>> 
>>> Is there a way to either specify an increased timeout when calling client.getSession(),
>> 
>> It shouldn't be necessary, but we need to figure out what's going on.
>> 
>>> or to use more than one non-concurrent session in a script?
>> 
>> That's not necessary nor really possible.
>> 
>>> Many thanks,
>>> Mark.
>> 
>> Cheers,
>> ~Josh
>> <Definiens_Export.py>
>