[ome-users] OMERO-CellProfiler: too many open files OMERO-Matlab: file descriptor leak

[josh.moore at gmx.de]

Hi Bram,

thanks for the extra info. That pretty much solves the puzzle. Ice
does in fact have an issue with too many file descriptors at
times. The differences you are seeing may be due to garbage collection
in the various languages. For example, in Python your example works
fine:

    >>> for i in range(300): omero.client("localhost")
    <omero.client object at 0x70870>
    <omero.client object at 0x6736ef0>
    ...

But if I modify it to prevent garabge collection, I get the exception.

    >>> [ omero.client("localhost") for i in range(300) ]                         
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/Users/josh/GlencoeSoftware.git/git/omero.git/dist/lib/python/omero/__init__.py", line 113, in __init__
        self._initData(id)
      File "/Users/josh/GlencoeSoftware.git/git/omero.git/dist/lib/python/omero/__init__.py", line 220, in _initData
        self.__oa = self.__ic.createObjectAdapter("omero.ClientCallback")
      File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/Ice.py", line 306, in createObjectAdapter
        adapter = self._impl.createObjectAdapter(name)
    Ice.SocketException: Ice.SocketException:
    Too many open files

To guarantee that the Ice.Communicator and the Ice.ObjectAdapter are
properly closed, always be sure to call client.closeSession(),
regardless of any exceptions or errors.

Hope that helps.
~Josh.


b.gerritsen at nki.nl writes:
 > Dear all,
 > 
 > It looks like my previously reported error is a file descriptor
 > leak that is limited to matlab-java. There are no problems when
 > repetitively creating omero sessions in python or java. In fact
 > calling the "omero.client" function in matlab is enough to create
 > file descriptors that will not be closed (not even after running
 > the java garbage collector). The problem persists also when Matlab
 > uses a different jvm from the default (in my case I tried OpenJDK
 > 6.0).
 > 
 > Example matlab code that produces the exception:
 > ---
 > 
 > for i=1:300, omero.client('localhost', 4063)
 > end
 > 
 > ---
 > 
 > Thanks in advance for your support!
 > Bram Gerritsen


 > From: ome-users-bounces at lists.openmicroscopy.org.uk [ome-users-bounces at lists.openmicroscopy.org.uk] On Behalf Of b.gerritsen at nki.nl [b.gerritsen at nki.nl]
 > Sent: Friday, May 29, 2009 12:02 PM
 > To: ome-users at lists.openmicroscopy.org.uk
 > Subject: [ome-users] OMERO-CellProfiler: too many open files
 > 
 > Dear all,
 > 
 > When I analyze many images from the omero db in CellProfiler, the
 > run fails after about 50 to 100 images with an
 > Ice.SocketException. In the same error report a java.io.Exception
 > is shown telling me there are too many open files. Closer
 > inspection of matlab shows there are 16 files opened with every
 > call to "createOmeroJavaService", but these files are not closed
 > during the run, not even after a call to "session.close()". As soon
 > as matlab tries to open more than 1024 files I get the exception.
 > 
 > Here is some more information about the system I'm running Omero on:
 > 
 > Red Hat Enterprise Linux 5.3
 > Omero Beta 4.0.3
 > Postgresql 8.1.4
 > ZeroC Ice 3.3.1
 > Python 2.4.3
 > Matlab 7.8.0 (R2009a)
 > CellProfiler (developer version) 1.0.5811 Branch 1.0.6027
 > 
 > I had to change the OmeroLoader.m, numImages.m, and CPOMEROimread.m
 > files to be able to analyze images from omero beta 4 in
 > cellprofiler (apparently the matlab scripts from
 > "http://svn.openmicroscopy.org.uk/svn/omero/tags/omero-4.0.3/" are
 > not compatible with the beta 4 release nor with recent versions of
 > cellprofiler).
 > 
 > Thanks in advance!
 > Bram Gerritsen