[ome-users] OMERO-CellProfiler: too many open files --> OMERO-Matlab: file descriptor leak; SOLVED!

Tue Jun 2 12:25:55 BST 2009

Dear all,

My sincerest apologies for overlooking Josh Moore's solution to my problem on the user mailinglist! I didn't receive it in my inbox.

Thanks for your help Josh!
Bram

---

Hi Bram,

thanks for the extra info. That pretty much solves the puzzle. Ice
does in fact have an issue with too many file descriptors at
times. The differences you are seeing may be due to garbage collection
in the various languages. For example, in Python your example works
fine:

    >>> for i in range(300): omero.client("localhost")
    <omero.client object at 0x70870>
    <omero.client object at 0x6736ef0>
    ...

But if I modify it to prevent garabge collection, I get the exception.

    >>> [ omero.client("localhost") for i in range(300) ]                         
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/Users/josh/GlencoeSoftware.git/git/omero.git/dist/lib/python/omero/__init__.py", line 113, in __init__
        self._initData(id)
      File "/Users/josh/GlencoeSoftware.git/git/omero.git/dist/lib/python/omero/__init__.py", line 220, in _initData
        self.__oa = self.__ic.createObjectAdapter("omero.ClientCallback")
      File "/opt/local/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/Ice.py", line 306, in createObjectAdapter
        adapter = self._impl.createObjectAdapter(name)
    Ice.SocketException: Ice.SocketException:
    Too many open files

To guarantee that the Ice.Communicator and the Ice.ObjectAdapter are
properly closed, always be sure to call client.closeSession(),
regardless of any exceptions or errors.

Hope that helps.
~Josh.

---
________________________________________
From: Bram Gerritsen
Sent: Tuesday, June 02, 2009 11:40 AM
To: Bram Gerritsen; ome-users at lists.openmicroscopy.org.uk
Subject: RE: OMERO-CellProfiler: too many open files --> OMERO-Matlab: file descriptor leak; SOLVED?

Dear all,

Previously I reported that there is no leak when creating sessions in python or java. This holds true for python, but *not* for java. After creating another separate java program to test the file descriptor leak I discovered that calling "omero.client.closeSession()" solves the problem. It was necessary to modify the "OmeroLoader.m" file to no longer use the "createOmeroJavaService" function since a reference to the client variable is needed to call "closeSession()".

I don't know why it is not necessary to call "omero.client.closeSession()" in python. The following code snippet does not result in an exception when a similar one in java does:

---

import omero
for i in range(1, 1000):
     omero.client('localhost', 4063)

---

Kind Regards,
Bram Gerritsen
________________________________________
From: Bram Gerritsen
Sent: Friday, May 29, 2009 6:19 PM
To: ome-users at lists.openmicroscopy.org.uk
Subject: OMERO-CellProfiler: too many open files --> OMERO-Matlab: file descriptor leak

Dear all,

It looks like my previously reported error is a file descriptor leak that is limited to matlab-java. There are no problems when repetitively creating omero sessions in python or java. In fact calling the "omero.client" function in matlab is enough to create file descriptors that will not be closed (not even after running the java garbage collector). The problem persists also when Matlab uses a different jvm from the default (in my case I tried OpenJDK 6.0).

Example matlab code that produces the exception:
---

for i=1:300, omero.client('localhost', 4063)
end

---

Thanks in advance for your support!
Bram Gerritsen
________________________________________
From: ome-users-bounces at lists.openmicroscopy.org.uk [ome-users-bounces at lists.openmicroscopy.org.uk] On Behalf Of b.gerritsen at nki.nl [b.gerritsen at nki.nl]
Sent: Friday, May 29, 2009 12:02 PM
To: ome-users at lists.openmicroscopy.org.uk
Subject: [ome-users] OMERO-CellProfiler: too many open files

Dear all,

When I analyze many images from the omero db in CellProfiler, the run fails after about 50 to 100 images with an Ice.SocketException. In the same error report a java.io.Exception is shown telling me there are too many open files. Closer inspection of matlab shows there are 16 files opened with every call to "createOmeroJavaService", but these files are not closed during the run, not even after a call to "session.close()". As soon as matlab tries to open more than 1024 files I get the exception.

Here is some more information about the system I'm running Omero on:

Red Hat Enterprise Linux 5.3
Omero Beta 4.0.3
Postgresql 8.1.4
ZeroC Ice 3.3.1
Python 2.4.3
Matlab 7.8.0 (R2009a)
CellProfiler (developer version) 1.0.5811 Branch 1.0.6027

I had to change the OmeroLoader.m, numImages.m, and CPOMEROimread.m files to be able to analyze images from omero beta 4 in cellprofiler (apparently the matlab scripts from "http://svn.openmicroscopy.org.uk/svn/omero/tags/omero-4.0.3/" are not compatible with the beta 4 release nor with recent versions of cellprofiler).

Thanks in advance!
Bram Gerritsen
_______________________________________________
ome-users mailing list
ome-users at lists.openmicroscopy.org.uk
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users