[ome-devel] Tables fail to startup up due to missing InternalRepository

Colin Blackburn C.Blackburn at dundee.ac.uk
Mon May 20 10:00:02 BST 2013 

Hi Julien,

> So, I re-tested without NFS and ... it worked ! Tables ran correctly and, moreover, it solved another problem : the pyramid creation system also worked.
> I also checked the .lock file on the binary repository. It changes when it's on a local drive, but not when it's on NFS. It's definitely our problem.

Thanks for letting us know and we're glad things are now working for you.

> By the way, there isn't any Exception in our Blitz-0.log.

There should be something logged there. If you still have the relevant log and you are able to send it to us (direct email to me would be fine) and point then it might be useful to us in improving our logging.

Cheers,

Colin



On 17 May 2013, at 13:32, Julien Jourde <julien.jourde at pasteur.fr> wrote:

> Le 16/05/13 10:59, Josh Moore a écrit :
>> On May 13, 2013, at 2:58 PM, Julien Jourde wrote:
>> 
>>> Le 13/05/13 12:22, Colin Blackburn a écrit :
>>> 
>>> Hi Colin,
>>>> Hi Julien,
>> Hi Julien,
> Hi Josh,
>> 
>>>>> I'm running a 4-4-7 OMERO server and tables fail to startup. It's exactly the same problem as the one in the ticket 3690, which should be fixed :-/
>>>> Sorry, that's not the clearest ticket in suggesting a solution!
>>>> 
>>>>> Any idea / advise / help is very welcome, thanks :)
>>>> It is possible that you have a lingering icegridregistry process running.
>>> There is no lingering icegridregistry process running (I didn't find any at least).
>>>> You could try killing that process (and any other lingering ice processes). If that doesn't address the issue then deleting the entire var directory under your server install might also help. On your installation it looks like that would be:
>>>> 
>>>> /home/omero/apps/omero/OMERO.server-4.4.7-ice34-b246/var
>>> Unfortunately, it didn't solve my problem :-/ But I noticed something strange comparing my production server (the one where Tables fails to startup) with my test server (the one without any problem) :
>>> - I did the same installation on each server : I installed a 4-4-5 version, then I upgraded it to a 4-4-6 version and finally a 4-4-7 one.
>>> - I wanted to have a look on the /MyBinaryRepository/.omero/repository/XXX-XXX-XXX/repo_uuid, to compare it with the one in my logs. That's where I noticed that there are 3 different "XXX-XXX-XXX" repositories on my test server (probably corresponding to the different versions of OMERO), but only one on my production server ! (for the production server, I moved my bynary repository to an NFS partition but between the 4-4-6 and the 4-4-7 versions).
>> Ah a smoking gun: NFS.
>> cF. https://www.openmicroscopy.org/site/support/omero4/sysadmins/unix/server-binary-repository.html#locking-and-remote-shares
>> 
>> Would it be possible to re-test without NFS in the mix, i.e. by copying (a portion of) your binary repository back to a local drive and starting up a copy of the database again it? Are you in fact seeing NullPointerExceptions in your Blitz-0.log?
> So, I re-tested without NFS and ... it worked ! Tables ran correctly and, moreover, it solved another problem : the pyramid creation system also worked.
> I also checked the .lock file on the binary repository. It changes when it's on a local drive, but not when it's on NFS. It's definitely our problem.
> By the way, there isn't any Exception in our Blitz-0.log.
> 
> Cheers,
> 
> JJ
>> 
>> Cheers,
>> ~Josh.
>> 
>>> Thanks for your help :)
>>> 
>>> Cheers,
>>> 
>>> JJ
>>>> Please do let us know whether this solves your problem.
>>>> 
>>>> Cheers,
>>>> 
>>>> Colin
>>>> 
>>>> 
>>>> On 13 May 2013, at 09:12, Julien Jourde <julien.jourde at pasteur.fr> wrote:
>>>> 
>>>>> Hi ome list,
>>>>> 
>>>>> I'm running a 4-4-7 OMERO server and tables fail to startup. It's exactly the same problem as the one in the ticket 3690, which should be fixed :-/
>>>>> Here are my logs :
>>>>> 
>>>>> 
>>>>> 2013-05-13 09:49:20,658 INFO  [                       omero.util.Server] (MainThread) Waiting 10000 ms on startup
>>>>> 2013-05-13 09:49:30,658 INFO  [                       omero.util.Server] (MainThread) Starting
>>>>> 2013-05-13 09:49:30,659 INFO  [                       omero.util.Server] (MainThread) Loaded dependency numpy (1.6.2)
>>>>> 2013-05-13 09:49:30,659 INFO  [                       omero.util.Server] (MainThread) Loaded dependency tables (2.4.0, hdf=1.8.5-patch1)
>>>>> 2013-05-13 09:49:30,663 INFO  [                    omero.util.Resources] (Thread-2  ) Starting
>>>>> 2013-05-13 09:49:30,791 INFO  [                             omero.utils] (MainThread) Failed to get session on attempt 1
>>>>> 2013-05-13 09:49:41,099 ERROR [                       omero.util.Server] (MainThread) Failed initialization
>>>>> Traceback (most recent call last):
>>>>>  File "/home/omero/apps/omero/OMERO.server-4.4.7-ice34-b246/lib/python/omero/util/__init__.py", line 437, in run
>>>>>    self.impl = self.impl_class(ctx)
>>>>>  File "/home/omero/apps/omero/OMERO.server-4.4.7-ice34-b246/lib/python/omero/tables.py", line 795, in __init__
>>>>>    self._get_repo()
>>>>>  File "/home/omero/apps/omero/OMERO.server-4.4.7-ice34-b246/lib/python/omero/tables.py", line 847, in _get_repo
>>>>>    self.repo_mgr = self._internal_repo_cast(self.repo_mgr)
>>>>>  File "/home/omero/apps/omero/OMERO.server-4.4.7-ice34-b246/lib/python/omero_Repositories_ice.py", line 907, in checkedCast
>>>>>    return _M_omero.grid.InternalRepositoryPrx.ice_checkedCast(proxy, '::omero::grid::InternalRepository', facetOrCtx, _ctx)
>>>>> NotRegisteredException: exception ::Ice::NotRegisteredException
>>>>> {
>>>>>    kindOfObject = object
>>>>>    id = InternalRepository-f5718416-9db9-486b-ba49-4c55d0562e6b
>>>>> }
>>>>> 2013-05-13 09:49:41,100 INFO  [                       omero.util.Server] (MainThread) Cleanup
>>>>> 2013-05-13 09:49:41,101 INFO  [                       omero.util.Server] (MainThread) Stopped
>>>>> 2013-05-13 09:49:41,101 INFO  [                       omero.util.Server] (MainThread) ********************************************************************************
>>>>> 2013-05-13 09:49:41,101 INFO  [                    omero.tables.TablesI] (MainThread) Cleaning up
>>>>> 2013-05-13 09:49:41,101 INFO  [                    omero.tables.TablesI] (MainThread) Done
>>>>> 2013-05-13 09:49:41,113 WARNI [                                  stderr] (MainThread) 2013-05-13 09:49:41.113020 Tables-0: error: Traceback (most recent call last):
>>>>>  File "/usr/lib64/python2.6/site-packages/Ice/Ice.py", line 984, in main
>>>>>    status = self.doMain(args, initData)
>>>>>  File "/usr/lib64/python2.6/site-packages/Ice/Ice.py", line 1031, in doMain
>>>>>    return self.run(args)
>>>>>  File "/home/omero/apps/omero/OMERO.server-4.4.7-ice34-b246/lib/python/omero/util/__init__.py", line 441, in run
>>>>>    sys.exit(100)
>>>>> SystemExit: 100
>>>>> 2013-05-13 09:49:41,123 INFO  [                    omero.util.Resources] (Thread-2  ) Halted
>>>>> 
>>>>> 
>>>>> Any idea / advise / help is very welcome, thanks :)
> 
> 
> -- 
> Julien JOURDE
> Plateforme d'Imagerie Dynamique(PFID)
> Imagopole
> Institut Pasteur
> 25-28 rue du Docteur Roux
> 75724 Paris cedex15
> FRANCE
> 
> www.imagopole.org/
> 
> tel:33 (0)1 45 68 82 18
> fax:33(0)1 45 68 89 49
> 
>

More information about the ome-devel mailing list