[ome-users] excessive time to upgrade database / cleanse

[Josh Moore]

On Fri, Apr 29, 2016 at 3:30 PM, Olivier Miquel
<olivier.miquel at mri.cnrs.fr> wrote:
> Dear Mark,
>
> I had tried to cleanse the binary repository, with no success. The script
> died after several minutes of silence.

Hi Olivier,

this actually looks like something went side-ways on the server-side.
Could you send us your the Blitz log from var/log for when this
happened?

Cheers,
~Josh.


> Is there an alternate way to achieve that?
>
> Log :
> omero at kansas:~$ /home/omero/apps/OMERO/OMERO.server/bin/omero admin cleanse
> /home/omero/apps/OMERO/OMERO.data
> Previous session expired for root on localhost:4064
> Server: [localhost:4064]
> Username: [root]
> Password:
> Created session 50852b9f-4f40-4dc2-ad14-b900a8527386 (root at localhost:4064).
> Idle timeout: 10 min. Current group: system
> Traceback (most recent call last):
>   File "/home/omero/apps/OMERO/OMERO.server/bin/omero", line 125, in
> <module>
>     rv = omero.cli.argv()
>   File "/home/omero/apps/OMERO/OMERO.server/lib/python/omero/cli.py", line
> 1438, in argv
>     cli.invoke(args[1:])
>   File "/home/omero/apps/OMERO/OMERO.server/lib/python/omero/cli.py", line
> 952, in invoke
>     stop = self.onecmd(line, previous_args)
>   File "/home/omero/apps/OMERO/OMERO.server/lib/python/omero/cli.py", line
> 1029, in onecmd
>     self.execute(line, previous_args)
>   File "/home/omero/apps/OMERO/OMERO.server/lib/python/omero/cli.py", line
> 1111, in execute
>     args.func(args)
>   File
> "/home/omero/apps/OMERO/OMERO.server/lib/python/omero/plugins/admin.py",
> line 1776, in cleanse
>     dry_run=args.dry_run)
>   File
> "/home/omero/apps/OMERO/OMERO.server/lib/python/omero/util/cleanse.py", line
> 242, in cleanse
>     delete_empty_dirs(proxy, root, client, dry_run)
>   File
> "/home/omero/apps/OMERO/OMERO.server/lib/python/omero/util/cleanse.py", line
> 250, in delete_empty_dirs
>     is_empty_dir(repo, '/', False, to_delete)
>   File
> "/home/omero/apps/OMERO/OMERO.server/lib/python/omero/util/cleanse.py", line
> 282, in is_empty_dir
>     is_empty_dir(repo, subdirectory, may_delete_subdir, empty_subdirs):
>   File
> "/home/omero/apps/OMERO/OMERO.server/lib/python/omero/util/cleanse.py", line
> 282, in is_empty_dir
>     is_empty_dir(repo, subdirectory, may_delete_subdir, empty_subdirs):
>   File
> "/home/omero/apps/OMERO/OMERO.server/lib/python/omero/util/cleanse.py", line
> 282, in is_empty_dir
>     is_empty_dir(repo, subdirectory, may_delete_subdir, empty_subdirs):
>   File
> "/home/omero/apps/OMERO/OMERO.server/lib/python/omero/util/cleanse.py", line
> 277, in is_empty_dir
>     for entry in repo.listFiles(directory):
>   File
> "/home/omero/apps/OMERO/OMERO.server/lib/python/omero_Repositories_ice.py",
> line 329, in listFiles
>     return _M_omero.grid.Repository._op_listFiles.invoke(self, ((path, ),
> _ctx))
> Ice.UnknownException: exception ::Ice::UnknownException
> {
>     unknown = ome.conditions.InternalException:  Wrapped Exception:
> (org.springframework.transaction.TransactionSystemException):
> Could not roll back JDBC transaction; nested exception is
> org.postgresql.util.PSQLException: This connection has been closed.
>         at
> org.springframework.jdbc.datasource.DataSourceTransactionManager.doRollback(DataSourceTransactionManager.java:286)
>         at
> org.springframework.transaction.support.AbstractPlatformTransactionManager.processRollback(AbstractPlatformTransactionManager.java:845)
>         at
> org.springframework.transaction.support.AbstractPlatformTransactionManager.rollback(AbstractPlatformTransactionManager.java:822)
>         at
> org.springframework.transaction.interceptor.TransactionAspectSupport.completeTransactionAfterThrowing(TransactionAspectSupport.java:430)
>         at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:112)
>         at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
>         at
> ome.tools.hibernate.ProxyCleanupFilter$Interceptor.invoke(ProxyCleanupFilter.java:249)
>         at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
>         at ome.services.util.ServiceHandler.invoke(ServiceHandler.java:121)
>         at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
>         at
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
>         at com.sun.proxy.$Proxy70.doWork(Unknown Source)
>         at ome.services.util.Executor$Impl.execute(Executor.java:443)
>         at
> ome.services.blitz.repo.RepositoryDaoImpl.getOriginalFiles(RepositoryDaoImpl.java:439)
>         at
> ome.services.blitz.repo.PublicRepositoryI.listFiles(PublicRepositoryI.java:263)
>         at
> omero.grid._ManagedRepositoryTie.listFiles(_ManagedRepositoryTie.java:136)
>         at omero.grid._RepositoryDisp.___listFiles(_RepositoryDisp.java:324)
>         at
> omero.grid._ManagedRepositoryDisp.__dispatch(_ManagedRepositoryDisp.java:585)
>         at IceInternal.Incoming.invoke(Incoming.java:222)
>         at Ice.ConnectionI.invokeAll(ConnectionI.java:2482)
>         at Ice.ConnectionI.dispatch(ConnectionI.java:1258)
>         at Ice.ConnectionI.message(ConnectionI.java:1213)
>         at IceInternal.ThreadPool.run(ThreadPool.java:321)
>         at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
>         at
> IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:693)
>         at java.lang.Thread.run(Thread.java:745)
>
> }
>
>
> On 08/03/2016 10:19, Mark Carroll wrote:
>>
>> Dear Olivier,
>>
>>> Is there somewhere a script to check and clean the database?
>>> Our omero.server  is in production for about five years now and perhaps
>>> we now need to clean some unused stuff in the database?
>>
>>
>> What I'd been mentioning to Carnë in this thread about periodic VACUUMs
>> and suchlike probably suffices for PostgreSQL; their documentation is
>> good on this point. If we notice that our server has wrongly been
>> leaving some kind of debris in the database then we usually include
>> cleanup in upgrade scripts so it gets done for our users anyway in time;
>> for instance, the upgrade that we've been discussing included some
>> removal of uselessly duplicate rows from a large table.
>>
>> However, separately from the database, there is also the binary
>> repository: it is probably worth glancing at the first part of
>>
>> https://www.openmicroscopy.org/site/support/omero/sysadmins/server-backup-and-restore.html
>> which talks about cleaning files up before doing a backup. If you
>> haven't run a "bin/omero admin cleanse" for a while then that might be
>> something that's worth adding to your infrequent maintenance checklist.
>>
>> -- Mark
>>
>> The University of Dundee is a registered Scottish Charity, No: SC015096
>> _______________________________________________
>> ome-users mailing list
>> ome-users at lists.openmicroscopy.org.uk
>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users