[ome-devel] Problem when importing >1000 images
Andrii Iudin
andrii at ebi.ac.uk
Fri Apr 22 11:41:33 BST 2016
Dear OMERO developers,
We are experiencing an issue when importing a large number of images in
a single consequent go. This usually happens after importing more than a
thousand images. Please see below excerpts from the logs. Increasing a
time period between each import seemed to helped a bit, however this
issue ultimately happened anyway. To get OMERO server working after this
happens, it is necessary to stop it, remove .lock files and start the
server again.
It would be much appreciated if you could point out to a possible way to
solve this issue.
Thank you and with best regards,
Andrii
Import log:
Traceback (most recent call last):
File "/nfs/msd/em/crontabs/prod/empiar3d/importEmpiar3d.py", line 83,
in processEntries
importImageIntoOMERO(e3d, entryName, inputDir, errorDir, extension,
deployment)
File "/nfs/msd/em/crontabs/prod/empiar3d/importEmpiar3d.py", line 59,
in importImageIntoOMERO
e3d.run_command('omero import -s {0} -u {1} -w {2} -d {3} {4} {5}
{6} {7} {8} {9}'.format(e3d.omeroHost, e3d.omeroUser, e3d.omeroPassword,
e3d.omeroDataset, topImage, frontImage, sideImage, topThumbImage, frontThum
bImage, sideThumbImage), f_stdLog)
File "/nfs/msd/em/crontabs/prod/empiar3d/empiar3d.py", line 143, in
run_command
raise Exception("Failure executing: " + command + " :::out::: " +
out + " :::err::: " + err)
Exception: Failure executing: omero import -s ves-ebi-8c -u root -w
ArSiXbaq -d 1
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries//EMD-5259/emd_5259-top.map
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries//EMD-5259/emd_5259-front.map
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries//EMD-5259/emd_5259-side.map
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries//EMD-5259/emd_5259-top-thumb.map
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries//EMD-5259/emd_5259-front-thumb.map
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries//EMD-5259/emd_5259-side-thumb.map
:::out::: 62561
62562
62563
:::err::: Using session b91b2492-69b5-4199-b98e-56b24055ea4d
(root at ves-ebi-8c:4064). Idle timeout: 10 min. Current group: system
2016-04-21 09:25:05,513 290 [ main] INFO
ome.formats.importer.ImportConfig - OMERO Version: 5.1.4-ice35-b55
2016-04-21 09:25:05,531 308 [ main] INFO
ome.formats.importer.ImportConfig - Bioformats version: 5.1.4 revision:
05840624ab3d1d1dca14d1ccfebabcb61c42ec27 date: 3 September 2015
2016-04-21 09:25:05,541 318 [ main] INFO
formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats:
ERROR OMERO.importer: INFO
2016-04-21 09:25:05,976 753 [ main] INFO
ome.formats.importer.ImportCandidates - Depth: 4 Metadata Level: MINIMUM
2016-04-21 09:25:07,102 1879 [ main] INFO
ome.formats.importer.ImportCandidates - 6 file(s) parsed into 6 group(s)
with 6 call(s) to setId in 868ms. (1126ms total) [0 unknowns]
2016-04-21 09:25:07,332 2109 [ main] INFO
ome.formats.OMEROMetadataStoreClient - Attempting initial SSL connection
to ves-ebi-8c:4064
2016-04-21 09:25:07,993 2770 [ main] INFO
ome.formats.OMEROMetadataStoreClient - Insecure connection requested,
falling back
2016-04-21 09:25:08,323 3100 [ main] INFO
ome.formats.OMEROMetadataStoreClient - Server: 5.1.4
2016-04-21 09:25:08,323 3100 [ main] INFO
ome.formats.OMEROMetadataStoreClient - Client: 5.1.4-ice35-b55
2016-04-21 09:25:08,323 3100 [ main] INFO
ome.formats.OMEROMetadataStoreClient - Java Version: 1.6.0_33
2016-04-21 09:25:08,324 3101 [ main] INFO
ome.formats.OMEROMetadataStoreClient - OS Name: Linux
2016-04-21 09:25:08,324 3101 [ main] INFO
ome.formats.OMEROMetadataStoreClient - OS Arch: amd64
2016-04-21 09:25:08,324 3101 [ main] INFO
ome.formats.OMEROMetadataStoreClient - OS Version: 2.6.32-504.1.3.el6.x86_64
2016-04-21 09:25:08,497 3274 [ main] INFO
ome.formats.OMEROMetadataStoreClient - Call context: {omero.group:0}
2016-04-21 09:25:08,526 3303 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - FILESET_UPLOAD_PREPARATION
2016-04-21 09:25:08,876 3653 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - FILESET_UPLOAD_START
2016-04-21 09:25:08,888 3665 [ main] INFO
ts.importer.transfers.UploadFileTransfer - Transferring
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries/EMD-5259/emd_5259-top.map...
2016-04-21 09:25:08,926 3703 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED:
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries/EMD-5259/emd_5259-top.map
2016-04-21 09:25:13,070 7847 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE:
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries/EMD-5259/emd_5259-top.map
2016-04-21 09:25:13,725 8502 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - FILESET_UPLOAD_END
2016-04-21 09:25:13,817 8594 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - IMPORT_STARTED Logfile: 189317
2016-04-21 09:25:13,873 8650 [l.Client-0] INFO
ormats.importer.cli.LoggingImportMonitor - METADATA_IMPORTED Step: 1 of
5 Logfile: 189317
2016-04-21 09:25:15,540 10317 [l.Client-1] INFO
ormats.importer.cli.LoggingImportMonitor - PIXELDATA_PROCESSED Step: 2
of 5 Logfile: 189317
2016-04-21 09:25:15,727 10504 [l.Client-0] INFO
ormats.importer.cli.LoggingImportMonitor - THUMBNAILS_GENERATED Step: 3
of 5 Logfile: 189317
2016-04-21 09:25:15,752 10529 [l.Client-1] INFO
ormats.importer.cli.LoggingImportMonitor - METADATA_PROCESSED Step: 4 of
5 Logfile: 189317
2016-04-21 09:25:15,775 10552 [l.Client-1] INFO
ormats.importer.cli.LoggingImportMonitor - OBJECTS_RETURNED Step: 5 of
5 Logfile: 189317
2016-04-21 09:25:15,909 10686 [l.Client-1] INFO
ormats.importer.cli.LoggingImportMonitor - IMPORT_DONE Imported file:
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries/EMD-5259/emd_5259-top.map
Imported pixels:
Other imported objects:
Fileset:62561
Image:62561
2016-04-21 09:25:15,909 10686 [l.Client-1] INFO
ome.formats.importer.cli.ErrorHandler - Number of errors: 0
2016-04-21 09:25:15,952 10729 [ main] INFO
ome.formats.OMEROMetadataStoreClient - Call context: {omero.group:0}
2016-04-21 09:25:15,976 10753 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - FILESET_UPLOAD_PREPARATION
2016-04-21 09:25:16,273 11050 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - FILESET_UPLOAD_START
2016-04-21 09:25:16,277 11054 [ main] INFO
ts.importer.transfers.UploadFileTransfer - Transferring
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries/EMD-5259/emd_5259-front.map...
2016-04-21 09:25:16,314 11091 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED:
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries/EMD-5259/emd_5259-front.map
2016-04-21 09:25:19,581 14358 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE:
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries/EMD-5259/emd_5259-front.map
2016-04-21 09:25:20,243 15020 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - FILESET_UPLOAD_END
2016-04-21 09:25:20,270 15047 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - IMPORT_STARTED Logfile: 189320
2016-04-21 09:25:20,415 15192 [l.Client-2] INFO
ormats.importer.cli.LoggingImportMonitor - METADATA_IMPORTED Step: 1 of
5 Logfile: 189320
2016-04-21 09:25:22,152 16929 [l.Client-2] INFO
ormats.importer.cli.LoggingImportMonitor - PIXELDATA_PROCESSED Step: 2
of 5 Logfile: 189320
2016-04-21 09:25:22,313 17090 [l.Client-2] INFO
ormats.importer.cli.LoggingImportMonitor - THUMBNAILS_GENERATED Step: 3
of 5 Logfile: 189320
2016-04-21 09:25:22,336 17113 [l.Client-0] INFO
ormats.importer.cli.LoggingImportMonitor - METADATA_PROCESSED Step: 4 of
5 Logfile: 189320
2016-04-21 09:25:22,356 17133 [l.Client-0] INFO
ormats.importer.cli.LoggingImportMonitor - OBJECTS_RETURNED Step: 5 of
5 Logfile: 189320
2016-04-21 09:25:22,443 17220 [l.Client-2] INFO
ormats.importer.cli.LoggingImportMonitor - IMPORT_DONE Imported file:
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries/EMD-5259/emd_5259-front.map
Imported pixels:
Other imported objects:
Fileset:62562
Image:62562
2016-04-21 09:25:22,443 17220 [l.Client-2] INFO
ome.formats.importer.cli.ErrorHandler - Number of errors: 0
2016-04-21 09:25:22,479 17256 [ main] INFO
ome.formats.OMEROMetadataStoreClient - Call context: {omero.group:0}
2016-04-21 09:25:22,504 17281 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - FILESET_UPLOAD_PREPARATION
2016-04-21 09:25:22,784 17561 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - FILESET_UPLOAD_START
2016-04-21 09:25:22,788 17565 [ main] INFO
ts.importer.transfers.UploadFileTransfer - Transferring
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries/EMD-5259/emd_5259-side.map...
2016-04-21 09:25:22,815 17592 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED:
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries/EMD-5259/emd_5259-side.map
2016-04-21 09:25:26,155 20932 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE:
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries/EMD-5259/emd_5259-side.map
2016-04-21 09:25:26,797 21574 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - FILESET_UPLOAD_END
2016-04-21 09:25:26,824 21601 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - IMPORT_STARTED Logfile: 189323
2016-04-21 09:25:26,958 21735 [l.Client-0] INFO
ormats.importer.cli.LoggingImportMonitor - METADATA_IMPORTED Step: 1 of
5 Logfile: 189323
2016-04-21 09:25:29,257 24034 [l.Client-2] INFO
ormats.importer.cli.LoggingImportMonitor - PIXELDATA_PROCESSED Step: 2
of 5 Logfile: 189323
2016-04-21 09:25:29,419 24196 [l.Client-1] INFO
ormats.importer.cli.LoggingImportMonitor - THUMBNAILS_GENERATED Step: 3
of 5 Logfile: 189323
2016-04-21 09:25:29,443 24220 [l.Client-1] INFO
ormats.importer.cli.LoggingImportMonitor - METADATA_PROCESSED Step: 4 of
5 Logfile: 189323
2016-04-21 09:25:29,462 24239 [l.Client-2] INFO
ormats.importer.cli.LoggingImportMonitor - OBJECTS_RETURNED Step: 5 of
5 Logfile: 189323
2016-04-21 09:25:29,539 24316 [l.Client-0] INFO
ormats.importer.cli.LoggingImportMonitor - IMPORT_DONE Imported file:
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries/EMD-5259/emd_5259-side.map
Imported pixels:
Other imported objects:
Fileset:62563
Image:62563
2016-04-21 09:25:29,540 24317 [l.Client-0] INFO
ome.formats.importer.cli.ErrorHandler - Number of errors: 0
2016-04-21 09:25:29,576 24353 [ main] INFO
ome.formats.OMEROMetadataStoreClient - Call context: {omero.group:0}
2016-04-21 09:25:29,598 24375 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - FILESET_UPLOAD_PREPARATION
2016-04-21 09:25:29,873 24650 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - FILESET_UPLOAD_START
2016-04-21 09:25:29,877 24654 [ main] INFO
ts.importer.transfers.UploadFileTransfer - Transferring
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries/EMD-5259/emd_5259-top-thumb.map...
2016-04-21 09:25:29,905 24682 [ main] INFO
ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED:
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries/EMD-5259/emd_5259-top-thumb.map
2016-04-21 11:26:08,467 7263244 [1-thread-1] ERROR
me.formats.importer.util.ClientKeepAlive - Exception while executing
ping(), logging Connector out:
java.lang.RuntimeException: Ice.ConnectionLostException
error = 0
at
ome.formats.OMEROMetadataStoreClient.ping(OMEROMetadataStoreClient.java:763)
~[blitz.jar:na]
at
ome.formats.importer.util.ClientKeepAlive.run(ClientKeepAlive.java:69)
~[blitz.jar:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.6.0_33]
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
~[na:1.6.0_33]
at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
~[na:1.6.0_33]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
~[na:1.6.0_33]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
~[na:1.6.0_33]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
~[na:1.6.0_33]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
~[na:1.6.0_33]
at java.lang.Thread.run(Thread.java:701) ~[na:1.6.0_33]
Caused by: Ice.ConnectionLostException: java.io.IOException: Connection
reset by peer
at IceInternal.Outgoing.invoke(Outgoing.java:158) ~[ice.jar:na]
at
omero.api._RawFileStoreDelM.write(_RawFileStoreDelM.java:379)
~[blitz.jar:na]
at
omero.api.RawFileStorePrxHelper.write(RawFileStorePrxHelper.java:1409)
~[blitz.jar:na]
at
omero.api.RawFileStorePrxHelper.write(RawFileStorePrxHelper.java:1381)
~[blitz.jar:na]
at
ome.formats.importer.transfers.UploadFileTransfer.transfer(UploadFileTransfer.java:79)
~[blitz.jar:na]
at
ome.formats.importer.ImportLibrary.uploadFile(ImportLibrary.java:430)
~[blitz.jar:na]
at
ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:503)
~[blitz.jar:na]
at
ome.formats.importer.ImportLibrary.importCandidates(ImportLibrary.java:287)
~[blitz.jar:na]
at
ome.formats.importer.cli.CommandLineImporter.start(CommandLineImporter.java:245)
~[blitz.jar:na]
at
ome.formats.importer.cli.CommandLineImporter.main(CommandLineImporter.java:858)
~[blitz.jar:na]
Caused by: java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcher.read0(Native Method) ~[na:1.6.0_33]
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
~[na:1.6.0_33]
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:251)
~[na:1.6.0_33]
at sun.nio.ch.IOUtil.read(IOUtil.java:224) ~[na:1.6.0_33]
at
sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:254) ~[na:1.6.0_33]
at IceInternal.TcpTransceiver.read(TcpTransceiver.java:245)
~[ice.jar:na]
at Ice.ConnectionI.message(ConnectionI.java:996) ~[ice.jar:na]
at IceInternal.ThreadPool.run(ThreadPool.java:321) ~[ice.jar:na]
at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
~[ice.jar:na]
at
IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:693)
~[ice.jar:na]
at java.lang.Thread.run(Thread.java:701) ~[na:1.6.0_33]
2016-04-21 11:26:08,468 7263245 [ main] ERROR
ome.formats.importer.cli.ErrorHandler - FILE_EXCEPTION:
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries/EMD-5259/emd_5259-top-thumb.map
Ice.ConnectionLostException: java.io.IOException: Connection reset by peer
at IceInternal.Outgoing.invoke(Outgoing.java:158) ~[ice.jar:na]
at
omero.api._RawFileStoreDelM.write(_RawFileStoreDelM.java:379)
~[blitz.jar:na]
at
omero.api.RawFileStorePrxHelper.write(RawFileStorePrxHelper.java:1409)
~[blitz.jar:na]
at
omero.api.RawFileStorePrxHelper.write(RawFileStorePrxHelper.java:1381)
~[blitz.jar:na]
at
ome.formats.importer.transfers.UploadFileTransfer.transfer(UploadFileTransfer.java:79)
~[blitz.jar:na]
at
ome.formats.importer.ImportLibrary.uploadFile(ImportLibrary.java:430)
[blitz.jar:na]
at
ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:503)
[blitz.jar:na]
at
ome.formats.importer.ImportLibrary.importCandidates(ImportLibrary.java:287)
[blitz.jar:na]
at
ome.formats.importer.cli.CommandLineImporter.start(CommandLineImporter.java:245)
~[blitz.jar:na]
at
ome.formats.importer.cli.CommandLineImporter.main(CommandLineImporter.java:858)
~[blitz.jar:na]
Caused by: java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcher.read0(Native Method) ~[na:1.6.0_33]
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
~[na:1.6.0_33]
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:251)
~[na:1.6.0_33]
at sun.nio.ch.IOUtil.read(IOUtil.java:224) ~[na:1.6.0_33]
at
sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:254) ~[na:1.6.0_33]
at IceInternal.TcpTransceiver.read(TcpTransceiver.java:245)
~[ice.jar:na]
at Ice.ConnectionI.message(ConnectionI.java:996) ~[ice.jar:na]
at IceInternal.ThreadPool.run(ThreadPool.java:321) ~[ice.jar:na]
at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
~[ice.jar:na]
at
IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:693)
~[ice.jar:na]
at java.lang.Thread.run(Thread.java:701) ~[na:1.6.0_33]
2016-04-21 11:26:08,493 7263270 [1-thread-1] WARN
ome.formats.OMEROMetadataStoreClient - Exception closing
b91b2492-69b5-4199-b98e-56b24055ea4d/b9f60438-fec1-40c7-8916-f2b6b2804dbdomero.api.RawFileStore
-t -e 1.0:tcp -h 10.3.2.140 -p 41375
Ice.ConnectionLostException: null
at
IceInternal.ConnectRequestHandler.getConnection(ConnectRequestHandler.java:244)
~[ice.jar:na]
at
IceInternal.ConnectRequestHandler.sendRequest(ConnectRequestHandler.java:141)
~[ice.jar:na]
at IceInternal.Outgoing.invoke(Outgoing.java:77) ~[ice.jar:na]
at
omero.api._RawFileStoreDelM.close(_RawFileStoreDelM.java:459)
~[blitz.jar:na]
at
omero.api.RawFileStorePrxHelper.close(RawFileStorePrxHelper.java:1877)
~[blitz.jar:na]
at
omero.api.RawFileStorePrxHelper.close(RawFileStorePrxHelper.java:1839)
~[blitz.jar:na]
at
ome.formats.OMEROMetadataStoreClient.closeQuietly(OMEROMetadataStoreClient.java:1054)
[blitz.jar:na]
at
ome.formats.OMEROMetadataStoreClient.closeServices(OMEROMetadataStoreClient.java:1075)
[blitz.jar:na]
at
ome.formats.OMEROMetadataStoreClient.logout(OMEROMetadataStoreClient.java:1101)
[blitz.jar:na]
at
ome.formats.importer.util.ClientKeepAlive.run(ClientKeepAlive.java:78)
~[blitz.jar:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.6.0_33]
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
~[na:1.6.0_33]
at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
~[na:1.6.0_33]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
~[na:1.6.0_33]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
~[na:1.6.0_33]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
~[na:1.6.0_33]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
~[na:1.6.0_33]
at java.lang.Thread.run(Thread.java:701) ~[na:1.6.0_33]
2016-04-21 11:26:08,494 7263271 [ main] ERROR
ome.formats.importer.ImportLibrary - Error on import
Ice.ConnectionLostException: java.io.IOException: Connection reset by peer
at Ice.ConnectionI.sendAsyncRequest(ConnectionI.java:391)
~[ice.jar:na]
at
IceInternal.ConnectionRequestHandler.sendAsyncRequest(ConnectionRequestHandler.java:53)
~[ice.jar:na]
at IceInternal.OutgoingAsync.__send(OutgoingAsync.java:400)
~[ice.jar:na]
at
Ice.RouterPrxHelper.begin_addProxies(RouterPrxHelper.java:184) ~[ice.jar:na]
at
Ice.RouterPrxHelper.begin_addProxies(RouterPrxHelper.java:158) ~[ice.jar:na]
at IceInternal.RouterInfo.addProxy(RouterInfo.java:179)
~[ice.jar:na]
at
IceInternal.ConnectRequestHandler.setConnection(ConnectRequestHandler.java:274)
~[ice.jar:na]
at
IceInternal.RoutableReference$3.setConnection(RoutableReference.java:922) ~[ice.jar:na]
at
IceInternal.OutgoingConnectionFactory$ConnectCallback.setConnection(OutgoingConnectionFactory.java:1104)
~[ice.jar:na]
at
IceInternal.OutgoingConnectionFactory.finishGetConnection(OutgoingConnectionFactory.java:804)
~[ice.jar:na]
at
IceInternal.OutgoingConnectionFactory.access$000(OutgoingConnectionFactory.java:12)
~[ice.jar:na]
at
IceInternal.OutgoingConnectionFactory$ConnectCallback.connectionStartCompleted(OutgoingConnectionFactory.java:1016)
~[ice.jar:na]
at Ice.ConnectionI.dispatch(ConnectionI.java:1226) ~[ice.jar:na]
at Ice.ConnectionI.message(ConnectionI.java:1213) ~[ice.jar:na]
at IceInternal.ThreadPool.run(ThreadPool.java:321) ~[ice.jar:na]
at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
~[ice.jar:na]
at
IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:693)
~[ice.jar:na]
at java.lang.Thread.run(Thread.java:701) ~[na:1.6.0_33]
Caused by: java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcher.read0(Native Method) ~[na:1.6.0_33]
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
~[na:1.6.0_33]
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:251)
~[na:1.6.0_33]
at sun.nio.ch.IOUtil.read(IOUtil.java:224) ~[na:1.6.0_33]
at
sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:254) ~[na:1.6.0_33]
at IceInternal.TcpTransceiver.read(TcpTransceiver.java:245)
~[ice.jar:na]
at Ice.ConnectionI.message(ConnectionI.java:996) ~[ice.jar:na]
... 4 common frames omitted
2016-04-21 11:26:08,494 7263271 [ main] INFO
ome.formats.importer.ImportLibrary - Exiting on error
2016-04-21 11:26:08,499 7263276 [1-thread-1] WARN
ome.formats.OMEROMetadataStoreClient - Exception closing
b91b2492-69b5-4199-b98e-56b24055ea4d/da6104c2-03bd-469f-b4b7-135a471f9963omero.api.RawPixelsStore
-t -e 1.0:tcp -h 10.3.2.140 -p 41375
Ice.ConnectionLostException: null
at
IceInternal.ConnectRequestHandler.getConnection(ConnectRequestHandler.java:244)
~[ice.jar:na]
at
IceInternal.ConnectRequestHandler.sendRequest(ConnectRequestHandler.java:141)
~[ice.jar:na]
at IceInternal.Outgoing.invoke(Outgoing.java:77) ~[ice.jar:na]
at
omero.api._RawPixelsStoreDelM.close(_RawPixelsStoreDelM.java:2003)
~[blitz.jar:na]
at
omero.api.RawPixelsStorePrxHelper.close(RawPixelsStorePrxHelper.java:9946)
~[blitz.jar:na]
at
omero.api.RawPixelsStorePrxHelper.close(RawPixelsStorePrxHelper.java:9908)
~[blitz.jar:na]
at
ome.formats.OMEROMetadataStoreClient.closeQuietly(OMEROMetadataStoreClient.java:1054)
[blitz.jar:na]
at
ome.formats.OMEROMetadataStoreClient.closeServices(OMEROMetadataStoreClient.java:1078)
[blitz.jar:na]
at
ome.formats.OMEROMetadataStoreClient.logout(OMEROMetadataStoreClient.java:1101)
[blitz.jar:na]
at
ome.formats.importer.util.ClientKeepAlive.run(ClientKeepAlive.java:78)
~[blitz.jar:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.6.0_33]
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
~[na:1.6.0_33]
at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
~[na:1.6.0_33]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
~[na:1.6.0_33]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
~[na:1.6.0_33]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
~[na:1.6.0_33]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
~[na:1.6.0_33]
at java.lang.Thread.run(Thread.java:701) ~[na:1.6.0_33]
==> Summary
3 files uploaded, 3 filesets created, 3 images imported, 1 error in
2:01:00.052
2016-04-21 11:26:08,502 7263279 [1-thread-1] WARN
ome.formats.OMEROMetadataStoreClient - Exception closing
b91b2492-69b5-4199-b98e-56b24055ea4d/71bb1da6-c4bb-4725-a8ed-d3364852a0fcomero.api.ThumbnailStore
-t -e 1.0:tcp -h 10.3.2.140 -p 41375
Ice.ConnectionLostException: null
at
IceInternal.ConnectRequestHandler.getConnection(ConnectRequestHandler.java:244)
~[ice.jar:na]
at
IceInternal.ConnectRequestHandler.sendRequest(ConnectRequestHandler.java:141)
~[ice.jar:na]
at IceInternal.Outgoing.invoke(Outgoing.java:77) ~[ice.jar:na]
at
omero.api._ThumbnailStoreDelM.close(_ThumbnailStoreDelM.java:74)
~[blitz.jar:na]
at
omero.api.ThumbnailStorePrxHelper.close(ThumbnailStorePrxHelper.java:356) ~[blitz.jar:na]
at
omero.api.ThumbnailStorePrxHelper.close(ThumbnailStorePrxHelper.java:318) ~[blitz.jar:na]
at
ome.formats.OMEROMetadataStoreClient.closeQuietly(OMEROMetadataStoreClient.java:1054)
[blitz.jar:na]
at
ome.formats.OMEROMetadataStoreClient.closeServices(OMEROMetadataStoreClient.java:1081)
[blitz.jar:na]
at
ome.formats.OMEROMetadataStoreClient.logout(OMEROMetadataStoreClient.java:1101)
[blitz.jar:na]
at
ome.formats.importer.util.ClientKeepAlive.run(ClientKeepAlive.java:78)
~[blitz.jar:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.6.0_33]
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
~[na:1.6.0_33]
at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
~[na:1.6.0_33]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
~[na:1.6.0_33]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
~[na:1.6.0_33]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
~[na:1.6.0_33]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
~[na:1.6.0_33]
at java.lang.Thread.run(Thread.java:701) ~[na:1.6.0_33]
2016-04-21 11:26:08,503 7263280 [ main] WARN
ome.formats.OMEROMetadataStoreClient - Exception closing
b91b2492-69b5-4199-b98e-56b24055ea4d/71bb1da6-c4bb-4725-a8ed-d3364852a0fcomero.api.ThumbnailStore
-t -e 1.0:tcp -h 10.3.2.140 -p 41375
Ice.ConnectionLostException: null
at
IceInternal.ConnectRequestHandler.getConnection(ConnectRequestHandler.java:244)
~[ice.jar:na]
at
IceInternal.ConnectRequestHandler.sendRequest(ConnectRequestHandler.java:141)
~[ice.jar:na]
at IceInternal.Outgoing.invoke(Outgoing.java:77) ~[ice.jar:na]
at
omero.api._ThumbnailStoreDelM.close(_ThumbnailStoreDelM.java:74)
~[blitz.jar:na]
at
omero.api.ThumbnailStorePrxHelper.close(ThumbnailStorePrxHelper.java:356) ~[blitz.jar:na]
at
omero.api.ThumbnailStorePrxHelper.close(ThumbnailStorePrxHelper.java:318) ~[blitz.jar:na]
at
ome.formats.OMEROMetadataStoreClient.closeQuietly(OMEROMetadataStoreClient.java:1054)
~[blitz.jar:na]
at
ome.formats.OMEROMetadataStoreClient.closeServices(OMEROMetadataStoreClient.java:1081)
~[blitz.jar:na]
at
ome.formats.OMEROMetadataStoreClient.logout(OMEROMetadataStoreClient.java:1101)
~[blitz.jar:na]
at
ome.formats.importer.util.ClientKeepAlive.run(ClientKeepAlive.java:78)
~[blitz.jar:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.6.0_33]
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
~[na:1.6.0_33]
at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
~[na:1.6.0_33]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
~[na:1.6.0_33]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
~[na:1.6.0_33]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
~[na:1.6.0_33]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
~[na:1.6.0_33]
at java.lang.Thread.run(Thread.java:701) ~[na:1.6.0_33]
2016-04-21 11:26:08,505 7263282 [1-thread-1] WARN
ome.formats.OMEROMetadataStoreClient - Exception closing
b91b2492-69b5-4199-b98e-56b24055ea4d/1cbe4095-56fd-4aca-83e1-8446606ec2ccomero.api.MetadataStore
-t -e 1.0:tcp -h 10.3.2.140 -p 41375
Ice.ConnectionLostException: java.io.IOException: Connection reset by peer
at IceInternal.Outgoing.invoke(Outgoing.java:158) ~[ice.jar:na]
at
omero.api._MetadataStoreDelM.close(_MetadataStoreDelM.java:412)
~[blitz.jar:na]
at
omero.api.MetadataStorePrxHelper.close(MetadataStorePrxHelper.java:1560)
~[blitz.jar:na]
at
omero.api.MetadataStorePrxHelper.close(MetadataStorePrxHelper.java:1522)
~[blitz.jar:na]
at
ome.formats.OMEROMetadataStoreClient.closeQuietly(OMEROMetadataStoreClient.java:1054)
[blitz.jar:na]
at
ome.formats.OMEROMetadataStoreClient.closeServices(OMEROMetadataStoreClient.java:1084)
[blitz.jar:na]
at
ome.formats.OMEROMetadataStoreClient.logout(OMEROMetadataStoreClient.java:1101)
[blitz.jar:na]
at
ome.formats.importer.util.ClientKeepAlive.run(ClientKeepAlive.java:78)
~[blitz.jar:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.6.0_33]
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
~[na:1.6.0_33]
at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
~[na:1.6.0_33]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
~[na:1.6.0_33]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
~[na:1.6.0_33]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
~[na:1.6.0_33]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
~[na:1.6.0_33]
at java.lang.Thread.run(Thread.java:701) ~[na:1.6.0_33]
Caused by: java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcher.read0(Native Method) ~[na:1.6.0_33]
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
~[na:1.6.0_33]
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:251)
~[na:1.6.0_33]
at sun.nio.ch.IOUtil.read(IOUtil.java:224) ~[na:1.6.0_33]
at
sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:254) ~[na:1.6.0_33]
at IceInternal.TcpTransceiver.read(TcpTransceiver.java:245)
~[ice.jar:na]
at Ice.ConnectionI.message(ConnectionI.java:996) ~[ice.jar:na]
at IceInternal.ThreadPool.run(ThreadPool.java:321) ~[ice.jar:na]
at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
~[ice.jar:na]
at
IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:693)
~[ice.jar:na]
... 1 common frames omitted
2016-04-21 11:26:08,505 7263282 [ main] WARN
ome.formats.OMEROMetadataStoreClient - Exception closing
b91b2492-69b5-4199-b98e-56b24055ea4d/1cbe4095-56fd-4aca-83e1-8446606ec2ccomero.api.MetadataStore
-t -e 1.0:tcp -h 10.3.2.140 -p 41375
Ice.ConnectionLostException: java.io.IOException: Connection reset by peer
at IceInternal.Outgoing.invoke(Outgoing.java:158) ~[ice.jar:na]
at
omero.api._MetadataStoreDelM.close(_MetadataStoreDelM.java:412)
~[blitz.jar:na]
at
omero.api.MetadataStorePrxHelper.close(MetadataStorePrxHelper.java:1560)
~[blitz.jar:na]
at
omero.api.MetadataStorePrxHelper.close(MetadataStorePrxHelper.java:1522)
~[blitz.jar:na]
at
ome.formats.OMEROMetadataStoreClient.closeQuietly(OMEROMetadataStoreClient.java:1054)
~[blitz.jar:na]
at
ome.formats.OMEROMetadataStoreClient.closeServices(OMEROMetadataStoreClient.java:1084)
~[blitz.jar:na]
at
ome.formats.OMEROMetadataStoreClient.logout(OMEROMetadataStoreClient.java:1101)
~[blitz.jar:na]
at
ome.formats.importer.util.ClientKeepAlive.run(ClientKeepAlive.java:78)
~[blitz.jar:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.6.0_33]
at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
~[na:1.6.0_33]
at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
~[na:1.6.0_33]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
~[na:1.6.0_33]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
~[na:1.6.0_33]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
~[na:1.6.0_33]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
~[na:1.6.0_33]
at java.lang.Thread.run(Thread.java:701) ~[na:1.6.0_33]
Caused by: java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcher.read0(Native Method) ~[na:1.6.0_33]
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
~[na:1.6.0_33]
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:251)
~[na:1.6.0_33]
at sun.nio.ch.IOUtil.read(IOUtil.java:224) ~[na:1.6.0_33]
at
sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:254) ~[na:1.6.0_33]
at IceInternal.TcpTransceiver.read(TcpTransceiver.java:245)
~[ice.jar:na]
at Ice.ConnectionI.message(ConnectionI.java:996) ~[ice.jar:na]
at IceInternal.ThreadPool.run(ThreadPool.java:321) ~[ice.jar:na]
at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
~[ice.jar:na]
at
IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:693)
~[ice.jar:na]
... 1 common frames omitted
-! 04/21/16 11:26:08.681 warning: Proxy keep alive failed.
Traceback (most recent call last):
File "/nfs/msd/em/crontabs/prod/empiar3d/importEmpiar3d.py", line 83,
in processEntries
importImageIntoOMERO(e3d, entryName, inputDir, errorDir, extension,
deployment)
File "/nfs/msd/em/crontabs/prod/empiar3d/importEmpiar3d.py", line 59,
in importImageIntoOMERO
e3d.run_command('omero import -s {0} -u {1} -w {2} -d {3} {4} {5}
{6} {7} {8} {9}'.format(e3d.omeroHost, e3d.omeroUser, e3d.omeroPassword,
e3d.omeroDataset, topImage, frontImage, sideImage, topThumbImage,
frontThumbImage, sideThumbImage), f_stdLog)
File "/nfs/msd/em/crontabs/prod/empiar3d/empiar3d.py", line 143, in
run_command
raise Exception("Failure executing: " + command + " :::out::: " +
out + " :::err::: " + err)
Exception: Failure executing: omero import -s ves-ebi-8c -u root -w
ArSiXbaq -d 1
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries//EMD-3256/emd_3256-top.map
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries//EMD-3256/emd_3256-front.map
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries//EMD-3256/emd_3256-side.map
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries//EMD-3256/emd_3256-top-thumb.map
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries//EMD-3256/emd_3256-front-thumb.map
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries//EMD-3256/emd_3256-side-thumb.map
:::out::: Previous session expired for root on ves-ebi-8c:4064
:::err::: InternalException: Failed to connect:
Ice.ConnectionLostException:
Connection reset by peer
Traceback (most recent call last):
File "/nfs/msd/em/crontabs/prod/empiar3d/importEmpiar3d.py", line 83,
in processEntries
importImageIntoOMERO(e3d, entryName, inputDir, errorDir, extension,
deployment)
File "/nfs/msd/em/crontabs/prod/empiar3d/importEmpiar3d.py", line 59,
in importImageIntoOMERO
e3d.run_command('omero import -s {0} -u {1} -w {2} -d {3} {4} {5}
{6} {7} {8} {9}'.format(e3d.omeroHost, e3d.omeroUser, e3d.omeroPassword,
e3d.omeroDataset, topImage, frontImage, sideImage, topThumbImage,
frontThumbImage, sideThumbImage), f_stdLog)
File "/nfs/msd/em/crontabs/prod/empiar3d/empiar3d.py", line 143, in
run_command
raise Exception("Failure executing: " + command + " :::out::: " +
out + " :::err::: " + err)
Exception: Failure executing: omero import -s ves-ebi-8c -u root -w
ArSiXbaq -d 1
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries//EMD-1097/emd_1097-top.map
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries//EMD-1097/emd_1097-front.map
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries//EMD-1097/emd_1097-side.map
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries//EMD-1097/emd_1097-top-thumb.map
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries//EMD-1097/emd_1097-front-thumb.map
/nfs/public/rw/pdbe/mol2cell/test-data/empiar3d-entries//EMD-1097/emd_1097-side-thumb.map
:::out::: Previously logged in to ves-ebi-8c:4064 as root
:::err::: InternalException: Failed to connect:
Ice.ConnectionLostException:
Connection reset by peer
master.err:
!! 21/04/16 09:25:34:419 error: Blitz-0-Ice.ThreadPool.Server-105:
java.lang.OutOfMemoryError: Java heap space
at IceInternal.BasicStream.readByteSeq(BasicStream.java:918)
at Ice.ByteSeqHelper.read(ByteSeqHelper.java:35)
at omero.api._RawFileStoreDisp.___write(_RawFileStoreDisp.java:300)
at
omero.api._RawFileStoreDisp.__dispatch(_RawFileStoreDisp.java:440)
at IceInternal.Incoming.invoke(Incoming.java:222)
at Ice.ConnectionI.invokeAll(ConnectionI.java:2482)
at Ice.ConnectionI.dispatch(ConnectionI.java:1258)
at Ice.ConnectionI.message(ConnectionI.java:1213)
at IceInternal.ThreadPool.run(ThreadPool.java:321)
at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
at
IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:693)
at java.lang.Thread.run(Thread.java:701)
Exception in thread "Blitz-0-Ice.ThreadPool.Server-112"
java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
at java.nio.ByteBuffer.allocate(ByteBuffer.java:329)
at IceInternal.Buffer.reserve(Buffer.java:166)
at IceInternal.Buffer.resize(Buffer.java:97)
at IceInternal.BasicStream.resize(BasicStream.java:181)
at Ice.ConnectionI.message(ConnectionI.java:1050)
at IceInternal.ThreadPool.run(ThreadPool.java:321)
at IceInternal.ThreadPool.access$300(ThreadPool.java:12)
at
IceInternal.ThreadPool$EventHandlerThread.run(ThreadPool.java:693)
at java.lang.Thread.run(Thread.java:701)
Indexer-0.log:
2016-04-21 09:24:50,201 INFO [ ome.services.fulltext.FullTextIndexer]
(2-thread-2) INDEXED 2 objects in batch#39140 [ 176 ms.] ~99%
done (2272558 of 2272563)
2016-04-21 09:25:14,929 INFO [ ome.services.fulltext.FullTextIndexer]
(2-thread-4) INDEXED 2 objects in batch#39152 [ 876 ms.] ~99%
done (2272594 of 2272599)
2016-04-21 09:25:22,207 INFO [ ome.services.fulltext.FullTextIndexer]
(2-thread-3) INDEXED 2 objects in batch#39156 [ 182 ms.] ~99%
done (2272630 of 2272635)
2016-04-21 09:25:28,204 INFO [ ome.services.fulltext.FullTextIndexer]
(2-thread-1) INDEXED 2 objects in batch#39159 [ 184 ms.] ~99%
done (2272666 of 2272671)
2016-04-21 09:39:30,276 INFO [ ome.system.metrics] (r-thread-1)
type=HISTOGRAM,
name=ome.services.fulltext.FullTextIndexer.percentCompleteFast,
count=7589, min=99, max=99, mean=99.0, stddev=0.0, m
edian=99.0, p75=99.0, p95=99.0, p98=99.0, p99=99.0, p999=99.0
2016-04-21 09:39:30,278 INFO [ ome.system.metrics] (r-thread-1)
type=HISTOGRAM,
name=ome.services.fulltext.FullTextIndexer.percentCompleteSlow,
count=91, min=99, max=99, mean=99.0, stddev=0.0, med
ian=99.0, p75=99.0, p95=99.0, p98=99.0, p99=99.0, p999=99.0
2016-04-21 09:39:30,279 INFO [ ome.system.metrics] (r-thread-1)
type=TIMER, name=ome.services.fulltext.FullTextIndexer.batch,
count=39580, min=0.5172059999999999, max=892.964662, mean=33.772100299
6109, stddev=98.63010058884122, median=1.489653, p75=1.7927864999999998,
p95=218.88564324999996, p98=338.4669497799999, p99=387.92385554000094,
p999=892.467126661, mean_rate=0.49974839091289713, m1=0.4958357429255289,
m5=0.4991677352501218, m15=0.4997274755749312, rate_unit=events/second,
duration_unit=milliseconds
2016-04-21 10:39:30,276 INFO [ ome.system.metrics] (r-thread-1)
type=HISTOGRAM,
name=ome.services.fulltext.FullTextIndexer.percentCompleteFast,
count=7589, min=99, max=99, mean=99.0, stddev=0.0, m
edian=99.0, p75=99.0, p95=99.0, p98=99.0, p99=99.0, p999=99.0
2016-04-21 10:39:30,277 INFO [ ome.system.metrics] (r-thread-1)
type=HISTOGRAM,
name=ome.services.fulltext.FullTextIndexer.percentCompleteSlow,
count=91, min=99, max=99, mean=99.0, stddev=0.0, med
ian=99.0, p75=99.0, p95=99.0, p98=99.0, p99=99.0, p999=99.0
More information about the ome-devel
mailing list