[ome-devel] OMERO upload problems
Douglas Russell
douglas.russell at bioch.ox.ac.uk
Thu Feb 27 16:12:24 GMT 2014
Hi Niko,
We have hit this same issue here in Oxford. It happens with gigantic
filesets. I think there is metadata for each of them and in combination, it
exceeds the 64MB. For now, we have set:
<property name="Ice.MessageSizeMax" value="262144"/>
I guess this should really be addressed to chunk the data into multiple
messages if it exceeds the size, but increasing this should get you going
for now.
Douglas
On 27 February 2014 15:59, Niko Ehrenfeuchter <
nikolaus.ehrenfeuchter at unibas.ch> wrote:
> Hi all,
>
> one of our users reported problems with uploading to OMERO with the error
> message below. He is one of our power users already having uploaded giant
> datasets into OMERO. According to him this problem occurs more or less
> randomly, however he notes this dataset has an unusual high amount of
> timepoints - maybe that's related?
>
> The MemoryLimitException below complains about a ridicoulous amount of RAM
> (64MB) being requested by this particular step, so maybe we have a bug
> somewhere in our configuration?
>
> Thanks a lot in advance
> Niko
>
>
> Ice.UnknownLocalException
>> unknown = "Ice::MemoryLimitException
>> Ice.MemoryLimitException
>> reason = "requested 67108872 bytes, maximum allowed
>> is 67108864 bytes (see Ice.MessageSizeMax)"
>> at IceInternal.Ex.throwMemoryLimitException(Ex.
>> java:23)
>> at IceInternal.BasicStream.
>> expand(BasicStream.java:2049)
>> at IceInternal.BasicStream.
>> writeLong(BasicStream.java:964)
>> at omero.sys.LongListHelper.
>> write(LongListHelper.java:37)
>> at omero.sys.EventContext.__
>> write(EventContext.java:136)
>> at IceInternal.BasicStream.
>> writeInstance(BasicStream.java:1721)
>> at IceInternal.BasicStream.writePendingObjects(
>> BasicStream.java:1639)
>> at omero.api._AMD_MetadataStore_
>> saveToDB.ice_response(_AMD_MetadataStore_saveToDB.java:40)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>> Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(
>> NativeMethodAccessorImpl.java:57)
>> at sun.reflect.DelegatingMethodAccessorImpl.
>> invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.
>> invoke(Method.java:601)
>> at ome.services.throttling.Task.
>> response(Task.java:63)
>> at ome.services.throttling.
>> Adapter.run(Adapter.java:63)
>> at ome.services.throttling.
>> InThreadThrottlingStrategy.runnableCall(InThreadThrottlingStrategy.
>> java:89)
>> at ome.services.blitz.impl.AbstractAmdServant.
>> runnableCall(AbstractAmdServant.java:163)
>> at ome.services.blitz.impl.
>> MetadataStoreI.saveToDB_async(MetadataStoreI.java:190)
>> at sun.reflect.GeneratedMethodAccessor1360.invoke(Unknown
>> Source)
>> at sun.reflect.DelegatingMethodAccessorImpl.
>> invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.
>> invoke(Method.java:601)
>> at org.springframework.aop.support.AopUtils.
>> invokeJoinpointUsingReflection(AopUtils.java:307)
>> at org.springframework.aop.framework.
>> ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.
>> java:183)
>> at org.springframework.aop.framework.
>> ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
>> at omero.cmd.CallContext.invoke(
>> CallContext.java:59)
>> at org.springframework.aop.framework.
>> ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
>> at org.springframework.aop.
>> framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
>> at $Proxy107.saveToDB_async(Unknown Source)
>> at omero.api._MetadataStoreTie.saveToDB_async(_
>> MetadataStoreTie.java:94)
>> at omero.api._MetadataStoreDisp._
>> __saveToDB(_MetadataStoreDisp.java:266)
>> at omero.api._MetadataStoreDisp._
>> _dispatch(_MetadataStoreDisp.java:414)
>> at IceInternal.Incoming.invoke(Incoming.java:159)
>> at Ice.ConnectionI.invokeAll(
>> ConnectionI.java:2357)
>> at Ice.ConnectionI.dispatch(
>> ConnectionI.java:1208)
>> at Ice.ConnectionI.message(ConnectionI.java:1163)
>> at IceInternal.ThreadPool.run(
>> ThreadPool.java:302)
>> at IceInternal.ThreadPool.access$
>> 300(ThreadPool.java:12)
>> at IceInternal.ThreadPool$EventHandlerThread.run(
>> ThreadPool.java:643)
>> at java.lang.Thread.run(Thread.java:722)
>> "
>> at IceInternal.Outgoing.invoke(Outgoing.java:158)
>> at omero.api._MetadataStoreDelM.saveToDB(_MetadataStoreDelM.
>> java:167)
>> at omero.api.MetadataStorePrxHelper.saveToDB(
>> MetadataStorePrxHelper.java:573)
>> at omero.api.MetadataStorePrxHelper.saveToDB(
>> MetadataStorePrxHelper.java:545)
>> at ome.formats.OMEROMetadataStoreClient.saveToDB(
>> OMEROMetadataStoreClient.java:2122)
>> at ome.formats.importer.ImportLibrary.importMetadata(
>> ImportLibrary.java:368)
>> at ome.formats.importer.ImportLibrary.importImage(
>> ImportLibrary.java:550)
>> at org.openmicroscopy.shoola.env.data.OMEROGateway.importImage(
>> OMEROGateway.java:6400)
>> at org.openmicroscopy.shoola.env.data.OmeroImageServiceImpl.
>> importFile(OmeroImageServiceImpl.java:1350)
>> at org.openmicroscopy.shoola.env.data.views.calls.
>> ImagesImporter.importFile(ImagesImporter.java:77)
>> at org.openmicroscopy.shoola.env.data.views.calls.
>> ImagesImporter.access$000(ImagesImporter.java:53)
>> at org.openmicroscopy.shoola.env.data.views.calls.
>> ImagesImporter$1.doCall(ImagesImporter.java:102)
>> at org.openmicroscopy.shoola.env.data.views.BatchCall.doStep(
>> BatchCall.java:144)
>> at org.openmicroscopy.shoola.util.concur.tasks.
>> CompositeTask.doStep(CompositeTask.java:226)
>> at org.openmicroscopy.shoola.env.data.views.CompositeBatchCall.
>> doStep(CompositeBatchCall.java:126)
>> at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.
>> exec(ExecCommand.java:165)
>> at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.
>> run(ExecCommand.java:276)
>> at org.openmicroscopy.shoola.util.concur.tasks.
>> AsyncProcessor$Runner.run(AsyncProcessor.java:91)
>> at java.lang.Thread.run(Thread.java:695)
>>
>> at org.openmicroscopy.shoola.env.data.OMEROGateway.importImage(
>> OMEROGateway.java:6454)
>> at org.openmicroscopy.shoola.env.data.OmeroImageServiceImpl.
>> importFile(OmeroImageServiceImpl.java:1350)
>> at org.openmicroscopy.shoola.env.data.views.calls.
>> ImagesImporter.importFile(ImagesImporter.java:77)
>> at org.openmicroscopy.shoola.env.data.views.calls.
>> ImagesImporter.access$000(ImagesImporter.java:53)
>> at org.openmicroscopy.shoola.env.data.views.calls.
>> ImagesImporter$1.doCall(ImagesImporter.java:102)
>> at org.openmicroscopy.shoola.env.data.views.BatchCall.doStep(
>> BatchCall.java:144)
>> at org.openmicroscopy.shoola.util.concur.tasks.
>> CompositeTask.doStep(CompositeTask.java:226)
>> at org.openmicroscopy.shoola.env.data.views.CompositeBatchCall.
>> doStep(CompositeBatchCall.java:126)
>> at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.
>> exec(ExecCommand.java:165)
>> at org.openmicroscopy.shoola.util.concur.tasks.ExecCommand.
>> run(ExecCommand.java:276)
>> at org.openmicroscopy.shoola.util.concur.tasks.
>> AsyncProcessor$Runner.run(AsyncProcessor.java:91)
>> at java.lang.Thread.run(Thread.java:695)
>> Caused by: Ice.UnknownLocalException
>> unknown = "Ice::MemoryLimitException
>> Ice.MemoryLimitException
>> reason = "requested 67108872 bytes, maximum allowed
>> is 67108864 bytes (see Ice.MessageSizeMax)"
>> at IceInternal.Ex.throwMemoryLimitException(Ex.
>> java:23)
>> at IceInternal.BasicStream.
>> expand(BasicStream.java:2049)
>> at IceInternal.BasicStream.
>> writeLong(BasicStream.java:964)
>> at omero.sys.LongListHelper.
>> write(LongListHelper.java:37)
>> at omero.sys.EventContext.__
>> write(EventContext.java:136)
>> at IceInternal.BasicStream.
>> writeInstance(BasicStream.java:1721)
>> at IceInternal.BasicStream.writePendingObjects(
>> BasicStream.java:1639)
>> at omero.api._AMD_MetadataStore_
>> saveToDB.ice_response(_AMD_MetadataStore_saveToDB.java:40)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>> Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(
>> NativeMethodAccessorImpl.java:57)
>> at sun.reflect.DelegatingMethodAccessorImpl.
>> invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.
>> invoke(Method.java:601)
>> at ome.services.throttling.Task.
>> response(Task.java:63)
>> at ome.services.throttling.
>> Adapter.run(Adapter.java:63)
>> at ome.services.throttling.
>> InThreadThrottlingStrategy.runnableCall(InThreadThrottlingStrategy.
>> java:89)
>> at ome.services.blitz.impl.AbstractAmdServant.
>> runnableCall(AbstractAmdServant.java:163)
>> at ome.services.blitz.impl.
>> MetadataStoreI.saveToDB_async(MetadataStoreI.java:190)
>> at sun.reflect.GeneratedMethodAccessor1360.invoke(Unknown
>> Source)
>> at sun.reflect.DelegatingMethodAccessorImpl.
>> invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.
>> invoke(Method.java:601)
>> at org.springframework.aop.support.AopUtils.
>> invokeJoinpointUsingReflection(AopUtils.java:307)
>> at org.springframework.aop.framework.
>> ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.
>> java:183)
>> at org.springframework.aop.framework.
>> ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
>> at omero.cmd.CallContext.invoke(
>> CallContext.java:59)
>> at org.springframework.aop.framework.
>> ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
>> at org.springframework.aop.
>> framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
>> at $Proxy107.saveToDB_async(Unknown Source)
>> at omero.api._MetadataStoreTie.saveToDB_async(_
>> MetadataStoreTie.java:94)
>> at omero.api._MetadataStoreDisp._
>> __saveToDB(_MetadataStoreDisp.java:266)
>> at omero.api._MetadataStoreDisp._
>> _dispatch(_MetadataStoreDisp.java:414)
>> at IceInternal.Incoming.invoke(Incoming.java:159)
>> at Ice.ConnectionI.invokeAll(
>> ConnectionI.java:2357)
>> at Ice.ConnectionI.dispatch(
>> ConnectionI.java:1208)
>> at Ice.ConnectionI.message(ConnectionI.java:1163)
>> at IceInternal.ThreadPool.run(
>> ThreadPool.java:302)
>> at IceInternal.ThreadPool.access$
>> 300(ThreadPool.java:12)
>> at IceInternal.ThreadPool$EventHandlerThread.run(
>> ThreadPool.java:643)
>> at java.lang.Thread.run(Thread.java:722)
>> "
>> at IceInternal.Outgoing.invoke(Outgoing.java:158)
>> at omero.api._MetadataStoreDelM.saveToDB(_MetadataStoreDelM.
>> java:167)
>> at omero.api.MetadataStorePrxHelper.saveToDB(
>> MetadataStorePrxHelper.java:573)
>> at omero.api.MetadataStorePrxHelper.saveToDB(
>> MetadataStorePrxHelper.java:545)
>> at ome.formats.OMEROMetadataStoreClient.saveToDB(
>> OMEROMetadataStoreClient.java:2122)
>> at ome.formats.importer.ImportLibrary.importMetadata(
>> ImportLibrary.java:368)
>> at ome.formats.importer.ImportLibrary.importImage(
>> ImportLibrary.java:550)
>> at org.openmicroscopy.shoola.env.data.OMEROGateway.importImage(
>> OMEROGateway.java:6400)
>> ... 11 more
>>
>
>
> --
> Niko Ehrenfeuchter
> Imaging Core Facility
> Kragenbau, Room G1055
> Biozentrum, University of Basel
> Klingelbergstrasse 50/70
> CH-4056 Basel (Switzerland)
>
> Office: +41 (61) 26 72673
> Email: nikolaus.ehrenfeuchter at unibas.ch
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20140227/263764ae/attachment-0001.html>
More information about the ome-devel
mailing list