<div dir="ltr">Hi Niko,<div><br></div><div>We have hit this same issue here in Oxford. It happens with gigantic filesets. I think there is metadata for each of them and in combination, it exceeds the 64MB. For now, we have set:</div>

<div><br></div><div>&lt;property name=&quot;Ice.MessageSizeMax&quot; value=&quot;262144&quot;/&gt;<br></div><div><br></div><div>I guess this should really be addressed to chunk the data into multiple messages if it exceeds the size, but increasing this should get you going for now.</div>

<div><br></div><div>Douglas</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On 27 February 2014 15:59, Niko Ehrenfeuchter <span dir="ltr">&lt;<a href="mailto:nikolaus.ehrenfeuchter@unibas.ch" target="_blank">nikolaus.ehrenfeuchter@unibas.ch</a>&gt;</span> wrote:<br>

<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi all,<br>
<br>
one of our users reported problems with uploading to OMERO with the error message below. He is one of our power users already having uploaded giant datasets into OMERO. According to him this problem occurs more or less randomly, however he notes this dataset has an unusual high amount of timepoints - maybe that&#39;s related?<br>


<br>
The MemoryLimitException below complains about a ridicoulous amount of RAM (64MB) being requested by this particular step, so maybe we have a bug somewhere in our configuration?<br>
<br>
Thanks a lot in advance<br>
Niko<br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Ice.UnknownLocalException<br>
     unknown = &quot;Ice::MemoryLimitException<br>
                Ice.MemoryLimitException<br>
                    reason = &quot;requested 67108872 bytes, maximum allowed is 67108864 bytes (see Ice.MessageSizeMax)&quot;<br>
                        at IceInternal.Ex.<u></u>throwMemoryLimitException(Ex.<u></u>java:23)<br>
                        at IceInternal.BasicStream.<u></u>expand(BasicStream.java:2049)<br>
                        at IceInternal.BasicStream.<u></u>writeLong(BasicStream.java:<u></u>964)<br>
                        at omero.sys.LongListHelper.<u></u>write(LongListHelper.java:37)<br>
                        at omero.sys.EventContext.__<u></u>write(EventContext.java:136)<br>
                        at IceInternal.BasicStream.<u></u>writeInstance(BasicStream.<u></u>java:1721)<br>
                        at IceInternal.BasicStream.<u></u>writePendingObjects(<u></u>BasicStream.java:1639)<br>
                        at omero.api._AMD_MetadataStore_<u></u>saveToDB.ice_response(_AMD_<u></u>MetadataStore_saveToDB.java:<u></u>40)<br>
                        at sun.reflect.<u></u>NativeMethodAccessorImpl.<u></u>invoke0(Native Method)<br>
                        at sun.reflect.<u></u>NativeMethodAccessorImpl.<u></u>invoke(<u></u>NativeMethodAccessorImpl.java:<u></u>57)<br>
                        at sun.reflect.<u></u>DelegatingMethodAccessorImpl.<u></u>invoke(<u></u>DelegatingMethodAccessorImpl.<u></u>java:43)<br>
                        at java.lang.reflect.Method.<u></u>invoke(Method.java:601)<br>
                        at ome.services.throttling.Task.<u></u>response(Task.java:63)<br>
                        at ome.services.throttling.<u></u>Adapter.run(Adapter.java:63)<br>
                        at ome.services.throttling.<u></u>InThreadThrottlingStrategy.<u></u>runnableCall(<u></u>InThreadThrottlingStrategy.<u></u>java:89)<br>
                        at ome.services.blitz.impl.<u></u>AbstractAmdServant.<u></u>runnableCall(<u></u>AbstractAmdServant.java:163)<br>
                        at ome.services.blitz.impl.<u></u>MetadataStoreI.saveToDB_async(<u></u>MetadataStoreI.java:190)<br>
                        at sun.reflect.<u></u>GeneratedMethodAccessor1360.<u></u>invoke(Unknown Source)<br>
                        at sun.reflect.<u></u>DelegatingMethodAccessorImpl.<u></u>invoke(<u></u>DelegatingMethodAccessorImpl.<u></u>java:43)<br>
                        at java.lang.reflect.Method.<u></u>invoke(Method.java:601)<br>
                        at org.springframework.aop.<u></u>support.AopUtils.<u></u>invokeJoinpointUsingReflection<u></u>(AopUtils.java:307)<br>
                        at org.springframework.aop.<u></u>framework.<u></u>ReflectiveMethodInvocation.<u></u>invokeJoinpoint(<u></u>ReflectiveMethodInvocation.<u></u>java:183)<br>
                        at org.springframework.aop.<u></u>framework.<u></u>ReflectiveMethodInvocation.<u></u>proceed(<u></u>ReflectiveMethodInvocation.<u></u>java:150)<br>
                        at omero.cmd.CallContext.invoke(<u></u>CallContext.java:59)<br>
                        at org.springframework.aop.<u></u>framework.<u></u>ReflectiveMethodInvocation.<u></u>proceed(<u></u>ReflectiveMethodInvocation.<u></u>java:172)<br>
                        at org.springframework.aop.<u></u>framework.JdkDynamicAopProxy.<u></u>invoke(JdkDynamicAopProxy.<u></u>java:202)<br>
                        at $Proxy107.saveToDB_async(<u></u>Unknown Source)<br>
                        at omero.api._MetadataStoreTie.<u></u>saveToDB_async(_<u></u>MetadataStoreTie.java:94)<br>
                        at omero.api._MetadataStoreDisp._<u></u>__saveToDB(_MetadataStoreDisp.<u></u>java:266)<br>
                        at omero.api._MetadataStoreDisp._<u></u>_dispatch(_MetadataStoreDisp.<u></u>java:414)<br>
                        at IceInternal.Incoming.invoke(<u></u>Incoming.java:159)<br>
                        at Ice.ConnectionI.invokeAll(<u></u>ConnectionI.java:2357)<br>
                        at Ice.ConnectionI.dispatch(<u></u>ConnectionI.java:1208)<br>
                        at Ice.ConnectionI.message(<u></u>ConnectionI.java:1163)<br>
                        at IceInternal.ThreadPool.run(<u></u>ThreadPool.java:302)<br>
                        at IceInternal.ThreadPool.access$<u></u>300(ThreadPool.java:12)<br>
                        at IceInternal.ThreadPool$<u></u>EventHandlerThread.run(<u></u>ThreadPool.java:643)<br>
                        at java.lang.Thread.run(Thread.<u></u>java:722)<br>
                &quot;<br>
        at IceInternal.Outgoing.invoke(<u></u>Outgoing.java:158)<br>
        at omero.api._MetadataStoreDelM.<u></u>saveToDB(_MetadataStoreDelM.<u></u>java:167)<br>
        at omero.api.<u></u>MetadataStorePrxHelper.<u></u>saveToDB(<u></u>MetadataStorePrxHelper.java:<u></u>573)<br>
        at omero.api.<u></u>MetadataStorePrxHelper.<u></u>saveToDB(<u></u>MetadataStorePrxHelper.java:<u></u>545)<br>
        at ome.formats.<u></u>OMEROMetadataStoreClient.<u></u>saveToDB(<u></u>OMEROMetadataStoreClient.java:<u></u>2122)<br>
        at ome.formats.importer.<u></u>ImportLibrary.importMetadata(<u></u>ImportLibrary.java:368)<br>
        at ome.formats.importer.<u></u>ImportLibrary.importImage(<u></u>ImportLibrary.java:550)<br>
        at org.openmicroscopy.shoola.env.<u></u>data.OMEROGateway.importImage(<u></u>OMEROGateway.java:6400)<br>
        at org.openmicroscopy.shoola.env.<u></u>data.OmeroImageServiceImpl.<u></u>importFile(<u></u>OmeroImageServiceImpl.java:<u></u>1350)<br>
        at org.openmicroscopy.shoola.env.<u></u>data.views.calls.<u></u>ImagesImporter.importFile(<u></u>ImagesImporter.java:77)<br>
        at org.openmicroscopy.shoola.env.<u></u>data.views.calls.<u></u>ImagesImporter.access$000(<u></u>ImagesImporter.java:53)<br>
        at org.openmicroscopy.shoola.env.<u></u>data.views.calls.<u></u>ImagesImporter$1.doCall(<u></u>ImagesImporter.java:102)<br>
        at org.openmicroscopy.shoola.env.<u></u>data.views.BatchCall.doStep(<u></u>BatchCall.java:144)<br>
        at org.openmicroscopy.shoola.<u></u>util.concur.tasks.<u></u>CompositeTask.doStep(<u></u>CompositeTask.java:226)<br>
        at org.openmicroscopy.shoola.env.<u></u>data.views.CompositeBatchCall.<u></u>doStep(CompositeBatchCall.<u></u>java:126)<br>
        at org.openmicroscopy.shoola.<u></u>util.concur.tasks.ExecCommand.<u></u>exec(ExecCommand.java:165)<br>
        at org.openmicroscopy.shoola.<u></u>util.concur.tasks.ExecCommand.<u></u>run(ExecCommand.java:276)<br>
        at org.openmicroscopy.shoola.<u></u>util.concur.tasks.<u></u>AsyncProcessor$Runner.run(<u></u>AsyncProcessor.java:91)<br>
        at java.lang.Thread.run(Thread.<u></u>java:695)<br>
<br>
        at org.openmicroscopy.shoola.env.<u></u>data.OMEROGateway.importImage(<u></u>OMEROGateway.java:6454)<br>
        at org.openmicroscopy.shoola.env.<u></u>data.OmeroImageServiceImpl.<u></u>importFile(<u></u>OmeroImageServiceImpl.java:<u></u>1350)<br>
        at org.openmicroscopy.shoola.env.<u></u>data.views.calls.<u></u>ImagesImporter.importFile(<u></u>ImagesImporter.java:77)<br>
        at org.openmicroscopy.shoola.env.<u></u>data.views.calls.<u></u>ImagesImporter.access$000(<u></u>ImagesImporter.java:53)<br>
        at org.openmicroscopy.shoola.env.<u></u>data.views.calls.<u></u>ImagesImporter$1.doCall(<u></u>ImagesImporter.java:102)<br>
        at org.openmicroscopy.shoola.env.<u></u>data.views.BatchCall.doStep(<u></u>BatchCall.java:144)<br>
        at org.openmicroscopy.shoola.<u></u>util.concur.tasks.<u></u>CompositeTask.doStep(<u></u>CompositeTask.java:226)<br>
        at org.openmicroscopy.shoola.env.<u></u>data.views.CompositeBatchCall.<u></u>doStep(CompositeBatchCall.<u></u>java:126)<br>
        at org.openmicroscopy.shoola.<u></u>util.concur.tasks.ExecCommand.<u></u>exec(ExecCommand.java:165)<br>
        at org.openmicroscopy.shoola.<u></u>util.concur.tasks.ExecCommand.<u></u>run(ExecCommand.java:276)<br>
        at org.openmicroscopy.shoola.<u></u>util.concur.tasks.<u></u>AsyncProcessor$Runner.run(<u></u>AsyncProcessor.java:91)<br>
        at java.lang.Thread.run(Thread.<u></u>java:695)<br>
Caused by: Ice.UnknownLocalException<br>
     unknown = &quot;Ice::MemoryLimitException<br>
                Ice.MemoryLimitException<br>
                    reason = &quot;requested 67108872 bytes, maximum allowed is 67108864 bytes (see Ice.MessageSizeMax)&quot;<br>
                        at IceInternal.Ex.<u></u>throwMemoryLimitException(Ex.<u></u>java:23)<br>
                        at IceInternal.BasicStream.<u></u>expand(BasicStream.java:2049)<br>
                        at IceInternal.BasicStream.<u></u>writeLong(BasicStream.java:<u></u>964)<br>
                        at omero.sys.LongListHelper.<u></u>write(LongListHelper.java:37)<br>
                        at omero.sys.EventContext.__<u></u>write(EventContext.java:136)<br>
                        at IceInternal.BasicStream.<u></u>writeInstance(BasicStream.<u></u>java:1721)<br>
                        at IceInternal.BasicStream.<u></u>writePendingObjects(<u></u>BasicStream.java:1639)<br>
                        at omero.api._AMD_MetadataStore_<u></u>saveToDB.ice_response(_AMD_<u></u>MetadataStore_saveToDB.java:<u></u>40)<br>
                        at sun.reflect.<u></u>NativeMethodAccessorImpl.<u></u>invoke0(Native Method)<br>
                        at sun.reflect.<u></u>NativeMethodAccessorImpl.<u></u>invoke(<u></u>NativeMethodAccessorImpl.java:<u></u>57)<br>
                        at sun.reflect.<u></u>DelegatingMethodAccessorImpl.<u></u>invoke(<u></u>DelegatingMethodAccessorImpl.<u></u>java:43)<br>
                        at java.lang.reflect.Method.<u></u>invoke(Method.java:601)<br>
                        at ome.services.throttling.Task.<u></u>response(Task.java:63)<br>
                        at ome.services.throttling.<u></u>Adapter.run(Adapter.java:63)<br>
                        at ome.services.throttling.<u></u>InThreadThrottlingStrategy.<u></u>runnableCall(<u></u>InThreadThrottlingStrategy.<u></u>java:89)<br>
                        at ome.services.blitz.impl.<u></u>AbstractAmdServant.<u></u>runnableCall(<u></u>AbstractAmdServant.java:163)<br>
                        at ome.services.blitz.impl.<u></u>MetadataStoreI.saveToDB_async(<u></u>MetadataStoreI.java:190)<br>
                        at sun.reflect.<u></u>GeneratedMethodAccessor1360.<u></u>invoke(Unknown Source)<br>
                        at sun.reflect.<u></u>DelegatingMethodAccessorImpl.<u></u>invoke(<u></u>DelegatingMethodAccessorImpl.<u></u>java:43)<br>
                        at java.lang.reflect.Method.<u></u>invoke(Method.java:601)<br>
                        at org.springframework.aop.<u></u>support.AopUtils.<u></u>invokeJoinpointUsingReflection<u></u>(AopUtils.java:307)<br>
                        at org.springframework.aop.<u></u>framework.<u></u>ReflectiveMethodInvocation.<u></u>invokeJoinpoint(<u></u>ReflectiveMethodInvocation.<u></u>java:183)<br>
                        at org.springframework.aop.<u></u>framework.<u></u>ReflectiveMethodInvocation.<u></u>proceed(<u></u>ReflectiveMethodInvocation.<u></u>java:150)<br>
                        at omero.cmd.CallContext.invoke(<u></u>CallContext.java:59)<br>
                        at org.springframework.aop.<u></u>framework.<u></u>ReflectiveMethodInvocation.<u></u>proceed(<u></u>ReflectiveMethodInvocation.<u></u>java:172)<br>
                        at org.springframework.aop.<u></u>framework.JdkDynamicAopProxy.<u></u>invoke(JdkDynamicAopProxy.<u></u>java:202)<br>
                        at $Proxy107.saveToDB_async(<u></u>Unknown Source)<br>
                        at omero.api._MetadataStoreTie.<u></u>saveToDB_async(_<u></u>MetadataStoreTie.java:94)<br>
                        at omero.api._MetadataStoreDisp._<u></u>__saveToDB(_MetadataStoreDisp.<u></u>java:266)<br>
                        at omero.api._MetadataStoreDisp._<u></u>_dispatch(_MetadataStoreDisp.<u></u>java:414)<br>
                        at IceInternal.Incoming.invoke(<u></u>Incoming.java:159)<br>
                        at Ice.ConnectionI.invokeAll(<u></u>ConnectionI.java:2357)<br>
                        at Ice.ConnectionI.dispatch(<u></u>ConnectionI.java:1208)<br>
                        at Ice.ConnectionI.message(<u></u>ConnectionI.java:1163)<br>
                        at IceInternal.ThreadPool.run(<u></u>ThreadPool.java:302)<br>
                        at IceInternal.ThreadPool.access$<u></u>300(ThreadPool.java:12)<br>
                        at IceInternal.ThreadPool$<u></u>EventHandlerThread.run(<u></u>ThreadPool.java:643)<br>
                        at java.lang.Thread.run(Thread.<u></u>java:722)<br>
                &quot;<br>
        at IceInternal.Outgoing.invoke(<u></u>Outgoing.java:158)<br>
        at omero.api._MetadataStoreDelM.<u></u>saveToDB(_MetadataStoreDelM.<u></u>java:167)<br>
        at omero.api.<u></u>MetadataStorePrxHelper.<u></u>saveToDB(<u></u>MetadataStorePrxHelper.java:<u></u>573)<br>
        at omero.api.<u></u>MetadataStorePrxHelper.<u></u>saveToDB(<u></u>MetadataStorePrxHelper.java:<u></u>545)<br>
        at ome.formats.<u></u>OMEROMetadataStoreClient.<u></u>saveToDB(<u></u>OMEROMetadataStoreClient.java:<u></u>2122)<br>
        at ome.formats.importer.<u></u>ImportLibrary.importMetadata(<u></u>ImportLibrary.java:368)<br>
        at ome.formats.importer.<u></u>ImportLibrary.importImage(<u></u>ImportLibrary.java:550)<br>
        at org.openmicroscopy.shoola.env.<u></u>data.OMEROGateway.importImage(<u></u>OMEROGateway.java:6400)<br>
        ... 11 more<br>
</blockquote>
<br>
<br>
-- <br>
Niko Ehrenfeuchter<br>
Imaging Core Facility<br>
Kragenbau, Room G1055<br>
Biozentrum, University of Basel<br>
Klingelbergstrasse 50/70<br>
CH-4056 Basel (Switzerland)<br>
<br>
Office:  +41 (61) 26 72673<br>
Email:   <a href="mailto:nikolaus.ehrenfeuchter@unibas.ch" target="_blank">nikolaus.ehrenfeuchter@unibas.<u></u>ch</a><br>
______________________________<u></u>_________________<br>
ome-devel mailing list<br>
<a href="mailto:ome-devel@lists.openmicroscopy.org.uk" target="_blank">ome-devel@lists.<u></u>openmicroscopy.org.uk</a><br>
<a href="http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel" target="_blank">http://lists.openmicroscopy.<u></u>org.uk/mailman/listinfo/ome-<u></u>devel</a><br>
</blockquote></div><br></div>