[ome-users] Error on max intensity for large volumes

[Josh Moore]

On Jun 13, 2013, at 10:04 PM, Josh Moore wrote:

> 
> On Jun 12, 2013, at 1:07 PM, ingvar wrote:
> 
>> Dear OMERO,
> 
> Hi Ingvar,

Hi again,


>> On an EM volume that contains more than 2^31 data points, I get an error when attempting to do "Max Intensity" in the OMERO.web image viewer.  The error message in OMEROweb.log and OMEROweb_request.log indicate that the cause is an overflow in an integer index in Java.  Assuming that the index points into a Java array, then there may be a bigger problem, as Java arrays in the standard Oracle implementation is limited to MAX_INTEGER - 8 (or some other small offset depending on who you believe).
>> 
>> There are several possible solutions here:
>> direct memory management (http://www.omsn.de/blog/big-arrays-in-java)
>> use a 3rd party library such as fastutil (http://fastutil.di.unimi.it)
>> rewrite the code, I do not think the whole data array has to be in memory for max intensity
> 
> It's unlikely we'd try any of the more advanced workarounds in the short-term, but certainly loading sections of the data should work.
> 
> 
>> Steps to reproduce:
>> Fetch the file ftp://ftp.ebi.ac.uk/pub/databases/emdb/structures/EMD-2362/map/emd_2362.map.gz (2.6 GB)
>> Upload into OMERO
>> In OMERO.web's image viewer select "Max Intensity"
> 
> Thanks for the steps and the data. I'll download and try to reproduce.

Against, 4.4.8 there is an exception. See https://trac.openmicroscopy.org.uk/ome/ticket/11126

The web client is left in the "Loading" state. It's highly unlikely a fix will be backported to 4.3.3, but we'll certainly look into fixing it for 4.4.


>> I am on a pretty old version, 4.3.3, if I remember correctly, so if this has been addressed since then, I guess this is the incentive for me to upgrade.
> 
> Let's hope so!
> 
>> Kind Regards,
>> Ingvar Lagerstedt
> 
> Cheers,
> ~Josh

Thanks again,
~Josh.