[ome-devel] batch export unsuccessful

[Josh Moore]

On Sep 9, 2014, at 5:58 PM, Yanling Liu wrote:

> Josh,

Hi Yanling,

> I have solved the problem by redirecting the OMERO tmp folder to a larger
> partition. It was the hard drive space limitation causing the problem.

For anyone looking for more information about this, OMERO_TEMPDIR is
described under:

https://www.openmicroscopy.org/site/support/omero5/sysadmins/unix/server-installation.html#environment-variables

and with more detail under:

https://www.openmicroscopy.org/site/support/omero5/developers/Modules/TempFileManager.html?highlight=omero_tempdir


> On the other hand, I have found that Blitz-0.log files can get really big
> along with Blitz-0.log.1, Blitz-0.log.2... files. Is there a way to control
> the size and life time for Blitz log files? Otherwise I would have to
> create a cron job to clean up Blitz log files periodically.


You can modify "FixedWindowRollingPolicy" and "SizeBasedTriggeringPolicy"
in https://github.com/openmicroscopy/openmicroscopy/blob/v.5.0.4/etc/logback.xml
to reduce the amount of logging that takes place.

For more information, see 
https://www.openmicroscopy.org/site/support/omero5/developers/logging.html?highlight=logging#java-servers


> Many thanks,
> Yanling

Cheers,
~Josh


> On Tue, Sep 9, 2014 at 5:14 AM, Josh Moore <josh at glencoesoftware.com> wrote:
> 
>> 
>> On Sep 8, 2014, at 7:30 PM, Yanling Liu wrote:
>> 
>>> Hello,
>> 
>> Hi Yanling,
>> 
>>> Tried to do batch image export over a 76 images dataset but failed.
>>> 
>>> As shown in the attached images.tif file, the processing seems to be
>>> interrupted as output log message got truncated. It failed at
>> 5963P_04.tif
>>> and there's around 20 images remain unprocessed.
>> 
>> Do you still have the Blitz-0.log and the Processor-0.log for that event?
>> var/log/master.out and var/log/master.err may also be useful in case the
>> Python process is crashing.
>> 
>>> Could this a memory issue? Any hint to modify ome configuration to
>> increase
>>> useable memory for batch export?
>> 
>> It really depends on which service is having issues. The documentation for
>> bumping memory is available at:
>> 
>> 
>> https://www.openmicroscopy.org/site/support/omero5/sysadmins/server-performance.html#jvm-memory-settings
>> 
>> But you might be able to wait until we pinpoint the error.
>> 
>>> In the attached batch_image_export.tiff file, the failed batch export
>> task
>>> is shown as green check mark and a previous batch export task with
>> smaller
>>> number of images was success.
>>> 
>>> Thanks,
>>> Yanling
>> 
>> 
>> Cheers,
>> ~Josh.