[ome-devel] Problem when importing >1000 images

[Andrii Iudin]

Dear Josh,

Thank you for pointing to the documentation on the remote shares. Those 
.lock files usually appear if we stop the server after one of the 
"crashes". When stopping and starting the server during its normal 
functioning they seem to be not created.

The run_command definition is following:
     def run_command(self, command, logFile=None):
         p = subprocess.Popen(shlex.split(command), 
env=self.environment, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
         (out, err) = p.communicate()
     if logFile != None:
             logFile.write("\n" + str(command) + "\n\nSTDOUT\n\n" + out 
+ "\n\nSTDERR\n\n" + err + "\n")
         if p.returncode:
             if logFile != None:
                  try:
                      logfile.close()
                  except:
                      pass
             raise Exception("Failure executing: " + command + " 
:::out::: " + out + " :::err::: " + err)


Best regards,
Andrii

On 26/04/2016 21:00, Josh Moore wrote:
> Hi Andrii,
>
> On Tue, Apr 26, 2016 at 10:56 AM, Andrii Iudin <andrii at ebi.ac.uk> wrote:
>> Dear Josh,
>>
>> Please find attached the import script. For each EMDB entry it performs an
>> import of six images - three sides and their thumbnails.
> Thanks for this. And where's the definition of `run_command`?
>
>
>> To stop OMERO we use "omero web stop" and then "omero admin stop" commands.
>> After this it is necessary to remove
>> var/OMERO.data/.omero/repository/*/.lock files before starting OMERO again.
>> The system is NFS.
> I'd assume then that disconnections & the .lock files are unrelated.
> Please see https://www.openmicroscopy.org/site/support/omero5.2/sysadmins/unix/server-binary-repository.html#locking-and-remote-shares
> regarding using remote shares.
>
> Cheers,
> ~Josh.
>
>
>
>> Best regards,
>> Andrii
>>
>>
>> On 25/04/2016 16:21, Josh Moore wrote:
>>> On Fri, Apr 22, 2016 at 12:41 PM, Andrii Iudin <andrii at ebi.ac.uk> wrote:
>>>> Dear OMERO developers,
>>>>
>>>> We are experiencing an issue when importing a large number of images in a
>>>> single consequent go. This usually happens after importing more than a
>>>> thousand images. Please see below excerpts from the logs. Increasing a
>>>> time
>>>> period between each import seemed to helped a bit, however this issue
>>>> ultimately happened anyway.
>>> Is this script available publicly? It would be useful to see how it's
>>> working.
>>>
>>>
>>>> To get OMERO server working after this happens,
>>>> it is necessary to stop it, remove .lock files and start the server
>>>> again.
>>>> It would be much appreciated if you could point out to a possible way to
>>>> solve this issue.
>>> How did you stop OMERO? Is your file system on NFS or another remote
>>> share?
>>>
>>>
>>>> Thank you and with best regards,
>>>> Andrii
>>> Cheers,
>>> ~Josh.
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel