[ome-devel] Crash of OMERO's PixelDataHandler

[Josh Moore]

Tag,


On Mon, Aug 29, 2016 at 1:46 PM, Niko Ehrenfeuchter
<nikolaus.ehrenfeuchter at unibas.ch> wrote:
> Hi Josh,
>
> On 29.08.2016 13:05, Josh Moore wrote:
>>
>> Kai & Niko,
>>
>> Thanks for the bug report.
>>
>> On Sat, Aug 27, 2016 at 3:31 PM, Kai Schleicher
>> <kai.schleicher at unibas.ch> wrote:
>>>
>>> Dear all,
>>>
>>> I have uploaded one of the images (ID 17318) which leads to the crash as
>>> reported by Niko.
>>>
>>> I used your "send file" option since I did not want to produce the same
>>> crash on your demo server - I hope this is ok:
>>>
>>> http://qa.openmicroscopy.org.uk/qa/feedback/17318/?token=1e90a4571044b5a36e817ca32f3393d5
>>
>>
>> Thanks for the file. We haven't been able to reproduce the failure
>> yet, but we'll keep you posted. (Any server config etc. that's worth
>> knowing?)
>
>
> don't know, what can I send you? bin/omero config get? Any additional XML
> configs? Storage setup? ;-)

bin/omero config get --hide-password
If you think there's anything of interest, but judging by the rest of
your email, there's not.



>>> Von: Niko Ehrenfeuchter [nikolaus.ehrenfeuchter at unibas.ch]
>>> Gesendet: Freitag, 26. August 2016 17:31
>>> An: OME devel
>>> Cc: Kai Schleicher
>>> Betreff: Crash of OMERO's PixelDataHandler
>>>
>>> Dear all,
>>>
>>> (not sure if -dev is the appropriate place for this, please feel free to
>>> redirect to -users if appropriate)
>>>
>>> we're experiencing crashes of the pyramid generation process in OMERO
>>> 5.2.5, happening multiple times today already. I have attached an
>>> excerpt of /var/log/PixelData-0.log around the time when the last crash
>>> happened.
>>>
>>> The major problem here is that the generation of pyramid data stops
>>> completely after this type of crash, until the entire OMERO services are
>>> restarted.
>>>
>>> Any hints for us?
>>
>>
>> Until we know more, the first workaround I can think of is to increase
>> omero.sessions.timeout. See:
>>
>>
>> https://www.openmicroscopy.org/site/support/omero5.2/sysadmins/config.html#omero-sessions-timeout
>
>
> I'll try to look into this later today!

Let us know how that goes.


>> Compared to other pyramids you've generated, how long would you
>> *expect* this to take?
>
>
> That's really hard to judge. There's quite some load on the server right
> now, plus we're sometimes experiencing some slowness on the storage side, so
> I do not have a good feeling about what to expect (with all the different
> components influencing the overall computation time for pyramids).
>
> In the end, it might just be caused by the above mentioned storage slowness,
> leading to the timeout. Would this be a possible explanation, i.e.
> would/could "lockups" of the storage system result in the reported log
> messages?

Having storage be slow would certainly be enough to lead to timeouts.
However, first, there shouldn't really be a timeout (but of course,
what happen if it's hanging forever...) and second, *if* a timeout is
hit, it shouldn't require a restart.


> Cheers,
> Niko

All the best,
~Josh