[ome-users] Problems with OMERO Imports

[Josh Moore]

Good morning,

On Thu, Dec 17, 2015 at 6:26 PM, Christian Carsten Sachs
<c.sachs at fz-juelich.de> wrote:

> On 12/17/2015 05:25 PM, Josh Moore wrote:
>>
>> On Thu, Dec 17, 2015 at 5:10 PM, Christian Carsten Sachs

>>> Nikon ND2 image stacks (.nd2 files) are imported using the command line
>>> client, in ln_s mode (called by a custom OMERO.dropbox-like script,
>>> enforcing some additional naming rules etc.).
>>
>> Is the script available somewhere?
>
> No, and we cannot easily upload is somewhere as it contains confidential
> info ...

Understood, but debugging without seeing what's going on can be
difficult. Do you have log files from your script? Can you elide out
the sensitive information?


> ... (Btw, having more complex import possibilities with OMERO.dropbox would
> be great, should we investigate creating a patch to OMERO.dropbox to
> enhance it with said functionality?)

In general, sure! You might be interested in
https://github.com/openmicroscopy/openmicroscopy/pull/4367 which will
hopefully be a part of 5.2.2 which will already simplify your dataset
creation.


>>> Import proceeds like normal, and all positions are added to OMERO, but
>>> the actual image data remains inaccessible. (Only the clock icon in
>>> thumbnail view in the webclient).
>>>
>>> In Blitz-0 (or similarily the individual file log file), the following
>>> errors are visible:
>>>
>>> (repeated for each plane)
>>> 2015-12-17 16:07:10,458 ERROR [ ome.formats.OMEROMetadataStoreClient]
>>> (2-thread-4) Server error setting extended properties for Pixels:8145
>>> Target file:sachs_102/2015-12/17/15-07-53.074/nd009.nd2

Unfortunately, that line:

https://github.com/openmicroscopy/openmicroscopy/blob/v5.2.0/components/blitz/src/ome/formats/OMEROMetadataStoreClient.java#L1882

swallows the exception. We could try to come up with a build that
would print out what is going on, but it might be that another log
file has related information. Could you possibly send us the
var/log/master* log files? The postgresql log files may also have some
information.


>> Is there any stack trace above this line? Perhaps send the whole log
>> (dir)?
>
> I did not see any stack trace, this is why I'm rather puzzled ...
> The whole logfile is some hundred megabytes large, I've split off the
> last import process (actually, since the last restart, as I had asked
> our admin to restart the system right before attempting the last import).
>
> As it's still too large to comfortably add it as attachment, please see
> following (temporary) link:
>
> https://fz-juelich.sciebo.de/index.php/s/zeltfZ4hfrJUMLU

Downloaded, thanks, but you're right, there's not enough info there.


> My wording might've been misunderstandable: It has worked in both the
> testing and production system; the production system was tested with the
> import script and some datasets in late November and everything worked fine.
> Yesterday we wanted to 'go live' and let users start to import data ...
> Thus we're a bit puzzled that problems occur now ...

One guess: is this file larger than others that were attempted
previously? The memory setting of your server is not particularly high
for production:

2015-12-17 15:03:48,260 INFO  [
ome.services.util.JvmSettingsCheck] (      main) Max Memory (MB):   =
 2185

Cheers,
~Josh


> PS: I forgot to mention that the file itself is properly openable in
> Fiji with current Bio-Formats.
>
> Thank you very much,
> Best regards
>
> Christian Sachs
>
>>> We're using OMERO 5.2.0 on a Windows server (which has - to my knowledge
>>> - been upgraded multiple times from earlier versions).
>>> As of the last three files imported, none was imported properly, we have
>>> told our users to stop imports till the problem is resolved.
>>>
>>> Thank you,
>>> best regards
>>>
>>> Christian Sachs