[ome-users] In-place import from CIFS share fails with permission error
Brian Dunnette
dunn0172 at umn.edu
Fri Jan 12 17:49:27 GMT 2018
Simon-
Thanks for the tip - however, when I add "UMask=0022" to the systemd file
and reload the service, I get the same error:
[Service]
> UMask=0022
>
> [dunn0172 at nunstudy-omero system]$ sudo systemctl daemon-reload && sudo
> systemctl reload-or-restart omero-admin
> [inplace_user at nunstudy-omero ~]$ omero import -- --transfer=ln_s
> /mnt/ahc-ns/Pathology/NunStudy/NS-1137/561.svs
> ...
> 11:36:07.268 [main] INFO o.f.i.transfers.SymlinkFileTransfer -
> Transferring /mnt/ahc-ns/Pathology/NunStudy/NS-1137/561.svs...
> 11:36:07.388 [main] INFO o.f.i.cli.LoggingImportMonitor -
> FILE_UPLOAD_STARTED: /mnt/ahc-ns/Pathology/NunStudy/NS-1137/561.svs
> 11:36:07.422 [main] DEBUG o.f.i.cli.LoggingImportMonitor - FILE_EXCEPTION:
> /mnt/ahc-ns/Pathology/NunStudy/NS-1137/561.svs
> 11:36:07.422 [main] ERROR o.formats.importer.cli.ErrorHandler -
> FILE_EXCEPTION: /mnt/ahc-ns/Pathology/NunStudy/NS-1137/561.svs
> java.lang.RuntimeException:
> ---------------------------------------------------
> Check failed:
> /omero/ManagedRepository/dunn0172admin_3/2018-01/12/11-36-06.810/561.svs
> cannot be modified locally!
> You likely do not have access to the ManagedRepository for in-place import.
> Aborting...
>
> [inplace_user at nunstudy-omero ~]$ ls -la
> /omero/ManagedRepository/dunn0172admin_3/2018-01/12/11-36-06.810/561.svs
> -rw-r--r-- 1 omero inplace_user 36 Jan 12 11:36
> /omero/ManagedRepository/dunn0172admin_3/2018-01/12/11-36-06.810/561.svs
>
However, when I change the umask to the 0002 mentioned in the
inplace-import document, it seems to move things along some, creating the
link and reading data (but still erroring later - maybe because the omero
system user doesn't own the link?):
[Service]
> UMask=0002
>
> [dunn0172 at nunstudy-omero system]$ sudo systemctl daemon-reload && sudo
> systemctl reload-or-restart omero-admin
>
> [inplace_user at nunstudy-omero ~]$ omero import -- --transfer=ln_s
> /mnt/ahc-ns/Pathology/NunStudy/NS-1137/561.svs
> ...
> 11:42:46.521 [main] INFO o.f.i.transfers.SymlinkFileTransfer -
> Transferring /mnt/ahc-ns/Pathology/NunStudy/NS-1137/561.svs...
> 11:42:46.778 [main] INFO o.f.i.cli.LoggingImportMonitor -
> FILE_UPLOAD_STARTED: /mnt/ahc-ns/Pathology/NunStudy/NS-1137/561.svs
> 11:42:59.066 [main] DEBUG o.f.i.cli.LoggingImportMonitor -
> FILE_UPLOAD_BYTES uploaded: 342957409 of: 342957409 bytes (ETA: 0' 0")
> 11:42:59.105 [main] DEBUG o.f.i.cli.LoggingImportMonitor - FILE_EXCEPTION:
> /mnt/ahc-ns/Pathology/NunStudy/NS-1137/561.svs
> 11:42:59.105 [main] ERROR o.formats.importer.cli.ErrorHandler -
> FILE_EXCEPTION: /mnt/ahc-ns/Pathology/NunStudy/NS-1137/561.svs
> omero.ResourceError: null
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method) ~[na:1.8.0_151]
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> ~[na:1.8.0_151]
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> ~[na:1.8.0_151]
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> ~[na:1.8.0_151]
> at java.lang.Class.newInstance(Class.java:442) ~[na:1.8.0_151]
> at
> IceInternal.BasicStream.createUserException(BasicStream.java:2779)
> ~[ice.jar:na]
> at IceInternal.BasicStream.access$300(BasicStream.java:14)
> ~[ice.jar:na]
> at
> IceInternal.BasicStream$EncapsDecoder10.throwException(BasicStream.java:3298)
> ~[ice.jar:na]
> at IceInternal.BasicStream.throwException(BasicStream.java:2291)
> ~[ice.jar:na]
> at
> IceInternal.OutgoingAsync.throwUserException(OutgoingAsync.java:399)
> ~[ice.jar:na]
> at
> omero.api.RawFileStorePrxHelper.end_save(RawFileStorePrxHelper.java:724)
> ~[blitz.jar:na]
> at
> omero.api.RawFileStorePrxHelper.save(RawFileStorePrxHelper.java:611)
> ~[blitz.jar:na]
> at
> omero.api.RawFileStorePrxHelper.save(RawFileStorePrxHelper.java:598)
> ~[blitz.jar:na]
> at
> ome.formats.importer.transfers.TransferState.save(TransferState.java:128)
> ~[blitz.jar:na]
> at
> ome.formats.importer.transfers.AbstractFileTransfer.finish(AbstractFileTransfer.java:114)
> ~[blitz.jar:na]
> at
> ome.formats.importer.transfers.AbstractExecFileTransfer.transfer(AbstractExecFileTransfer.java:70)
> ~[blitz.jar:na]
> at
> ome.formats.importer.ImportLibrary.uploadFile(ImportLibrary.java:437)
> [blitz.jar:na]
> at
> ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:511)
> [blitz.jar:na]
> at
> ome.formats.importer.ImportLibrary.importCandidates(ImportLibrary.java:289)
> [blitz.jar:na]
> at
> ome.formats.importer.cli.CommandLineImporter.start(CommandLineImporter.java:289)
> [blitz.jar:na]
> at
> ome.formats.importer.cli.CommandLineImporter.main(CommandLineImporter.java:958)
> [blitz.jar:na]
> 11:42:59.110 [main] DEBUG o.f.i.cli.LoggingImportMonitor -
> FILE_UPLOAD_ERROR
> 11:42:59.110 [main] ERROR ome.formats.importer.ImportLibrary - Error on
> import: cannot flush
> /omero/ManagedRepository/dunn0172admin_3/2018-01/12/11-42-45.714/561.svs:
> java.io.IOException: Permission denied
> omero.ResourceError: null
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method) ~[na:1.8.0_151]
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> ~[na:1.8.0_151]
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> ~[na:1.8.0_151]
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> ~[na:1.8.0_151]
> at java.lang.Class.newInstance(Class.java:442) ~[na:1.8.0_151]
> at
> IceInternal.BasicStream.createUserException(BasicStream.java:2779)
> ~[ice.jar:na]
> at IceInternal.BasicStream.access$300(BasicStream.java:14)
> ~[ice.jar:na]
> at
> IceInternal.BasicStream$EncapsDecoder10.throwException(BasicStream.java:3298)
> ~[ice.jar:na]
> at IceInternal.BasicStream.throwException(BasicStream.java:2291)
> ~[ice.jar:na]
> at
> IceInternal.OutgoingAsync.throwUserException(OutgoingAsync.java:399)
> ~[ice.jar:na]
> at
> omero.api.RawFileStorePrxHelper.end_save(RawFileStorePrxHelper.java:724)
> ~[blitz.jar:na]
> at
> omero.api.RawFileStorePrxHelper.save(RawFileStorePrxHelper.java:611)
> ~[blitz.jar:na]
> at
> omero.api.RawFileStorePrxHelper.save(RawFileStorePrxHelper.java:598)
> ~[blitz.jar:na]
> at
> ome.formats.importer.transfers.TransferState.save(TransferState.java:128)
> ~[blitz.jar:na]
> at
> ome.formats.importer.transfers.AbstractFileTransfer.finish(AbstractFileTransfer.java:114)
> ~[blitz.jar:na]
> at
> ome.formats.importer.transfers.AbstractExecFileTransfer.transfer(AbstractExecFileTransfer.java:70)
> ~[blitz.jar:na]
> at
> ome.formats.importer.ImportLibrary.uploadFile(ImportLibrary.java:437)
> ~[blitz.jar:na]
> at
> ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:511)
> ~[blitz.jar:na]
> at
> ome.formats.importer.ImportLibrary.importCandidates(ImportLibrary.java:289)
> ~[blitz.jar:na]
> at
> ome.formats.importer.cli.CommandLineImporter.start(CommandLineImporter.java:289)
> [blitz.jar:na]
> at
> ome.formats.importer.cli.CommandLineImporter.main(CommandLineImporter.java:958)
> [blitz.jar:na]
> 11:42:59.110 [main] INFO ome.formats.importer.ImportLibrary - Exiting on
> error
>
> ==> Summary
> 0 files uploaded, 0 filesets created, 0 images imported, 1 error in
> 0:00:13.564
> 11:42:59.166 [main] DEBUG ome.formats.OMEROMetadataStoreClient - closing
> client session.
> 11:42:59.190 [main] DEBUG ome.formats.OMEROMetadataStoreClient - client
> closed.
> 11:42:59.190 [main] DEBUG ome.formats.OMEROMetadataStoreClient - Logout
> called, shutting keep alive down.
> 11:42:59.190 [main] DEBUG ome.formats.OMEROMetadataStoreClient - keepalive
> shut down.
> !! 01/12/18 11:42:59.336 error: communicator not destroyed during global
> destruction.
>
> [inplace_user at nunstudy-omero ~]$ ls -la
> /omero/ManagedRepository/dunn0172admin_3/2018-01/12/11-42-45.714/561.svs
> lrwxrwxrwx 1 inplace_user inplace_user 46 Jan 12 11:42
> /omero/ManagedRepository/dunn0172admin_3/2018-01/12/11-42-45.714/561.svs ->
> /mnt/ahc-ns/Pathology/NunStudy/NS-1137/561.svs
>
Thanks,
Brian Dunnette
Digital Imaging & Info Tech
Lab Medicine and Pathology <http://pathology.umn.edu>
University of Minnesota
dunn0172 at umn.edu
(612) 424-8187 (Mobile)
(612) 624-6176 (Office)
-----
The information transmitted in this e-mail is intended only for the person
or entity to which it is addressed and may contain confidential and/or
privileged material, including 'protected health information'. If you are
not the intended recipient, you are hereby notified that any review,
retransmission, dissemination, distribution, or copying of this message is
strictly prohibited. If you have received this communication in error,
please destroy and delete this message from any computer and contact us
immediately by return e-mail.
On Fri, Jan 12, 2018 at 4:38 AM, Simon Li <spli at dundee.ac.uk> wrote:
> Hi Brian
>
> Systemd services default to a different umask (0002):
> https://www.freedesktop.org/software/systemd/man/systemd.exec.html#UMask=
> If you add "UMask=0022" to the [Service] section of omero-systemd.service
> (note capitalisation is important), then run "systemctl daemon-reload" to
> update systemd, and finally restart OMERO, I think this will fix things.
>
> I'll check with the rest of out team whether it's safe to update our docs
> to change the default.
>
> Simon
>
> On 11 January 2018 at 21:13, Brian Dunnette <dunn0172 at umn.edu> wrote:
>
>> Simon-
>>
>> Here are the details (to the best of my knowledge) of our server:
>>
>> - We're running Red Hat Enterprise v7.4
>> - OMERO was installed based on the CentOS 7 doc here:
>> https://docs.openmicroscopy.org/omero/5.3.5/sysadmins/unix/
>> server-centos7-ice36.html
>> <https://docs.openmicroscopy.org/omero/5.3.5/sysadmins/unix/server-centos7-ice36.html>
>> - OMERO.admin is started via systemd, using the script from the docs (
>> https://docs.openmicroscopy.org/omero/5.3.5/_downloads/omer
>> o-systemd.service
>> <https://docs.openmicroscopy.org/omero/5.3.5/_downloads/omero-systemd.service>
>> )
>>
>> Root, /home, /var, and /omero (the binary repository) are each their own
>> XFS volumes, while the images we're trying to import/link are on an eNAS
>> (mounted as the omero system user):
>>
>> [dunn0172 at nunstudy-omero system]$ grep inplace /etc/passwd /etc/group
>>> /etc/passwd:inplace_user:x:1003:1003::/home/inplace_user:/bin/bash
>>> /etc/group:omero:x:1002:inplace_user
>>> /etc/group:inplace_user:x:1003:
>>>
>>> # /etc/fstab
>>> UUID=e96e0f81-c366-4c30-bc2b-3a6f47d54d13 / xfs defaults
>>> 0 0
>>> /dev/mapper/vg_os-home /home xfs defaults
>>> 0 0
>>> /dev/mapper/vg_os-var /var xfs defaults
>>> 0 0
>>> /dev/mapper/vg_os-swap swap swap defaults
>>> 0 0
>>> /dev/mapper/vg_os-omero /omero xfs defaults
>>> 0 0
>>> //files.umn.edu/ahc-ns /mnt/ahc-ns/ cifs
>>> credentials=/root/files.umn.edu.creds,dir_mode=0777,file_mod
>>> e=0777,sec=ntlmssp,uid=1002,gid=1003 0 0
>>>
>>
>> Thanks,
>>
>>
>> Brian Dunnette
>>
>> Digital Imaging & Info Tech
>>
>> Lab Medicine and Pathology <http://pathology.umn.edu>
>>
>> University of Minnesota
>>
>> dunn0172 at umn.edu
>> (612) 424-8187 (Mobile)
>> (612) 624-6176 (Office)
>> -----
>> The information transmitted in this e-mail is intended only for the
>> person or entity to which it is addressed and may contain confidential
>> and/or privileged material, including 'protected health information'. If
>> you are not the intended recipient, you are hereby notified that any
>> review, retransmission, dissemination, distribution, or copying of this
>> message is strictly prohibited. If you have received this communication in
>> error, please destroy and delete this message from any computer and contact
>> us immediately by return e-mail.
>>
>> On Thu, Jan 11, 2018 at 12:34 PM, Simon Li <spli at dundee.ac.uk> wrote:
>>
>>> Hi Brian
>>>
>>> Could you give us some details about your OMERO system, including:
>>> - Operating system and version
>>> - How you installed OMERO (e.g. manually, scripted, instructions on one
>>> of our web-pages)
>>> - How you're starting/stopping OMERO (e.g. manually, systemd, upstart)
>>> - Disk volume mount options
>>>
>>> Thanks, Simon
>>>
>>> On 11 January 2018 at 17:54, Brian Dunnette <dunn0172 at umn.edu> wrote:
>>>
>>>> Mark-
>>>>
>>>> It appears that the svs file is owned by the omero system user;
>>>> however, though I ran the chmod/chgrp commands from the inplace-import
>>>> document (and my umask is 0002), the permissions on new import directories
>>>> and their files don't seem to be set correctly:
>>>>
>>>> [omero at nunstudy-omero ~]$ umask
>>>>> 0002
>>>>> [inplace_user at nunstudy-omero 11-43-02.765]$ umask
>>>>> 0002
>>>>>
>>>>> # Change-group script
>>>>> chgrp inplace_user /omero/ManagedRepository
>>>>> chmod g+rws /omero/ManagedRepository
>>>>> chmod g+rws /omero/ManagedRepository/*
>>>>> chmod g+rws /omero/ManagedRepository/*/*
>>>>> chmod g+rws /omero/ManagedRepository/*/*/*
>>>>> chmod g+rws /omero/ManagedRepository/*/*/*/*
>>>>> chgrp inplace_user /omero/ManagedRepository/*
>>>>> chgrp inplace_user /omero/ManagedRepository/*/*
>>>>> chgrp inplace_user /omero/ManagedRepository/*/*/*
>>>>> chgrp inplace_user /omero/ManagedRepository/*/*/*/*
>>>>>
>>>>> [inplace_user at nunstudy-omero 11]$ omero import -- --transfer=ln_s
>>>>> /mnt/ahc-ns/Pathology/NunStudy/NS-1137/562.svs
>>>>> ...
>>>>> 11:43:03.083 [main] INFO o.f.i.cli.LoggingImportMonitor -
>>>>> FILE_UPLOAD_STARTED: /mnt/ahc-ns/Pathology/NunStudy/NS-1137/562.svs
>>>>> 11:43:03.105 [main] DEBUG o.f.i.cli.LoggingImportMonitor -
>>>>> FILE_EXCEPTION: /mnt/ahc-ns/Pathology/NunStudy/NS-1137/562.svs
>>>>> 11:43:03.106 [main] ERROR o.formats.importer.cli.ErrorHandler -
>>>>> FILE_EXCEPTION: /mnt/ahc-ns/Pathology/NunStudy/NS-1137/562.svs
>>>>> java.lang.RuntimeException:
>>>>> ---------------------------------------------------
>>>>> Check failed: /omero/ManagedRepository/dunn0
>>>>> 172admin_3/2018-01/11/11-43-02.765/562.svs cannot be modified locally!
>>>>> You likely do not have access to the ManagedRepository for in-place
>>>>> import.
>>>>> Aborting...
>>>>>
>>>>> [inplace_user at nunstudy-omero ~]$ ls -la /omero/ManagedRepository/dunn0
>>>>> 172admin_3/2018-01/
>>>>> total 0
>>>>> drwxrwsr-x 4 omero inplace_user 26 Jan 11 11:40 .
>>>>> drwxrwsr-x 3 omero inplace_user 21 Jan 10 11:22 ..
>>>>> drwxrwsr-x 3 omero inplace_user 26 Jan 10 11:22 10
>>>>> drwxr-sr-x 4 omero inplace_user 46 Jan 11 11:43 11
>>>>>
>>>>> [inplace_user at nunstudy-omero ~]$ ls -la /omero/ManagedRepository/dunn0
>>>>> 172admin_3/2018-01/11/
>>>>> total 0
>>>>> drwxr-sr-x 4 omero inplace_user 46 Jan 11 11:43 .
>>>>> drwxrwsr-x 4 omero inplace_user 26 Jan 11 11:40 ..
>>>>> drwxrwxr-x 2 omero inplace_user 21 Jan 11 11:40 11-40-49.349
>>>>> drwxr-sr-x 2 omero inplace_user 21 Jan 11 11:43 11-43-02.765
>>>>>
>>>>> [inplace_user at nunstudy-omero ~]$ ls -la /omero/ManagedRepository/dunn0
>>>>> 172admin_3/2018-01/11/11-43-02.765/
>>>>> total 4
>>>>> drwxr-sr-x 2 omero inplace_user 21 Jan 11 11:43 .
>>>>> drwxr-sr-x 4 omero inplace_user 46 Jan 11 11:43 ..
>>>>> -rw-r--r-- 1 omero inplace_user 36 Jan 11 11:43 562.svs
>>>>>
>>>>>
>>>> Suggestions?
>>>>
>>>> Thanks,
>>>>
>>>> Brian Dunnette
>>>>
>>>> Digital Imaging & Info Tech
>>>>
>>>> Lab Medicine and Pathology <http://pathology.umn.edu>
>>>>
>>>> University of Minnesota
>>>>
>>>> dunn0172 at umn.edu
>>>> (612) 424-8187 (Mobile)
>>>> (612) 624-6176 (Office)
>>>> -----
>>>> The information transmitted in this e-mail is intended only for the
>>>> person or entity to which it is addressed and may contain confidential
>>>> and/or privileged material, including 'protected health information'. If
>>>> you are not the intended recipient, you are hereby notified that any
>>>> review, retransmission, dissemination, distribution, or copying of this
>>>> message is strictly prohibited. If you have received this communication in
>>>> error, please destroy and delete this message from any computer and contact
>>>> us immediately by return e-mail.
>>>>
>>>> On Thu, Jan 11, 2018 at 4:35 AM, Mark Carroll <
>>>> m.t.b.carroll at dundee.ac.uk> wrote:
>>>>
>>>>> Dear Brian,
>>>>>
>>>>> We're attempting to do an in-place import of SVS files from a CIFS
>>>>>> share
>>>>>> into our new OMERO 5.3.5 install, and followed the in-place import
>>>>>> guide
>>>>>> (set up the inplace_user user & group, gave them ownership of the
>>>>>> ManagedRepository, verified that inplace_user can create a directory
>>>>>> in
>>>>>> ManagedRepository and the omero system user can delete it).
>>>>>>
>>>>>
>>>>> Check failed:
>>>>>> /omero/ManagedRepository/dunn0172admin_3/2018-01/10/11-15-14
>>>>>> .918/561.svs
>>>>>> cannot be modified locally!
>>>>>>
>>>>>> You likely do not have access to the ManagedRepository for
>>>>>> in-place
>>>>>> import.
>>>>>>
>>>>>
>>>>> Is this file owned by the omero system user? Is the inplace_user able
>>>>> to:
>>>>>
>>>>> * append to it?
>>>>> * delete it?
>>>>> * create another file in that same
>>>>> /omero/ManagedRepository/dunn0172admin_3/2018-01/10/11-15-14.918/
>>>>> directory?
>>>>>
>>>>> -- Mark
>>>>>
>>>>> The University of Dundee is a registered Scottish Charity, No: SC015096
>>>>> _______________________________________________
>>>>> ome-users mailing list
>>>>> ome-users at lists.openmicroscopy.org.uk
>>>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> ome-users mailing list
>>>> ome-users at lists.openmicroscopy.org.uk
>>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>>>>
>>>>
>>>
>>> The University of Dundee is a registered Scottish Charity, No: SC015096
>>>
>>> _______________________________________________
>>> ome-users mailing list
>>> ome-users at lists.openmicroscopy.org.uk
>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>>>
>>>
>>
>> _______________________________________________
>> ome-users mailing list
>> ome-users at lists.openmicroscopy.org.uk
>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>>
>>
>
> The University of Dundee is a registered Scottish Charity, No: SC015096
>
> _______________________________________________
> ome-users mailing list
> ome-users at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20180112/3c7f5254/attachment.html>
More information about the ome-users
mailing list