[ome-users] In-place import from CIFS share fails with permission error
Simon Li
spli at dundee.ac.uk
Mon Jan 15 13:24:21 GMT 2018
Hi Brian
Please accept my apologies. As you've figured out I should've said 0022 is the default and 0002 should be in the systemd service.
The fact that the symlink was created implies the permissions are correct. To try and track down the problem could you do an in-place import of a file on your local disk? If the SVS is too large to copy try a small image (e.g. tiff/png/jpg) instead.
Simon
On 12 January 2018 at 17:49, Brian Dunnette <dunn0172 at umn.edu<mailto:dunn0172 at umn.edu>> wrote:
Simon-
Thanks for the tip - however, when I add "UMask=0022" to the systemd file and reload the service, I get the same error:
[Service]
UMask=0022
[dunn0172 at nunstudy-omero system]$ sudo systemctl daemon-reload && sudo systemctl reload-or-restart omero-admin
[inplace_user at nunstudy-omero ~]$ omero import -- --transfer=ln_s /mnt/ahc-ns/Pathology/NunStudy/NS-1137/561.svs
...
11:36:07.268 [main] INFO o.f.i.transfers.SymlinkFileTransfer - Transferring /mnt/ahc-ns/Pathology/NunStudy/NS-1137/561.svs...
11:36:07.388 [main] INFO o.f.i.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /mnt/ahc-ns/Pathology/NunStudy/NS-1137/561.svs
11:36:07.422 [main] DEBUG o.f.i.cli.LoggingImportMonitor - FILE_EXCEPTION: /mnt/ahc-ns/Pathology/NunStudy/NS-1137/561.svs
11:36:07.422 [main] ERROR o.formats.importer.cli.ErrorHandler - FILE_EXCEPTION: /mnt/ahc-ns/Pathology/NunStudy/NS-1137/561.svs
java.lang.RuntimeException:
---------------------------------------------------
Check failed: /omero/ManagedRepository/dunn0172admin_3/2018-01/12/11-36-06.810/561.svs cannot be modified locally!
You likely do not have access to the ManagedRepository for in-place import.
Aborting...
[inplace_user at nunstudy-omero ~]$ ls -la /omero/ManagedRepository/dunn0172admin_3/2018-01/12/11-36-06.810/561.svs
-rw-r--r-- 1 omero inplace_user 36 Jan 12 11:36 /omero/ManagedRepository/dunn0172admin_3/2018-01/12/11-36-06.810/561.svs
However, when I change the umask to the 0002 mentioned in the inplace-import document, it seems to move things along some, creating the link and reading data (but still erroring later - maybe because the omero system user doesn't own the link?):
[Service]
UMask=0002
[dunn0172 at nunstudy-omero system]$ sudo systemctl daemon-reload && sudo systemctl reload-or-restart omero-admin
[inplace_user at nunstudy-omero ~]$ omero import -- --transfer=ln_s /mnt/ahc-ns/Pathology/NunStudy/NS-1137/561.svs
...
11:42:46.521 [main] INFO o.f.i.transfers.SymlinkFileTransfer - Transferring /mnt/ahc-ns/Pathology/NunStudy/NS-1137/561.svs...
11:42:46.778 [main] INFO o.f.i.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /mnt/ahc-ns/Pathology/NunStudy/NS-1137/561.svs
11:42:59.066 [main] DEBUG o.f.i.cli.LoggingImportMonitor - FILE_UPLOAD_BYTES uploaded: 342957409 of: 342957409 bytes (ETA: 0' 0")
11:42:59.105 [main] DEBUG o.f.i.cli.LoggingImportMonitor - FILE_EXCEPTION: /mnt/ahc-ns/Pathology/NunStudy/NS-1137/561.svs
11:42:59.105 [main] ERROR o.formats.importer.cli.ErrorHandler - FILE_EXCEPTION: /mnt/ahc-ns/Pathology/NunStudy/NS-1137/561.svs
omero.ResourceError: null
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_151]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_151]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_151]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_151]
at java.lang.Class.newInstance(Class.java:442) ~[na:1.8.0_151]
at IceInternal.BasicStream.createUserException(BasicStream.java:2779) ~[ice.jar:na]
at IceInternal.BasicStream.access$300(BasicStream.java:14) ~[ice.jar:na]
at IceInternal.BasicStream$EncapsDecoder10.throwException(BasicStream.java:3298) ~[ice.jar:na]
at IceInternal.BasicStream.throwException(BasicStream.java:2291) ~[ice.jar:na]
at IceInternal.OutgoingAsync.throwUserException(OutgoingAsync.java:399) ~[ice.jar:na]
at omero.api.RawFileStorePrxHelper.end_save(RawFileStorePrxHelper.java:724) ~[blitz.jar:na]
at omero.api.RawFileStorePrxHelper.save(RawFileStorePrxHelper.java:611) ~[blitz.jar:na]
at omero.api.RawFileStorePrxHelper.save(RawFileStorePrxHelper.java:598) ~[blitz.jar:na]
at ome.formats.importer.transfers.TransferState.save(TransferState.java:128) ~[blitz.jar:na]
at ome.formats.importer.transfers.AbstractFileTransfer.finish(AbstractFileTransfer.java:114) ~[blitz.jar:na]
at ome.formats.importer.transfers.AbstractExecFileTransfer.transfer(AbstractExecFileTransfer.java:70) ~[blitz.jar:na]
at ome.formats.importer.ImportLibrary.uploadFile(ImportLibrary.java:437) [blitz.jar:na]
at ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:511) [blitz.jar:na]
at ome.formats.importer.ImportLibrary.importCandidates(ImportLibrary.java:289) [blitz.jar:na]
at ome.formats.importer.cli.CommandLineImporter.start(CommandLineImporter.java:289) [blitz.jar:na]
at ome.formats.importer.cli.CommandLineImporter.main(CommandLineImporter.java:958) [blitz.jar:na]
11:42:59.110 [main] DEBUG o.f.i.cli.LoggingImportMonitor - FILE_UPLOAD_ERROR
11:42:59.110 [main] ERROR ome.formats.importer.ImportLibrary - Error on import: cannot flush /omero/ManagedRepository/dunn0172admin_3/2018-01/12/11-42-45.714/561.svs: java.io.IOException: Permission denied
omero.ResourceError: null
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_151]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_151]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_151]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_151]
at java.lang.Class.newInstance(Class.java:442) ~[na:1.8.0_151]
at IceInternal.BasicStream.createUserException(BasicStream.java:2779) ~[ice.jar:na]
at IceInternal.BasicStream.access$300(BasicStream.java:14) ~[ice.jar:na]
at IceInternal.BasicStream$EncapsDecoder10.throwException(BasicStream.java:3298) ~[ice.jar:na]
at IceInternal.BasicStream.throwException(BasicStream.java:2291) ~[ice.jar:na]
at IceInternal.OutgoingAsync.throwUserException(OutgoingAsync.java:399) ~[ice.jar:na]
at omero.api.RawFileStorePrxHelper.end_save(RawFileStorePrxHelper.java:724) ~[blitz.jar:na]
at omero.api.RawFileStorePrxHelper.save(RawFileStorePrxHelper.java:611) ~[blitz.jar:na]
at omero.api.RawFileStorePrxHelper.save(RawFileStorePrxHelper.java:598) ~[blitz.jar:na]
at ome.formats.importer.transfers.TransferState.save(TransferState.java:128) ~[blitz.jar:na]
at ome.formats.importer.transfers.AbstractFileTransfer.finish(AbstractFileTransfer.java:114) ~[blitz.jar:na]
at ome.formats.importer.transfers.AbstractExecFileTransfer.transfer(AbstractExecFileTransfer.java:70) ~[blitz.jar:na]
at ome.formats.importer.ImportLibrary.uploadFile(ImportLibrary.java:437) ~[blitz.jar:na]
at ome.formats.importer.ImportLibrary.importImage(ImportLibrary.java:511) ~[blitz.jar:na]
at ome.formats.importer.ImportLibrary.importCandidates(ImportLibrary.java:289) ~[blitz.jar:na]
at ome.formats.importer.cli.CommandLineImporter.start(CommandLineImporter.java:289) [blitz.jar:na]
at ome.formats.importer.cli.CommandLineImporter.main(CommandLineImporter.java:958) [blitz.jar:na]
11:42:59.110 [main] INFO ome.formats.importer.ImportLibrary - Exiting on error
==> Summary
0 files uploaded, 0 filesets created, 0 images imported, 1 error in 0:00:13.564
11:42:59.166 [main] DEBUG ome.formats.OMEROMetadataStoreClient - closing client session.
11:42:59.190 [main] DEBUG ome.formats.OMEROMetadataStoreClient - client closed.
11:42:59.190 [main] DEBUG ome.formats.OMEROMetadataStoreClient - Logout called, shutting keep alive down.
11:42:59.190 [main] DEBUG ome.formats.OMEROMetadataStoreClient - keepalive shut down.
!! 01/12/18 11:42:59.336 error: communicator not destroyed during global destruction.
[inplace_user at nunstudy-omero ~]$ ls -la /omero/ManagedRepository/dunn0172admin_3/2018-01/12/11-42-45.714/561.svs
lrwxrwxrwx 1 inplace_user inplace_user 46 Jan 12 11:42 /omero/ManagedRepository/dunn0172admin_3/2018-01/12/11-42-45.714/561.svs -> /mnt/ahc-ns/Pathology/NunStudy/NS-1137/561.svs
Thanks,
Brian Dunnette
Digital Imaging & Info Tech
Lab Medicine and Pathology<http://pathology.umn.edu>
University of Minnesota
dunn0172 at umn.edu<mailto:dunn0172 at umn.edu>
(612) 424-8187<tel:(612)%20424-8187> (Mobile)
(612) 624-6176<tel:(612)%20624-6176> (Office)
-----
The information transmitted in this e-mail is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material, including 'protected health information'. If you are not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please destroy and delete this message from any computer and contact us immediately by return e-mail.
On Fri, Jan 12, 2018 at 4:38 AM, Simon Li <spli at dundee.ac.uk<mailto:spli at dundee.ac.uk>> wrote:
Hi Brian
Systemd services default to a different umask (0002): https://www.freedesktop.org/software/systemd/man/systemd.exec.html#UMask=
If you add "UMask=0022" to the [Service] section of omero-systemd.service (note capitalisation is important), then run "systemctl daemon-reload" to update systemd, and finally restart OMERO, I think this will fix things.
I'll check with the rest of out team whether it's safe to update our docs to change the default.
Simon
On 11 January 2018 at 21:13, Brian Dunnette <dunn0172 at umn.edu<mailto:dunn0172 at umn.edu>> wrote:
Simon-
Here are the details (to the best of my knowledge) of our server:
* We're running Red Hat Enterprise v7.4
* OMERO was installed based on the CentOS 7 doc here: https://docs.openmicroscopy.org/omero/5.3.5/sysadmins/unix/server-centos7-ice36.html
* OMERO.admin is started via systemd, using the script from the docs (https://docs.openmicroscopy.org/omero/5.3.5/_downloads/omero-systemd.service)
Root, /home, /var, and /omero (the binary repository) are each their own XFS volumes, while the images we're trying to import/link are on an eNAS (mounted as the omero system user):
[dunn0172 at nunstudy-omero system]$ grep inplace /etc/passwd /etc/group
/etc/passwd:inplace_user:x:1003:1003::/home/inplace_user:/bin/bash
/etc/group:omero:x:1002:inplace_user
/etc/group:inplace_user:x:1003:
# /etc/fstab
UUID=e96e0f81-c366-4c30-bc2b-3a6f47d54d13 / xfs defaults 0 0
/dev/mapper/vg_os-home /home xfs defaults 0 0
/dev/mapper/vg_os-var /var xfs defaults 0 0
/dev/mapper/vg_os-swap swap swap defaults 0 0
/dev/mapper/vg_os-omero /omero xfs defaults 0 0
//files.umn.edu/ahc-ns<http://files.umn.edu/ahc-ns> /mnt/ahc-ns/ cifs credentials=/root/files.umn.edu.creds,dir_mode=0777,file_mode=0777,sec=ntlmssp,uid=1002,gid=1003 0 0
Thanks,
Brian Dunnette
Digital Imaging & Info Tech
Lab Medicine and Pathology<http://pathology.umn.edu>
University of Minnesota
dunn0172 at umn.edu<mailto:dunn0172 at umn.edu>
(612) 424-8187<tel:(612)%20424-8187> (Mobile)
(612) 624-6176<tel:(612)%20624-6176> (Office)
-----
The information transmitted in this e-mail is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material, including 'protected health information'. If you are not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please destroy and delete this message from any computer and contact us immediately by return e-mail.
On Thu, Jan 11, 2018 at 12:34 PM, Simon Li <spli at dundee.ac.uk<mailto:spli at dundee.ac.uk>> wrote:
Hi Brian
Could you give us some details about your OMERO system, including:
- Operating system and version
- How you installed OMERO (e.g. manually, scripted, instructions on one of our web-pages)
- How you're starting/stopping OMERO (e.g. manually, systemd, upstart)
- Disk volume mount options
Thanks, Simon
On 11 January 2018 at 17:54, Brian Dunnette <dunn0172 at umn.edu<mailto:dunn0172 at umn.edu>> wrote:
Mark-
It appears that the svs file is owned by the omero system user; however, though I ran the chmod/chgrp commands from the inplace-import document (and my umask is 0002), the permissions on new import directories and their files don't seem to be set correctly:
[omero at nunstudy-omero ~]$ umask
0002
[inplace_user at nunstudy-omero 11-43-02.765]$ umask
0002
# Change-group script
chgrp inplace_user /omero/ManagedRepository
chmod g+rws /omero/ManagedRepository
chmod g+rws /omero/ManagedRepository/*
chmod g+rws /omero/ManagedRepository/*/*
chmod g+rws /omero/ManagedRepository/*/*/*
chmod g+rws /omero/ManagedRepository/*/*/*/*
chgrp inplace_user /omero/ManagedRepository/*
chgrp inplace_user /omero/ManagedRepository/*/*
chgrp inplace_user /omero/ManagedRepository/*/*/*
chgrp inplace_user /omero/ManagedRepository/*/*/*/*
[inplace_user at nunstudy-omero 11]$ omero import -- --transfer=ln_s /mnt/ahc-ns/Pathology/NunStudy/NS-1137/562.svs
...
11:43:03.083 [main] INFO o.f.i.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /mnt/ahc-ns/Pathology/NunStudy/NS-1137/562.svs
11:43:03.105 [main] DEBUG o.f.i.cli.LoggingImportMonitor - FILE_EXCEPTION: /mnt/ahc-ns/Pathology/NunStudy/NS-1137/562.svs
11:43:03.106 [main] ERROR o.formats.importer.cli.ErrorHandler - FILE_EXCEPTION: /mnt/ahc-ns/Pathology/NunStudy/NS-1137/562.svs
java.lang.RuntimeException:
---------------------------------------------------
Check failed: /omero/ManagedRepository/dunn0172admin_3/2018-01/11/11-43-02.765/562.svs cannot be modified locally!
You likely do not have access to the ManagedRepository for in-place import.
Aborting...
[inplace_user at nunstudy-omero ~]$ ls -la /omero/ManagedRepository/dunn0172admin_3/2018-01/
total 0
drwxrwsr-x 4 omero inplace_user 26 Jan 11 11:40 .
drwxrwsr-x 3 omero inplace_user 21 Jan 10 11:22 ..
drwxrwsr-x 3 omero inplace_user 26 Jan 10 11:22 10
drwxr-sr-x 4 omero inplace_user 46 Jan 11 11:43 11
[inplace_user at nunstudy-omero ~]$ ls -la /omero/ManagedRepository/dunn0172admin_3/2018-01/11/
total 0
drwxr-sr-x 4 omero inplace_user 46 Jan 11 11:43 .
drwxrwsr-x 4 omero inplace_user 26 Jan 11 11:40 ..
drwxrwxr-x 2 omero inplace_user 21 Jan 11 11:40 11-40-49.349
drwxr-sr-x 2 omero inplace_user 21 Jan 11 11:43 11-43-02.765
[inplace_user at nunstudy-omero ~]$ ls -la /omero/ManagedRepository/dunn0172admin_3/2018-01/11/11-43-02.765/
total 4
drwxr-sr-x 2 omero inplace_user 21 Jan 11 11:43 .
drwxr-sr-x 4 omero inplace_user 46 Jan 11 11:43 ..
-rw-r--r-- 1 omero inplace_user 36 Jan 11 11:43 562.svs
Suggestions?
Thanks,
Brian Dunnette
Digital Imaging & Info Tech
Lab Medicine and Pathology<http://pathology.umn.edu>
University of Minnesota
dunn0172 at umn.edu<mailto:dunn0172 at umn.edu>
(612) 424-8187<tel:(612)%20424-8187> (Mobile)
(612) 624-6176<tel:(612)%20624-6176> (Office)
-----
The information transmitted in this e-mail is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material, including 'protected health information'. If you are not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please destroy and delete this message from any computer and contact us immediately by return e-mail.
On Thu, Jan 11, 2018 at 4:35 AM, Mark Carroll <m.t.b.carroll at dundee.ac.uk<mailto:m.t.b.carroll at dundee.ac.uk>> wrote:
Dear Brian,
We're attempting to do an in-place import of SVS files from a CIFS share
into our new OMERO 5.3.5 install, and followed the in-place import guide
(set up the inplace_user user & group, gave them ownership of the
ManagedRepository, verified that inplace_user can create a directory in
ManagedRepository and the omero system user can delete it).
Check failed:
/omero/ManagedRepository/dunn0172admin_3/2018-01/10/11-15-14.918/561.svs
cannot be modified locally!
You likely do not have access to the ManagedRepository for in-place
import.
Is this file owned by the omero system user? Is the inplace_user able to:
* append to it?
* delete it?
* create another file in that same
/omero/ManagedRepository/dunn0172admin_3/2018-01/10/11-15-14.918/ directory?
-- Mark
The University of Dundee is a registered Scottish Charity, No: SC015096
_______________________________________________
ome-users mailing list
ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
_______________________________________________
ome-users mailing list
ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
The University of Dundee is a registered Scottish Charity, No: SC015096
_______________________________________________
ome-users mailing list
ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
_______________________________________________
ome-users mailing list
ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
The University of Dundee is a registered Scottish Charity, No: SC015096
_______________________________________________
ome-users mailing list
ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
_______________________________________________
ome-users mailing list
ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
The University of Dundee is a registered Scottish Charity, No: SC015096
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20180115/f0788247/attachment.html>
More information about the ome-users
mailing list