[ome-devel] Plan for transferring data from one OMERO to another
Eilidh Troup
e.troup at epcc.ed.ac.uk
Thu Jan 4 11:04:34 GMT 2018
Hi,
Thanks for your effort with this. I looked into storing the password on the user's computer and I’m thinking of using python keyring <https://pypi.python.org/pypi/keyring#installing-and-using-python-keyring-lib> to store the login to the remote OMERO server.
Install:
pip install keyring
User sets password using this command on the command line:
keyring set omero username
Then in the OMERO script the user would only have to enter their username, and the script could look up their password as follows:
python
>>> import keyring
>>> keyring.get_password("omero","username")
u'omero_password'
This would work, and be fairly convenient, but the security seems to be that if you're logged into the computer where you entered your password then it's accessible. I'll need to discuss this with my team. For example, on the command line:
% keyring set omero username
Password for 'username' in 'omero':
% keyring get omero username
omero_password
> On 22 Dec 2017, at 15:47, TROUP Eilidh <e.troup at epcc.ed.ac.uk <mailto:e.troup at epcc.ed.ac.uk>> wrote:
>
> Hi Will,
>
> Thanks for this. I’ve been off sick and haven’t had time to consider this properly, but I think user-friendliness is going to be an issue, unless there’s a way to get hold of the session ID from the browser programatically.
>
> Have a good holiday,
> Eilidh
>
>> On 15 Dec 2017, at 16:45, William Moore (Staff) <W.Moore at dundee.ac.uk <mailto:W.Moore at dundee.ac.uk>> wrote:
>>
>> Hi Eilidh,
>>
>> We are still discussing this, but one option we are exploring is to use an existing login via the JSON api in webclient that can give you a
>> session ID.
>>
>> If you could go to your remote server’s /webclient/login/ page and open the browser dev-tools (right-click on the page and -> Inspect Element)
>> Then open the Console.
>> Then paste in the following command, editing the username and password to yours… (this is easier in Chrome than Firefox, since Firefox you have to type ‘allow pasting’ for security).
>>
>> $.post('/api/v0/login/', {'username': ‘your_username', 'password’:'secret', 'server': 1}, data => console.log(data.eventContext.sessionUuid));
>>
>> This should give you a session ID.
>> Then you can paste this into your script UI (in Insight) and use the code sample in my previous e-mail to join a session from the script.
>>
>> NB: it’s important to close this session as soon as you’ve finished with it, in case someone got hold of the session ID.
>>
>> Give this a try and see if it works for you.
>>
>> There’s maybe other tests, checks and gotcha’s we need to consider, so we need more discussion before
>> this workflow gets the green light for production use.
>>
>> But it would be good to know if this looks viable from your point of view?
>>
>> Cheers,
>>
>> Will.
>>
>>
>>
>>> On 15 Dec 2017, at 15:21, William Moore (Staff) <wmoore at dundee.ac.uk <mailto:wmoore at dundee.ac.uk>> wrote:
>>>
>>> Hi Eilidh,
>>>
>>> Unfortunately, following more discussion in the wider team, we can’t recommend the entry of passwords into script parameters.
>>> You can see some of the ongoing discussion at https://github.com/openmicroscopy/openmicroscopy/pull/5598 <https://github.com/openmicroscopy/openmicroscopy/pull/5598> .
>>>
>>> Fundamentally, the script service wasn’t designed to handle that level of security.
>>> Everywhere we handle passwords in OMERO, we need to use a secure (encrypted) connection and ensure nothing is
>>> logged, printed or stored anywhere.
>>>
>>> We know that some DEBUG logging levels will log script parameters in OMERO.web (see above) and that OMERO.insight’s
>>> connection to the server is not encrypted (except during login) and we just can’t guarantee that there aren’t other security loop-holes.
>>>
>>> The support of a “Password” field in the script UI would give users the false impression that
>>> their passwords are guaranteed to be safe, which we’d rather not do.
>>>
>>> One suggestion was to use some other key to look-up passwords from some protected resource that was only available
>>> to your script to read. Although I’m not sure whether this is viable or will scale for your use case?
>>> This is included in the ongoing discussion on the PR above.
>>>
>>> The safest way to do what you want is to use a Session key to “join” a current session on the server, although the webclient and Insight
>>> don’t currently let users get hold of their session key.
>>>
>>> If your users could use the command line to login to the remote server then you’d do something like this….
>>>
>>> $ bin/omero login
>>> Previous session expired for root on eel.openmicroscopy.org <http://eel.openmicroscopy.org/>:4064
>>> Server: [eel.openmicroscopy.org <http://eel.openmicroscopy.org/>:4064]
>>> Username: [root]user-3
>>> Password:
>>>
>>> $ bin/omero sessions key
>>> b2c87a07-6728-4a0b-a9a0-1444cdd35ad7
>>>
>>> They would then enter this key as a script parameter and you’d get a connection like this:
>>>
>>> >>> from omero.gateway import BlitzGateway
>>> >>> conn=BlitzGateway(host='eel.openmicroscopy.org <http://eel.openmicroscopy.org/>', port=4064)
>>> >>> conn.connect('b2c87a07-6728-4a0b-a9a0-1444cdd35ad7’)
>>> True
>>>
>>>
>>> We’ll look at supporting session key in webclient but this won’t be in the next release I’m afraid.
>>> So hopefully there’s some workaround you can use in the meantime.
>>> Please let us know if there’s any other way we can help with this...
>>>
>>> Regards,
>>>
>>>
>>> Will.
>>>
>>>
>>>
>>>> On 11 Dec 2017, at 14:33, Eilidh Troup <e.troup at epcc.ed.ac.uk <mailto:e.troup at epcc.ed.ac.uk>> wrote:
>>>>
>>>> Thanks very much Will.
>>>>
>>>>> On 11 Dec 2017, at 13:37, William Moore (Staff) <W.Moore at dundee.ac.uk <mailto:W.Moore at dundee.ac.uk>> wrote:
>>>>>
>>>>> Hi Eilidh,
>>>>>
>>>>> I’ve opened at PR for this in web: https://github.com/openmicroscopy/openmicroscopy/pull/5598 <https://github.com/openmicroscopy/openmicroscopy/pull/5598> and Jean-Marie tells me
>>>>> that it should also be doable in Insight, so I’ve created a card at
>>>>> https://trello.com/c/C3EqebXJ/61-script-parameter-password <https://trello.com/c/C3EqebXJ/61-script-parameter-password>
>>>>> These are likely to be included in the next release of OMERO.
>>>>>
>>>>> Apart from the visibility of the password in the script UI, it’s important to be aware of other places that the password
>>>>> could be exposed to others:
>>>>>
>>>>> - The connection between Insight and OMERO is not encrypted by default (except for the login). We’ll look at using encrypted connection when submitting script parameters containing passwords, but in the meantime there is the potential for this data to be read.
>>>>> - Make sure you’re not logging or printing the script params (as we do in many of our example scripts). Currently, Blitz.log and Processor.log don’t log script params but make sure you don’t log them yourself.
>>>>> - If you’re using the webclient, you should use https to encrypt the connection between the browser and OMERO.web.
>>>>> - Currently in webcilent, errors in the handling of script submission are not handled well (display error on page instead of displaying feedback error submission page). This shouldn’t lead to any exposure of passwords, but be careful not to submit feedback etc that contains the password and if you think you might have done, then be sure to change your password.
>>>>>
>>>>>
>>>>> Regards,
>>>>>
>>>>> Will.
>>>>>
>>>>>
>>>>>> On 8 Dec 2017, at 11:55, Eilidh Troup <e.troup at epcc.ed.ac.uk <mailto:e.troup at epcc.ed.ac.uk>> wrote:
>>>>>>
>>>>>> Hi Josh,
>>>>>>
>>>>>> Thanks very much. This is a screenshot from running the script in OMERO.insight. I’d like the text entered in the password to be hidden, like this *****.
>>>>>>
>>>>>> Eilidh
>>>>>>
>>>>>> <PastedGraphic-2.png>
>>>>>>
>>>>>>> On 7 Dec 2017, at 14:39, Josh Moore <josh at glencoesoftware.com <mailto:josh at glencoesoftware.com>> wrote:
>>>>>>>
>>>>>>> Hi Eilidh,
>>>>>>>
>>>>>>> On Tue, Dec 5, 2017 at 10:51 PM, Eilidh Troup <e.troup at epcc.ed.ac.uk <mailto:e.troup at epcc.ed.ac.uk>> wrote:
>>>>>>>> Hi Josh,
>>>>>>>>
>>>>>>>> Thanks for your help. I can now transfer an image from one OMERO to another
>>>>>>>> : )
>>>>>>>
>>>>>>> Congratulations.
>>>>>>>
>>>>>>>
>>>>>>>> I’ve put the latest version of the script at
>>>>>>>> https://github.com/SynthSys/omero-user-scripts/blob/master/Export_to_other_omero.py <https://github.com/SynthSys/omero-user-scripts/blob/master/Export_to_other_omero.py>
>>>>>>>>
>>>>>>>> I’ve got more questions though:
>>>>>>>> How can I find the managed repository directory? (Something like
>>>>>>>> omero.managed.dir, Default: ${omero.data.dir}/ManagedRepository ?)
>>>>>>>
>>>>>>> In [2]: client.sf.getConfigService().getConfigValue("omero.managed.dir")
>>>>>>>
>>>>>>>
>>>>>>>> How can I hide the password text entry?
>>>>>>>
>>>>>>> Where are you seeing password text entry?
>>>>>>>
>>>>>>>
>>>>>>>> How can I get the newly created image id? It is printed to stdout as
>>>>>>>> "Image:920" for example, but I don't know how to get hold of that number in
>>>>>>>> my script.
>>>>>>>
>>>>>>> You can use something like:
>>>>>>>
>>>>>>> import sys
>>>>>>> old_stdout = sys.stdout
>>>>>>> sys.stdout = open(temporary_file_name, "w")
>>>>>>> try:
>>>>>>> # do import
>>>>>>> finally:
>>>>>>> sys.stdout = old_stdout
>>>>>>>
>>>>>>>
>>>>>>>> Also, the docker script you sent didn’t quite work (don’t worry about it
>>>>>>>> though, I got what I needed from your example) - I got this error when I ran
>>>>>>>> the python script:
>>>>>>>>
>>>>>>>> bash-4.2$ python /gist/export_to_remote_omero.py
>>>>>>>> Traceback (most recent call last):
>>>>>>> ...
>>>>>>>> ::Glacier2::PermissionDeniedException
>>>>>>>> {
>>>>>>>> reason = internal server error
>>>>>>>> }
>>>>>>>
>>>>>>> Alright. If you'd like to investigate further, we'll need the server logs.
>>>>>>>
>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Eilidh
>>>>>>>
>>>>>>> All the best,
>>>>>>> ~Josh
>>>>>>>
>>>>>>>
>>>>>>>> On 10 Nov 2017, at 09:22, Josh Moore <josh at glencoesoftware.com <mailto:josh at glencoesoftware.com>> wrote:
>>>>>>>>
>>>>>>>> Hi Elidh,
>>>>>>>>
>>>>>>>> On Thu, Nov 9, 2017 at 5:07 PM, Eilidh Troup <e.troup at epcc.ed.ac.uk <mailto:e.troup at epcc.ed.ac.uk>> wrote:
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I’ve run into a similar problem as last time.
>>>>>>>>
>>>>>>>>
>>>>>>>> Turns out, there's a similar solution.
>>>>>>>>
>>>>>>>> I’ve written a script that
>>>>>>>> will upload an image to a remote OMERO server, but only from the python
>>>>>>>> command line, and not from within a script running in a local OMERO. As
>>>>>>>> before, the script will work from the command line or from with OMERO when
>>>>>>>> transferring the image to the local OMERO server. When I try to import an
>>>>>>>> image on the remote server, I get the following error:
>>>>>>>>
>>>>>>>> ...
>>>>>>>>
>>>>>>>>
>>>>>>>> The code, and full error output are below.
>>>>>>>> I’ve also put it into a different script on gist that demonstrates the error
>>>>>>>> by running it from within OMERO as a script.
>>>>>>>> https://gist.github.com/eilidh-t/e763dab7d4f3515cc85b0b6ddfb8d9f4 <https://gist.github.com/eilidh-t/e763dab7d4f3515cc85b0b6ddfb8d9f4>
>>>>>>>>
>>>>>>>>
>>>>>>>> I've update your script:
>>>>>>>> https://gist.github.com/joshmoore/0c2e5319d8d5ae7f79ee82e3e68b721b <https://gist.github.com/joshmoore/0c2e5319d8d5ae7f79ee82e3e68b721b>
>>>>>>>>
>>>>>>>> The most important change is:
>>>>>>>>
>>>>>>>> del os.environ["ICE_CONFIG"] before importing
>>>>>>>>
>>>>>>>> which does the equivalent of `Ice.Config=/dev/null` but for the import
>>>>>>>> process. Likely, that should be done closer to the top.
>>>>>>>>
>>>>>>>> Other changes include:
>>>>>>>>
>>>>>>>> * use ome.cli.CLI rather than calling subprocess (simplicity)
>>>>>>>> * only register the script if it doesn't exist (less test mess)
>>>>>>>> * pass a key to not store the root's password (security)
>>>>>>>> * adding a warning about the security whole that this script could
>>>>>>>> cause (security)
>>>>>>>>
>>>>>>>> My general workflow was:
>>>>>>>>
>>>>>>>> * docker-compose up -d
>>>>>>>> * docker-compose exec omero1 bash
>>>>>>>> * cd /opt/omero/server/OMERO.server
>>>>>>>> * export PYTHONPATH=lib/python
>>>>>>>> * python /gist/export_to_remote_omero.py
>>>>>>>>
>>>>>>>> If I needed to change the script, I deleted the uploaded file from
>>>>>>>> lib/script/export_to_remote_omero.py and re-ran.
>>>>>>>>
>>>>>>>> Thanks for your help with this,
>>>>>>>> Eilidh
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Let us know if that works for you.
>>>>>>>> ~J
>>>>>>>> _______________________________________________
>>>>>>>> ome-devel mailing list
>>>>>>>> ome-devel at lists.openmicroscopy.org.uk <mailto:ome-devel at lists.openmicroscopy.org.uk>
>>>>>>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel <http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> The University of Edinburgh is a charitable body, registered in
>>>>>>>> Scotland, with registration number SC005336.
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> ome-devel mailing list
>>>>>>>> ome-devel at lists.openmicroscopy.org.uk <mailto:ome-devel at lists.openmicroscopy.org.uk>
>>>>>>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel <http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel>
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> ome-devel mailing list
>>>>>>> ome-devel at lists.openmicroscopy.org.uk <mailto:ome-devel at lists.openmicroscopy.org.uk>
>>>>>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel <http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel>
>>>>>> The University of Edinburgh is a charitable body, registered in
>>>>>> Scotland, with registration number SC005336.
>>>>>> _______________________________________________
>>>>>> ome-devel mailing list
>>>>>> ome-devel at lists.openmicroscopy.org.uk <mailto:ome-devel at lists.openmicroscopy.org.uk>
>>>>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel <http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel>
>>>>>
>>>>>
>>>>> The University of Dundee is a registered Scottish Charity, No: SC015096
>>>>> _______________________________________________
>>>>> ome-devel mailing list
>>>>> ome-devel at lists.openmicroscopy.org.uk <mailto:ome-devel at lists.openmicroscopy.org.uk>
>>>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel <http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel>
>>>>
>>>> The University of Edinburgh is a charitable body, registered in
>>>> Scotland, with registration number SC005336.
>>>> _______________________________________________
>>>> ome-devel mailing list
>>>> ome-devel at lists.openmicroscopy.org.uk <mailto:ome-devel at lists.openmicroscopy.org.uk>
>>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel <http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel>
>>>
>>
>>
>> The University of Dundee is a registered Scottish Charity, No: SC015096
>> _______________________________________________
>> ome-devel mailing list
>> ome-devel at lists.openmicroscopy.org.uk <mailto:ome-devel at lists.openmicroscopy.org.uk>
>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel <http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel>
>
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk <mailto:ome-devel at lists.openmicroscopy.org.uk>
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20180104/507d93bb/attachment.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: not available
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20180104/507d93bb/attachment.ksh>
More information about the ome-devel
mailing list