[ome-devel] Plan for transferring data from one OMERO to another

TROUP Eilidh e.troup at epcc.ed.ac.uk
Fri Dec 22 15:47:33 GMT 2017 

Hi Will,

Thanks for this. I’ve been off sick and haven’t had time to consider this properly, but I think user-friendliness is going to be an issue, unless there’s a way to get hold of the session ID from the browser programatically.

Have a good holiday,
Eilidh

On 15 Dec 2017, at 16:45, William Moore (Staff) <W.Moore at dundee.ac.uk<mailto:W.Moore at dundee.ac.uk>> wrote:

Hi Eilidh,

  We are still discussing this, but one option we are exploring is to use an existing login via the JSON api in webclient that can give you a
session ID.

If you could go to your remote server’s /webclient/login/ page and open the browser dev-tools (right-click on the page and -> Inspect Element)
Then open the Console.
Then paste in the following command, editing the username and password to yours… (this is easier in Chrome than Firefox, since Firefox you have to type ‘allow pasting’ for security).

$.post('/api/v0/login/', {'username': ‘your_username', 'password’:'secret', 'server': 1}, data => console.log(data.eventContext.sessionUuid));

This should give you a session ID.
Then you can paste this into your script UI (in Insight) and use the code sample in my previous e-mail to join a session from the script.

NB: it’s important to close this session as soon as you’ve finished with it, in case someone got hold of the session ID.

Give this a try and see if it works for you.

There’s maybe other tests, checks and gotcha’s we need to consider, so we need more discussion before
this workflow gets the green light for production use.

But it would be good to know if this looks viable from your point of view?

 Cheers,

   Will.



On 15 Dec 2017, at 15:21, William Moore (Staff) <wmoore at dundee.ac.uk<mailto:wmoore at dundee.ac.uk>> wrote:

Hi Eilidh,

 Unfortunately, following more discussion in the wider team, we can’t recommend the entry of passwords into script parameters.
You can see some of the ongoing discussion at https://github.com/openmicroscopy/openmicroscopy/pull/5598 .

Fundamentally, the script service wasn’t designed to handle that level of security.
Everywhere we handle passwords in OMERO, we need to use a secure (encrypted) connection and ensure nothing is
logged, printed or stored anywhere.

We know that some DEBUG logging levels will log script parameters in OMERO.web (see above) and that OMERO.insight’s
connection to the server is not encrypted (except during login) and we just can’t guarantee that there aren’t other security loop-holes.

The support of a “Password” field in the script UI would give users the false impression that
their passwords are guaranteed to be safe, which we’d rather not do.

One suggestion was to use some other key to look-up passwords from some protected resource that was only available
to your script to read. Although I’m not sure whether this is viable or will scale for your use case?
This is included in the ongoing discussion on the PR above.

The safest way to do what you want is to use a Session key to “join” a current session on the server, although the webclient and Insight
don’t currently let users get hold of their session key.

If your users could use the command line to login to the remote server then you’d do something like this….

$ bin/omero login
Previous session expired for root on eel.openmicroscopy.org<http://eel.openmicroscopy.org/>:4064
Server: [eel.openmicroscopy.org<http://eel.openmicroscopy.org/>:4064]
Username: [root]user-3
Password:

$ bin/omero sessions key
b2c87a07-6728-4a0b-a9a0-1444cdd35ad7

They would then enter this key as a script parameter and you’d get a connection like this:

>>> from omero.gateway import BlitzGateway
>>> conn=BlitzGateway(host='eel.openmicroscopy.org<http://eel.openmicroscopy.org/>', port=4064)
>>> conn.connect('b2c87a07-6728-4a0b-a9a0-1444cdd35ad7’)
True


We’ll look at supporting session key in webclient but this won’t be in the next release I’m afraid.
So hopefully there’s some workaround you can use in the meantime.
Please let us know if there’s any other way we can help with this...

Regards,


  Will.



On 11 Dec 2017, at 14:33, Eilidh Troup <e.troup at epcc.ed.ac.uk<mailto:e.troup at epcc.ed.ac.uk>> wrote:

Thanks very much Will.

On 11 Dec 2017, at 13:37, William Moore (Staff) <W.Moore at dundee.ac.uk<mailto:W.Moore at dundee.ac.uk>> wrote:

Hi Eilidh,

I’ve opened at PR for this in web: https://github.com/openmicroscopy/openmicroscopy/pull/5598 and Jean-Marie tells me
that it should also be doable in Insight, so I’ve created a card at
https://trello.com/c/C3EqebXJ/61-script-parameter-password
These are likely to be included in the next release of OMERO.

Apart from the visibility of the password in the script UI, it’s important to be aware of other places that the password
could be exposed to others:

 - The connection between Insight and OMERO is not encrypted by default (except for the login). We’ll look at using encrypted connection when submitting script parameters containing passwords, but in the meantime there is the potential for this data to be read.
 - Make sure you’re not logging or printing the script params (as we do in many of our example scripts). Currently, Blitz.log and Processor.log don’t log script params but make sure you don’t log them yourself.
 - If you’re using the webclient, you should use https to encrypt the connection between the browser and OMERO.web.
 - Currently in webcilent, errors in the handling of script submission are not handled well (display error on page instead of displaying feedback error submission page). This shouldn’t lead to any exposure of passwords, but be careful not to submit feedback etc that contains the password and if you think you might have done, then be sure to change your password.


Regards,

  Will.


On 8 Dec 2017, at 11:55, Eilidh Troup <e.troup at epcc.ed.ac.uk<mailto:e.troup at epcc.ed.ac.uk>> wrote:

Hi Josh,

Thanks very much. This is a screenshot from running the script in OMERO.insight. I’d like the text entered in the password to be hidden, like this *****.

Eilidh

<PastedGraphic-2.png>

On 7 Dec 2017, at 14:39, Josh Moore <josh at glencoesoftware.com<mailto:josh at glencoesoftware.com>> wrote:

Hi Eilidh,

On Tue, Dec 5, 2017 at 10:51 PM, Eilidh Troup <e.troup at epcc.ed.ac.uk<mailto:e.troup at epcc.ed.ac.uk>> wrote:
Hi Josh,

Thanks for your help. I can now transfer an image from one OMERO to another
: )

Congratulations.


I’ve put the latest version of the script at
https://github.com/SynthSys/omero-user-scripts/blob/master/Export_to_other_omero.py

I’ve got more questions though:
How can I find the managed repository directory? (Something like
omero.managed.dir, Default: ${omero.data.dir}/ManagedRepository ?)

In [2]: client.sf.getConfigService().getConfigValue("omero.managed.dir")


How can I hide the password text entry?

Where are you seeing password text entry?


How can I get the newly created image id? It is printed to stdout as
"Image:920" for example, but I don't know how to get hold of that number in
my script.

You can use something like:

  import sys
  old_stdout = sys.stdout
  sys.stdout = open(temporary_file_name, "w")
  try:
      # do import
  finally:
      sys.stdout = old_stdout


Also, the docker script you sent didn’t quite work (don’t worry about it
though, I got what I needed from your example) - I got this error when I ran
the python script:

bash-4.2$ python /gist/export_to_remote_omero.py
Traceback (most recent call last):
...
::Glacier2::PermissionDeniedException
{
   reason = internal server error
}

Alright. If you'd like to investigate further, we'll need the server logs.


Thanks,
Eilidh

All the best,
~Josh


On 10 Nov 2017, at 09:22, Josh Moore <josh at glencoesoftware.com<mailto:josh at glencoesoftware.com>> wrote:

Hi Elidh,

On Thu, Nov 9, 2017 at 5:07 PM, Eilidh Troup <e.troup at epcc.ed.ac.uk<mailto:e.troup at epcc.ed.ac.uk>> wrote:

Hi,

I’ve run into a similar problem as last time.


Turns out, there's a similar solution.

I’ve written a script that
will upload an image to a remote OMERO server, but only from the python
command line, and not from within a script running in a local OMERO. As
before, the script will work from the command line or from with OMERO when
transferring the image to the local OMERO server. When I try to import an
image on the remote server, I get the following error:

...


The code, and full error output are below.
I’ve also put it into a different script on gist that demonstrates the error
by running it from within OMERO as a script.
https://gist.github.com/eilidh-t/e763dab7d4f3515cc85b0b6ddfb8d9f4


I've update your script:
https://gist.github.com/joshmoore/0c2e5319d8d5ae7f79ee82e3e68b721b

The most important change is:

  del os.environ["ICE_CONFIG"] before importing

which does the equivalent of `Ice.Config=/dev/null` but for the import
process. Likely, that should be done closer to the top.

Other changes include:

* use ome.cli.CLI rather than calling subprocess (simplicity)
* only register the script if it doesn't exist (less test mess)
* pass a key to not store the root's password (security)
* adding a warning about the security whole that this script could
cause (security)

My general workflow was:

* docker-compose up -d
* docker-compose exec omero1 bash
* cd /opt/omero/server/OMERO.server
* export PYTHONPATH=lib/python
* python /gist/export_to_remote_omero.py

If I needed to change the script, I deleted the uploaded file from
lib/script/export_to_remote_omero.py and re-ran.

Thanks for your help with this,
Eilidh



Let us know if that works for you.
~J
_______________________________________________
ome-devel mailing list
ome-devel at lists.openmicroscopy.org.uk<mailto:ome-devel at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel



The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

_______________________________________________
ome-devel mailing list
ome-devel at lists.openmicroscopy.org.uk<mailto:ome-devel at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel

_______________________________________________
ome-devel mailing list
ome-devel at lists.openmicroscopy.org.uk<mailto:ome-devel at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
_______________________________________________
ome-devel mailing list
ome-devel at lists.openmicroscopy.org.uk<mailto:ome-devel at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel


The University of Dundee is a registered Scottish Charity, No: SC015096
_______________________________________________
ome-devel mailing list
ome-devel at lists.openmicroscopy.org.uk<mailto:ome-devel at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel

The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
_______________________________________________
ome-devel mailing list
ome-devel at lists.openmicroscopy.org.uk<mailto:ome-devel at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel



The University of Dundee is a registered Scottish Charity, No: SC015096
_______________________________________________
ome-devel mailing list
ome-devel at lists.openmicroscopy.org.uk<mailto:ome-devel at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20171222/da80c927/attachment.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: not available
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20171222/da80c927/attachment.ksh>

More information about the ome-devel mailing list