[ome-devel] Plan for transferring data from one OMERO to another

Fri Dec 15 19:13:14 GMT 2017

Hi guys,

If running an additional service is an option, then you could try using
Smuggler:

   - http://c0c0n3.github.io/ome-smuggler/

It comes with a JSON API that lets you create an OMERO session:

*
http://c0c0n3.github.io/ome-smuggler/javadoc/server/ome/smuggler/web/omero/CreateSessionRequest.html

You can use the same Smuggler instance to create sessions with different
OMERO servers and optionally specify a session keep-alive duration. If you
specify a keep-alive of D minutes/hours/etc. Smuggler will take care of
pinging OMERO to keep that session alive for D minutes/hrs/etc. This may
come in handy if you want to run a script multiple times with the same
session key without having to worry about OMERO expiring the session
between runs.

You can find an example of creating a session with curl in this directory:

*
https://github.com/c0c0n3/omero-ms-queue/tree/master/components/server/src/test/scripts/http-omero

I'll gladly provide more details if you think this is a good fit for your
scenario :)

Cheers

/a

On 15 December 2017 at 17:45, William Moore (Staff) <W.Moore at dundee.ac.uk>
wrote:

> Hi Eilidh,
>
>   We are still discussing this, but one option we are exploring is to use
> an existing login via the JSON api in webclient that can give you a
> session ID.
>
> If you could go to your remote server’s /webclient/login/ page and open
> the browser dev-tools (right-click on the page and -> Inspect Element)
> Then open the Console.
> Then paste in the following command, editing the username and password to
> yours… (this is easier in Chrome than Firefox, since Firefox you have to
> type ‘allow pasting’ for security).
>
> $.post('/api/v0/login/', {'username': ‘your_username',
> 'password’:'secret', 'server': 1}, data => console.log(data.eventContext.
> sessionUuid));
>
> This should give you a session ID.
> Then you can paste this into your script UI (in Insight) and use the code
> sample in my previous e-mail to join a session from the script.
>
> NB: it’s important to close this session as soon as you’ve finished with
> it, in case someone got hold of the session ID.
>
> Give this a try and see if it works for you.
>
> There’s maybe other tests, checks and gotcha’s we need to consider, so we
> need more discussion before
> this workflow gets the green light for production use.
>
> But it would be good to know if this looks viable from your point of view?
>
>  Cheers,
>
>    Will.
>
>
>
> On 15 Dec 2017, at 15:21, William Moore (Staff) <wmoore at dundee.ac.uk>
> wrote:
>
> Hi Eilidh,
>
>  Unfortunately, following more discussion in the wider team, we can’t
> recommend the entry of passwords into script parameters.
> You can see some of the ongoing discussion at https://github.com/
> openmicroscopy/openmicroscopy/pull/5598 .
>
> Fundamentally, the script service wasn’t designed to handle that level of
> security.
> Everywhere we handle passwords in OMERO, we need to use a secure
> (encrypted) connection and ensure nothing is
> logged, printed or stored anywhere.
>
> We know that some DEBUG logging levels will log script parameters in
> OMERO.web (see above) and that OMERO.insight’s
> connection to the server is not encrypted (except during login) and we
> just can’t guarantee that there aren’t other security loop-holes.
>
> The support of a “Password” field in the script UI would give users the
> false impression that
> their passwords are guaranteed to be safe, which we’d rather not do.
>
> One suggestion was to use some other key to look-up passwords from some
> protected resource that was only available
> to your script to read. Although I’m not sure whether this is viable or
> will scale for your use case?
> This is included in the ongoing discussion on the PR above.
>
> The safest way to do what you want is to use a Session key to “join” a
> current session on the server, although the webclient and Insight
> don’t currently let users get hold of their session key.
>
> If your users could use the command line to login to the remote server
> then you’d do something like this….
>
> $ bin/omero login
> Previous session expired for root on eel.openmicroscopy.org:4064
> Server: [eel.openmicroscopy.org:4064]
> Username: [root]user-3
> Password:
>
> $ bin/omero sessions key
> b2c87a07-6728-4a0b-a9a0-1444cdd35ad7
>
> They would then enter this key as a script parameter and you’d get a
> connection like this:
>
> >>> from omero.gateway import BlitzGateway
> >>> conn=BlitzGateway(host='eel.openmicroscopy.org', port=4064)
> >>> conn.connect('b2c87a07-6728-4a0b-a9a0-1444cdd35ad7’)
> True
>
>
> We’ll look at supporting session key in webclient but this won’t be in the
> next release I’m afraid.
> So hopefully there’s some workaround you can use in the meantime.
> Please let us know if there’s any other way we can help with this...
>
> Regards,
>
>
>   Will.
>
>
>
> On 11 Dec 2017, at 14:33, Eilidh Troup <e.troup at epcc.ed.ac.uk> wrote:
>
> Thanks very much Will.
>
> On 11 Dec 2017, at 13:37, William Moore (Staff) <W.Moore at dundee.ac.uk>
> wrote:
>
> Hi Eilidh,
>
> I’ve opened at PR for this in web: https://github.com/
> openmicroscopy/openmicroscopy/pull/5598 and Jean-Marie tells me
> that it should also be doable in Insight, so I’ve created a card at
> https://trello.com/c/C3EqebXJ/61-script-parameter-password
> These are likely to be included in the next release of OMERO.
>
> Apart from the visibility of the password in the script UI, it’s important
> to be aware of other places that the password
> could be exposed to others:
>
>  - The connection between Insight and OMERO is not encrypted by default
> (except for the login). We’ll look at using encrypted connection when
> submitting script parameters containing passwords, but in the meantime
> there is the potential for this data to be read.
>  - Make sure you’re not logging or printing the script params (as we do in
> many of our example scripts). Currently, Blitz.log and Processor.log don’t
> log script params but make sure you don’t log them yourself.
>  - If you’re using the webclient, you should use https to encrypt the
> connection between the browser and OMERO.web.
>  - Currently in webcilent, errors in the handling of script submission are
> not handled well (display error on page instead of displaying feedback
> error submission page). This shouldn’t lead to any exposure of passwords,
> but be careful not to submit feedback etc that contains the password and if
> you think you might have done, then be sure to change your password.
>
>
> Regards,
>
>   Will.
>
>
> On 8 Dec 2017, at 11:55, Eilidh Troup <e.troup at epcc.ed.ac.uk> wrote:
>
> Hi Josh,
>
> Thanks very much. This is a screenshot from running the script in
> OMERO.insight. I’d like the text entered in the password to be hidden, like
> this *****.
>
> Eilidh
>
> <PastedGraphic-2.png>
>
> On 7 Dec 2017, at 14:39, Josh Moore <josh at glencoesoftware.com> wrote:
>
> Hi Eilidh,
>
> On Tue, Dec 5, 2017 at 10:51 PM, Eilidh Troup <e.troup at epcc.ed.ac.uk>
> wrote:
>
> Hi Josh,
>
> Thanks for your help. I can now transfer an image from one OMERO to another
> : )
>
>
> Congratulations.
>
>
> I’ve put the latest version of the script at
> https://github.com/SynthSys/omero-user-scripts/blob/
> master/Export_to_other_omero.py
>
> I’ve got more questions though:
> How can I find the managed repository directory? (Something like
> omero.managed.dir, Default: ${omero.data.dir}/ManagedRepository ?)
>
>
> In [2]: client.sf.getConfigService().getConfigValue("omero.managed.dir")
>
>
> How can I hide the password text entry?
>
>
> Where are you seeing password text entry?
>
>
> How can I get the newly created image id? It is printed to stdout as
> "Image:920" for example, but I don't know how to get hold of that number in
> my script.
>
>
> You can use something like:
>
>   import sys
>   old_stdout = sys.stdout
>   sys.stdout = open(temporary_file_name, "w")
>   try:
>       # do import
>   finally:
>       sys.stdout = old_stdout
>
>
> Also, the docker script you sent didn’t quite work (don’t worry about it
> though, I got what I needed from your example) - I got this error when I
> ran
> the python script:
>
> bash-4.2$ python /gist/export_to_remote_omero.py
> Traceback (most recent call last):
>
> ...
>
> ::Glacier2::PermissionDeniedException
> {
>    reason = internal server error
> }
>
>
> Alright. If you'd like to investigate further, we'll need the server logs.
>
>
> Thanks,
> Eilidh
>
>
> All the best,
> ~Josh
>
>
> On 10 Nov 2017, at 09:22, Josh Moore <josh at glencoesoftware.com> wrote:
>
> Hi Elidh,
>
> On Thu, Nov 9, 2017 at 5:07 PM, Eilidh Troup <e.troup at epcc.ed.ac.uk>
> wrote:
>
> Hi,
>
> I’ve run into a similar problem as last time.
>
>
> Turns out, there's a similar solution.
>
> I’ve written a script that
> will upload an image to a remote OMERO server, but only from the python
> command line, and not from within a script running in a local OMERO. As
> before, the script will work from the command line or from with OMERO when
> transferring the image to the local OMERO server. When I try to import an
> image on the remote server, I get the following error:
>
> ...
>
>
> The code, and full error output are below.
> I’ve also put it into a different script on gist that demonstrates the
> error
> by running it from within OMERO as a script.
> https://gist.github.com/eilidh-t/e763dab7d4f3515cc85b0b6ddfb8d9f4
>
>
> I've update your script:
> https://gist.github.com/joshmoore/0c2e5319d8d5ae7f79ee82e3e68b721b
>
> The most important change is:
>
>   del os.environ["ICE_CONFIG"] before importing
>
> which does the equivalent of `Ice.Config=/dev/null` but for the import
> process. Likely, that should be done closer to the top.
>
> Other changes include:
>
> * use ome.cli.CLI rather than calling subprocess (simplicity)
> * only register the script if it doesn't exist (less test mess)
> * pass a key to not store the root's password (security)
> * adding a warning about the security whole that this script could
> cause (security)
>
> My general workflow was:
>
> * docker-compose up -d
> * docker-compose exec omero1 bash
> * cd /opt/omero/server/OMERO.server
> * export PYTHONPATH=lib/python
> * python /gist/export_to_remote_omero.py
>
> If I needed to change the script, I deleted the uploaded file from
> lib/script/export_to_remote_omero.py and re-ran.
>
> Thanks for your help with this,
> Eilidh
>
>
>
> Let us know if that works for you.
> ~J
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>
>
>
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>
>
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>
>
>
> The University of Dundee is a registered Scottish Charity, No: SC015096
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>
>
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>
>
>
>
> The University of Dundee is a registered Scottish Charity, No: SC015096
>
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20171215/1b4e55aa/attachment.html>