[ome-devel] OME python binding

Colin Blackburn C.Blackburn at dundee.ac.uk
Wed Oct 1 17:30:24 BST 2014 

Hi Yanling,

Again, we are planning to release a transfer mode that uploads, imports and then deletes the original files. It would do this only on a successful upload (all files checked against their sha1s) and a successful import (completes without error). This transfer mode could then be used for DropBox to effectively clean the DropBox directories of files once imported.

This feature is intended to be available in the 5.1 release. If you would like to test the this feature prior to that release please do let us know.

Cheers,

Colin

On 1 Oct 2014, at 16:54, Yanling Liu <vrnova at gmail.com> wrote:

> This is fabulous!
>
> Then the only thing I need to work on is to setup a cron job to delete old files from the DropBox folder.
>
> Can I dream about DropBox facility to be able to auto-delete imported files? I know there's in-place auto-delete but our data is 20TB+ on NFS storage and OME doesn't support NFS.
>
> Cheers,
>
> Yanling
>
> On Wed, Oct 1, 2014 at 11:40 AM, Colin Blackburn <C.Blackburn at dundee.ac.uk> wrote:
> Hi Yanling,
>
> We are planning to implement a DropBox facility for selecting the destination container for imports in our next major release, 5.1.
>
> Cheers,
>
> Colin
>
>
> On 30 Sep 2014, at 19:33, Yanling Liu <vrnova at gmail.com> wrote:
>
> > Tested Dropbox and it works very well.
> >
> > Just wondering when will the extended path /OMERO/DropBox/user/group/project/dataset being implemented? This is very very useful to make OME "useful" for imaging facilities where imaging facilities upload images for users instead of users themselves, which is exactly the case we have.
> >
> > Thanks,
> >
> > Yanling
> >
> > On Fri, Sep 26, 2014 at 9:13 AM, Aleksandra Tarkowska <A.Tarkowska at dundee.ac.uk> wrote:
> > Hi Yanling
> >
> > > Dropbox documentation says NAS is not supported, is a NFS mount considered a network attached share (NAS) to OMERO?
> >
> > Dropbox can monitor only local hard drive. But you can set up samba dir locally to drop files and set configuration parameters like: "--transfer=ln_rm" or "--transfer=upload_rm" to delete when they are imported. If you prefer to use NFS you can use In-place import https://www.openmicroscopy.org/site/support/omero5/sysadmins/in-place-import.html. That should simplify your workflow.
> >
> > > Dropbox import new images under "Orphaned Images", then user has to login to organise new images into projects and datasets, is that correct?
> >
> > That is correct. Extended path /OMERO/DropBox/user/group/project/dataset/ is on our list to do http://trac.openmicroscopy.org.uk/ome/ticket/4032 as well as templates http://trac.openmicroscopy.org.uk/ome/ticket/12039
> >
> > You can customise name of  "Orphaned Images" if is not suitable.
> >
> > Kind regards
> > Ola
> >
> > On Fri, Sep 26, 2014 at 8:38 AM, Aleksandra Tarkowska <A.Tarkowska at dundee.ac.uk> wrote:
> > Hi Yanling
> >
> > As Carne mentioned you can invoke import process via python script.
> > Similar test is available on
> > https://github.com/openmicroscopy/openmicroscopy/blob/develop/components/to
> > ols/OmeroPy/test/integration/library.py#L187
> >
> > args.extend(["-s", server, "-k", key, "-p", port, "import", "--"])
> >
> > Allows you to pass all parameters you normally pass via bin/omero import Š
> > For more details refer to --help.
> >
> > Can I ask why you just simply cannot use dropbox to do that work
> > automatically for you?
> > Did you see
> > http://www.openmicroscopy.org/site/support/omero5/sysadmins/dropbox.html
> >
> > Kind regards
> > Ola
> >
> >
> >
> >
> >
> > On 26/09/2014 12:48, "Carnë Draug" <carandraug+dev at gmail.com> wrote:
> >
> > >On Thu, 25 Sep 2014 13:31:53 -0400, Yanling Liu <vrnova at gmail.com> wrote:
> > >> Hi,
> > >>
> > >> I was able to use the CLI import (bin/omero import) to import
> > >> images into OMERO. My question is that if OMERO python binding has
> > >> similar capability? Can I write a little python script and call OMERO
> > >> import function to upload some images to OMERO?
> > >>
> > >> What I mean is if there's python binding to the import function,
> > >> rather than doing system call in python script to call bin/omero.
> > >
> > >You can do something like the following:
> > >
> > >    import omero
> > >    import omero.cli
> > >
> > >    client = omero.client (host = server_ip_or_hostname)
> > >    client.createSession (username = "username", password = "password")
> > >
> > >    cli = omero.cli.CLI()
> > >    cli.loadplugins()
> > >    cli._client = client.createClient(secure = True)
> > >    cli.invoke(["import", path_to_image_file])
> > >
> > >Note that you need to create a copy of the client when setting `_client`
> > >because that session will be closed after the image import.
> > >
> > >To find out the ID of the image you imported, you will need to get
> > >this call STDOUT.  I have routinely [1] been doing the following (note
> > >that this imports a single file but should be easy to adjust for
> > >multiple files):
> > >
> > >
> > >    cli = omero.cli.CLI()
> > >    cli.loadplugins()
> > >    cli._client = self.client.createClient(secure = True)
> > >      cmd = [
> > >      "import",
> > >      "--debug", "ERROR",
> > >    ]
> > >    if self.datasetID:
> > >      cmd.extend(["-d", str(self.datasetID)])
> > >    if self.child_name:
> > >      cmd.extend(["-n", self.child_name])
> > >
> > >    cid = None
> > >    with tempfile.NamedTemporaryFile(suffix=".stdout") as stdout:
> > >      cmd.extend([
> > >        "---errs", os.devnull,
> > >        "---file", stdout.name,
> > >      ])
> > >      cmd.append(self.fout.name)
> > >
> > >      STDERR = sys.stderr
> > >      try:
> > >        with open(os.devnull, 'w') as DEVNULL:
> > >          sys.stderr = DEVNULL
> > >          cli.invoke(cmd)
> > >      finally:
> > >        sys.stderr = STDERR
> > >        ret_code = cli.rv
> > >        if ret_code == 0:
> > >          ## we only need to read one line or something is very wrong
> > >          cid = int(stdout.readline())
> > >          if not cid:
> > >            raise Exception("unable to get exported image ID")
> > >        else:
> > >          raise Exception("failed to import processed image")
> > >
> > >    self.child = self.conn.getObject("Image", cid)
> > >
> > >
> > >[1]
> > >https://github.com/MicronOxford/omero_scripts_processing/blob/40e29af8/ome
> > >ro_scripts_processing.py#L293
> > >_______________________________________________
> > >ome-devel mailing list
> > >ome-devel at lists.openmicroscopy.org.uk
> > >http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
> >
> >
> > The University of Dundee is a registered Scottish Charity, No: SC015096
> >
> > _______________________________________________ ome-devel mailing list ome-devel at lists.openmicroscopy.org.uk http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
> > The University of Dundee is a registered Scottish Charity, No: SC015096
> >
> > _______________________________________________
> > ome-devel mailing list
> > ome-devel at lists.openmicroscopy.org.uk
> > http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>
>
> The University of Dundee is a registered Scottish Charity, No: SC015096
>


The University of Dundee is a registered Scottish Charity, No: SC015096

More information about the ome-devel mailing list