[ome-users] OMERO with iRODS?

[Harri Jäälinoja]

Hello all,

I've made a simple iRODS/OMERO integration prototype. The idea is simply 
to make the iRODS storage area readable by DropBox.

You can find documentation for the prototype here:
http://wiki.helsinki.fi/display/LMUISA

The OMERO specific part is here:
http://wiki.helsinki.fi/display/LMUISA/iRODS+and+OMERO+integration

Here I've listed some points why this approach might be interesting:
http://wiki.helsinki.fi/pages/viewpage.action?pageId=67212966

The prototype is basically functional, but slow, unreliable and not 
connected to any serious disk space. I will build the next version on my 
new desktop computer when it arrives. Then I will hopefully be able to 
have a better idea about performance.

Cheers,
Harri


Curtis Rueden wrote:
> Hi Harri,
> 
>     Microservices are written in C. So we couldn't access the
>     Bio-Formats C++ bindings from there. Maybe just use a system() call
>     to run B-F?
> 
> 
> Sure, this is how we integrated Bio-Formats with the original C- and 
> Perl-driven OME server. The "OMEIS" process (written in C) just invoked 
> a particular Bio-Formats Java class responsible for parsing the file in 
> question. This Bio-Formats class then wrote the pixel out to a 
> particular binary file designated by OMEIS, and dumped the OME-XML 
> metadata to stdout, which the OME server's perl layer then consumed. 
> Maybe sounds complicated, but it worked like a charm!
> 
> The main thing is to create a Java helper class that behaves the way you 
> want. You then make a single system call from C to that class.
> 
>     Making the conversion to OME-TIFF in iRODS requires some coding, the
>     operation should be wrapped into a "microservice":
>     https://www.irods.org/index.php/Micro-Services
> 
> 
> Since all you need to do is convert to OME-TIFF, system calls will be 
> just fine. The main advantage of the C++ bindings is the ability to 
> perform very granular operations (at the same level as the Java API), 
> which is nice for some applications.
> 
> You might even be able to get away with calling the Bio-Formats 
> "bfconvert" tool (loci.formats.tools.ImageConverter) without need for 
> your own Java helper class.
> 
> -Curtis
> 
> 2010/10/22 Harri Jäälinoja <harri.jaalinoja at helsinki.fi 
> <mailto:harri.jaalinoja at helsinki.fi>>
> 
>     Hi Josh and others,
> 
> 
>         Does iRODS have facilities for monitoring file access times and
>         taking actions based on them?
> 
> 
>     yes, seems so. The attributes that are stored in the iRODS database
>     include timestamps for modification and creation, and I assume it is
>     possible to use those for querying:
>     https://www.irods.org/index.php/icatAttributes
> 
>     It is possible to create delayed or repeated rules, there is a
>     server process that checks every two minutes if there is something
>     to do. So one could run a job every day to move old files to another
>     location for example.
> 
> 
> 
>          From the OMERO point-of-view, this sounds very
>         straight-forward. The only issue I can think of off-hand is that
>         OMERO.fs would have to handle the read-only status. I've added a
>         story for this: http://trac.openmicroscopy.org.uk/omero/ticket/3162
> 
>     Thanks. This is my setup so far:
> 
>     1. host1 (university general purpose server)
>     - iRODS main server, with iCAT database
> 
>     2. host2 (Centos virtual server running in VMWare)
>     - OMERO
>     -- still need to enable OMERO.fs
>     - iRODS server 2, no database.
>     -- cannot connect to host1 iRODS, have to figure out why
> 
>     Once I get host2 set up, I can start learning iRODS controls with
>     some simple tests.
> 
> 
>         This is certainly a much larger undertaking, and could possibly
>         involve making Bio-Formats iRODS-aware (just glancing at Jargon
>         -- it does provide a RandomAccessFile interface, so the
>         necessary modifications may be limited). Perhaps there's a way
>         using callbacks to do the reverse of dropbox? i.e. OMERO writes
>         out to a location, and then iRODS takes control of that location
>         on receiving notification?
> 
>     I think it might be possible to accomplish the reverse drop-box with
>     either FUSE (Linux) or WebDAV (Windows, Linux, Mac):
>     https://www.irods.org/index.php/iRODS_FUSE
>     https://projects.arcs.org.au/trac/davis
> 
>     It seems those will provide an area iRODS is aware of, or a drop-box
>     into iRODS. I am not sure about this, though. For example, I was
>     explained that WebDAV doesn't really make a folder where you can
>     write, but rather just a GUI like WinSCP. I don't know about FUSE yet.
> 
> 
>         That's of course difficult for everyone involved to say. When
>         you've got both systems running, perhaps we could start with a
>         very simple test, like opening a file stored in iRODS via
>         Bio-Formats and converting it to OME-TIFF in iRODS. That would
>         let us know if there are any barriers to moving forward.
> 
>     Making the conversion to OME-TIFF in iRODS requires some coding, the
>     operation should be wrapped into a "microservice":
>     https://www.irods.org/index.php/Micro-Services
> 
>     Microservices are written in C. So we couldn't access the
>     Bio-Formats C++ bindings from there. Maybe just use a system() call
>     to run B-F?
> 
>     I will let you know when I make progress.
> 
>     Cheers,
>     Harri
> 
> 
>             Hoping to hear your comments!
> 
>             Best regards,
>             Harri
> 
> 
>         Cheers,
>         ~Josh.
> 
> 
>     _______________________________________________
>     ome-users mailing list
>     ome-users at lists.openmicroscopy.org.uk
>     <mailto:ome-users at lists.openmicroscopy.org.uk>
>     http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
> 
>