[ome-users] OMERO with iRODS?
Colin Blackburn
c.blackburn at dundee.ac.uk
Thu Mar 17 14:56:27 GMT 2011
Hi Harri,
This looks great, thanks for letting us know how thing are going. We'd
certainly be interested with any feedback on performance issues when
you develop things further.
Cheers,
Colin
On 15 Mar 2011, at 12:17, Harri Jäälinoja wrote:
>
> Hello all,
>
> I've made a simple iRODS/OMERO integration prototype. The idea is
> simply to make the iRODS storage area readable by DropBox.
>
> You can find documentation for the prototype here:
> http://wiki.helsinki.fi/display/LMUISA
>
> The OMERO specific part is here:
> http://wiki.helsinki.fi/display/LMUISA/iRODS+and+OMERO+integration
>
> Here I've listed some points why this approach might be interesting:
> http://wiki.helsinki.fi/pages/viewpage.action?pageId=67212966
>
> The prototype is basically functional, but slow, unreliable and not
> connected to any serious disk space. I will build the next version
> on my new desktop computer when it arrives. Then I will hopefully be
> able to have a better idea about performance.
>
> Cheers,
> Harri
>
>
> Curtis Rueden wrote:
>> Hi Harri,
>> Microservices are written in C. So we couldn't access the
>> Bio-Formats C++ bindings from there. Maybe just use a system()
>> call
>> to run B-F?
>> Sure, this is how we integrated Bio-Formats with the original C-
>> and Perl-driven OME server. The "OMEIS" process (written in C) just
>> invoked a particular Bio-Formats Java class responsible for parsing
>> the file in question. This Bio-Formats class then wrote the pixel
>> out to a particular binary file designated by OMEIS, and dumped the
>> OME-XML metadata to stdout, which the OME server's perl layer then
>> consumed. Maybe sounds complicated, but it worked like a charm!
>> The main thing is to create a Java helper class that behaves the
>> way you want. You then make a single system call from C to that
>> class.
>> Making the conversion to OME-TIFF in iRODS requires some coding,
>> the
>> operation should be wrapped into a "microservice":
>> https://www.irods.org/index.php/Micro-Services
>> Since all you need to do is convert to OME-TIFF, system calls will
>> be just fine. The main advantage of the C++ bindings is the ability
>> to perform very granular operations (at the same level as the Java
>> API), which is nice for some applications.
>> You might even be able to get away with calling the Bio-Formats
>> "bfconvert" tool (loci.formats.tools.ImageConverter) without need
>> for your own Java helper class.
>> -Curtis
>> 2010/10/22 Harri Jäälinoja <harri.jaalinoja at helsinki.fi <mailto:harri.jaalinoja at helsinki.fi
>> >>
>> Hi Josh and others,
>> Does iRODS have facilities for monitoring file access times
>> and
>> taking actions based on them?
>> yes, seems so. The attributes that are stored in the iRODS
>> database
>> include timestamps for modification and creation, and I assume
>> it is
>> possible to use those for querying:
>> https://www.irods.org/index.php/icatAttributes
>> It is possible to create delayed or repeated rules, there is a
>> server process that checks every two minutes if there is something
>> to do. So one could run a job every day to move old files to
>> another
>> location for example.
>> From the OMERO point-of-view, this sounds very
>> straight-forward. The only issue I can think of off-hand is
>> that
>> OMERO.fs would have to handle the read-only status. I've
>> added a
>> story for this: http://trac.openmicroscopy.org.uk/omero/ticket/3162
>> Thanks. This is my setup so far:
>> 1. host1 (university general purpose server)
>> - iRODS main server, with iCAT database
>> 2. host2 (Centos virtual server running in VMWare)
>> - OMERO
>> -- still need to enable OMERO.fs
>> - iRODS server 2, no database.
>> -- cannot connect to host1 iRODS, have to figure out why
>> Once I get host2 set up, I can start learning iRODS controls with
>> some simple tests.
>> This is certainly a much larger undertaking, and could
>> possibly
>> involve making Bio-Formats iRODS-aware (just glancing at
>> Jargon
>> -- it does provide a RandomAccessFile interface, so the
>> necessary modifications may be limited). Perhaps there's a way
>> using callbacks to do the reverse of dropbox? i.e. OMERO
>> writes
>> out to a location, and then iRODS takes control of that
>> location
>> on receiving notification?
>> I think it might be possible to accomplish the reverse drop-box
>> with
>> either FUSE (Linux) or WebDAV (Windows, Linux, Mac):
>> https://www.irods.org/index.php/iRODS_FUSE
>> https://projects.arcs.org.au/trac/davis
>> It seems those will provide an area iRODS is aware of, or a drop-
>> box
>> into iRODS. I am not sure about this, though. For example, I was
>> explained that WebDAV doesn't really make a folder where you can
>> write, but rather just a GUI like WinSCP. I don't know about
>> FUSE yet.
>> That's of course difficult for everyone involved to say. When
>> you've got both systems running, perhaps we could start with a
>> very simple test, like opening a file stored in iRODS via
>> Bio-Formats and converting it to OME-TIFF in iRODS. That would
>> let us know if there are any barriers to moving forward.
>> Making the conversion to OME-TIFF in iRODS requires some coding,
>> the
>> operation should be wrapped into a "microservice":
>> https://www.irods.org/index.php/Micro-Services
>> Microservices are written in C. So we couldn't access the
>> Bio-Formats C++ bindings from there. Maybe just use a system()
>> call
>> to run B-F?
>> I will let you know when I make progress.
>> Cheers,
>> Harri
>> Hoping to hear your comments!
>> Best regards,
>> Harri
>> Cheers,
>> ~Josh.
>> _______________________________________________
>> ome-users mailing list
>> ome-users at lists.openmicroscopy.org.uk
>> <mailto:ome-users at lists.openmicroscopy.org.uk>
>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>
> _______________________________________________
> ome-users mailing list
> ome-users at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
Colin Blackburn
Wellcome Trust Centre for Gene Regulation & Expression
College of Life Sciences
MSI/WTB/JBC Complex
University of Dundee
Dow Street
Dundee DD1 5EH
http://openmicroscopy.org
More information about the ome-users
mailing list