[ome-users] OMERO with iRODS?
Harri Jäälinoja
harri.jaalinoja at helsinki.fi
Mon Mar 21 14:03:31 GMT 2011
Hi Colin,
thanks for your reply! One thing that comes to mind: at the moment
DropBox works fine, when new images arrive at acquisition speed.
However, in this proposed setup someone might easily get the idea to
import an entire project to OMERO via DropBox, so it would be good to be
prepared for this somehow.
Cheers,
Harri
Colin Blackburn wrote:
> Hi Harri,
>
> This looks great, thanks for letting us know how thing are going. We'd
> certainly be interested with any feedback on performance issues when you
> develop things further.
>
> Cheers,
>
> Colin
>
>
>
> On 15 Mar 2011, at 12:17, Harri Jäälinoja wrote:
>
>>
>> Hello all,
>>
>> I've made a simple iRODS/OMERO integration prototype. The idea is
>> simply to make the iRODS storage area readable by DropBox.
>>
>> You can find documentation for the prototype here:
>> http://wiki.helsinki.fi/display/LMUISA
>>
>> The OMERO specific part is here:
>> http://wiki.helsinki.fi/display/LMUISA/iRODS+and+OMERO+integration
>>
>> Here I've listed some points why this approach might be interesting:
>> http://wiki.helsinki.fi/pages/viewpage.action?pageId=67212966
>>
>> The prototype is basically functional, but slow, unreliable and not
>> connected to any serious disk space. I will build the next version on
>> my new desktop computer when it arrives. Then I will hopefully be able
>> to have a better idea about performance.
>>
>> Cheers,
>> Harri
>>
>>
>> Curtis Rueden wrote:
>>> Hi Harri,
>>> Microservices are written in C. So we couldn't access the
>>> Bio-Formats C++ bindings from there. Maybe just use a system() call
>>> to run B-F?
>>> Sure, this is how we integrated Bio-Formats with the original C- and
>>> Perl-driven OME server. The "OMEIS" process (written in C) just
>>> invoked a particular Bio-Formats Java class responsible for parsing
>>> the file in question. This Bio-Formats class then wrote the pixel out
>>> to a particular binary file designated by OMEIS, and dumped the
>>> OME-XML metadata to stdout, which the OME server's perl layer then
>>> consumed. Maybe sounds complicated, but it worked like a charm!
>>> The main thing is to create a Java helper class that behaves the way
>>> you want. You then make a single system call from C to that class.
>>> Making the conversion to OME-TIFF in iRODS requires some coding, the
>>> operation should be wrapped into a "microservice":
>>> https://www.irods.org/index.php/Micro-Services
>>> Since all you need to do is convert to OME-TIFF, system calls will be
>>> just fine. The main advantage of the C++ bindings is the ability to
>>> perform very granular operations (at the same level as the Java API),
>>> which is nice for some applications.
>>> You might even be able to get away with calling the Bio-Formats
>>> "bfconvert" tool (loci.formats.tools.ImageConverter) without need for
>>> your own Java helper class.
>>> -Curtis
>>> 2010/10/22 Harri Jäälinoja <harri.jaalinoja at helsinki.fi
>>> <mailto:harri.jaalinoja at helsinki.fi>>
>>> Hi Josh and others,
>>> Does iRODS have facilities for monitoring file access times and
>>> taking actions based on them?
>>> yes, seems so. The attributes that are stored in the iRODS database
>>> include timestamps for modification and creation, and I assume it is
>>> possible to use those for querying:
>>> https://www.irods.org/index.php/icatAttributes
>>> It is possible to create delayed or repeated rules, there is a
>>> server process that checks every two minutes if there is something
>>> to do. So one could run a job every day to move old files to another
>>> location for example.
>>> From the OMERO point-of-view, this sounds very
>>> straight-forward. The only issue I can think of off-hand is that
>>> OMERO.fs would have to handle the read-only status. I've added a
>>> story for this:
>>> http://trac.openmicroscopy.org.uk/omero/ticket/3162
>>> Thanks. This is my setup so far:
>>> 1. host1 (university general purpose server)
>>> - iRODS main server, with iCAT database
>>> 2. host2 (Centos virtual server running in VMWare)
>>> - OMERO
>>> -- still need to enable OMERO.fs
>>> - iRODS server 2, no database.
>>> -- cannot connect to host1 iRODS, have to figure out why
>>> Once I get host2 set up, I can start learning iRODS controls with
>>> some simple tests.
>>> This is certainly a much larger undertaking, and could possibly
>>> involve making Bio-Formats iRODS-aware (just glancing at Jargon
>>> -- it does provide a RandomAccessFile interface, so the
>>> necessary modifications may be limited). Perhaps there's a way
>>> using callbacks to do the reverse of dropbox? i.e. OMERO writes
>>> out to a location, and then iRODS takes control of that location
>>> on receiving notification?
>>> I think it might be possible to accomplish the reverse drop-box with
>>> either FUSE (Linux) or WebDAV (Windows, Linux, Mac):
>>> https://www.irods.org/index.php/iRODS_FUSE
>>> https://projects.arcs.org.au/trac/davis
>>> It seems those will provide an area iRODS is aware of, or a drop-box
>>> into iRODS. I am not sure about this, though. For example, I was
>>> explained that WebDAV doesn't really make a folder where you can
>>> write, but rather just a GUI like WinSCP. I don't know about FUSE
>>> yet.
>>> That's of course difficult for everyone involved to say. When
>>> you've got both systems running, perhaps we could start with a
>>> very simple test, like opening a file stored in iRODS via
>>> Bio-Formats and converting it to OME-TIFF in iRODS. That would
>>> let us know if there are any barriers to moving forward.
>>> Making the conversion to OME-TIFF in iRODS requires some coding, the
>>> operation should be wrapped into a "microservice":
>>> https://www.irods.org/index.php/Micro-Services
>>> Microservices are written in C. So we couldn't access the
>>> Bio-Formats C++ bindings from there. Maybe just use a system() call
>>> to run B-F?
>>> I will let you know when I make progress.
>>> Cheers,
>>> Harri
>>> Hoping to hear your comments!
>>> Best regards,
>>> Harri
>>> Cheers,
>>> ~Josh.
>>> _______________________________________________
>>> ome-users mailing list
>>> ome-users at lists.openmicroscopy.org.uk
>>> <mailto:ome-users at lists.openmicroscopy.org.uk>
>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>>
>> _______________________________________________
>> ome-users mailing list
>> ome-users at lists.openmicroscopy.org.uk
>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>
> Colin Blackburn
> Wellcome Trust Centre for Gene Regulation & Expression
> College of Life Sciences
> MSI/WTB/JBC Complex
> University of Dundee
> Dow Street
> Dundee DD1 5EH
>
> http://openmicroscopy.org
>
>
>
>
More information about the ome-users
mailing list