[ome-devel] Who's who in Pixels? (iRODS ideas...)

[Josh Moore]

On Dec 16, 2011, at 4:20 PM, Harri Jäälinoja wrote:

> 
> Hi again,
> I was just browsing the list archives looking for info about OMERO scripting, and my eye caught this:
> 
> http://lists.openmicroscopy.org.uk/pipermail/ome-users/2011-June/002652.html
> 
> "The OME Team is starting to see many requests for the kind of thing you are asking:  "... manage image data across multiple centers/institutes and research labs"."
> 
> This is also something that can be accomplished with iRODS. So I think if it is possible to achieve a working integration between the two systems, there is no need to implement the same functionality again in OMERO.
> 
> What I suggested earlier, the possibility register data into OMERO, would be one way to allow "image access" to iRODS. In this case, when I want to see my collaborators data in OMERO, I would first use iRODS to replicate it on a local storage resource (where OMERO can see it), and then register it to OMERO.

Hi Harri,

Since often it's not just the data that users want to collaborate on, but also the annotations and other metadata associated with it, the views would disconnected if the same data lived in multiple OMERO instances. But iRODs would certainly be an option for sharing the binary data between sites.

Cheers,
~Josh.


> It might be possible to make this process easier to the end user, as there is a Java API to iRODS, but that would be more difficult, as we've discussed earlier :)
> http://lists.openmicroscopy.org.uk/pipermail/ome-users/2010-October/002452.html
> 
> Cheers,
> Harri
> 
> On 16/12/11 14:07, Harri Jäälinoja wrote:
>> 
>> Hi Josh,
>> 
>> one more question/comment about the possibility to have OMERO use
>> original images:
>> 
>>>> This is still sketchy, and not the optimal situation (as pointed out
>>>> by Josh in his reply on the mailing list) where one of the replicas
>>>> of image1.jpg is known to OMERO. If OMERO were to understand
>>>> image1.jpg as such, without the conversion to the internal format, I
>>>> think we would have the optimal situation. And now that I think about
>>>> it, maybe also programmatic access to OMERO will be useful here, not
>>>> just snooping around in the repository.
>>>> 
>>>> Any comments?
>>> 
>>> There's been some recent work on making OMERO understand images
>>> without conversion, once that API is ready for general consumption,
>>> we'll contact the lists. It's possible that that'll be one of the 4.4
>>> releases.
>>> 
>> 
>> Do you think in this case it will be possible for OMERO to use the
>> original images regardless of the directory where they are? In this way,
>> it would for sure be possible to have one iRODS replica known by OMERO.
>> 
>> I guess what I am after is the possibility to register, as opposed to
>> import, data into OMERO. I picked this up from the iRODS system which
>> has this distinction. When data is imported, for example via a GUI, the
>> original stays behind on the remote client machine. Registering can be
>> used when the data is already on the server file system. The data can be
>> registered into iRODS as long as it is readable by the iRODS server
>> user, and from then on, iRODS will maintain metadata about it, just the
>> same way as it does for data that resides in dedicated iRODS vault
>> directories.
>> 
>> I can make a dedicated vault directory a Dropbox into OMERO. When new
>> data arrives, OMERO will create another copy in its own binary storage,
>> and forgets about the original copy. So this is still an import
>> operation. Would it be possible for OMERO just to use the file in the
>> Dropbox for all subsequent work? Or maybe add a command to the command
>> line tool, something like:
>> 
>> bin/omero register --project="p1" --dataset "d1"
>> /data/iRODS/Vault1/home/user1/Data/Screen1
>> 
>> What do you think? If this were possible, it would be easy to use iRODS
>> for backing up the image data. This might be useful also if the OMERO
>> server is used for image processing tasks and has a limited number of
>> fast disk where it is not possible to store all data. Then to make room
>> for active projects, we could simply remove the iRODS replicas on the
>> fast disk for data that has not been accessed recently. Trying to access
>> them in OMERO would result in an error, which would indicate to the user
>> that s/he should ask the administrator to recover the data. This could
>> be done in iRODS by making a fresh replica on the fast disk.
>> 
>> Cheers,
>> Harri
>> 
> 
> 
> -- 
> __________________________________________________
> Harri Jäälinoja
> Light Microscopy Unit
> Institute of Biotechnology, University of Helsinki
> http://www.biocenter.helsinki.fi/bi/lmu/
> +358 9 191 59370 fax +358 9 191 59366
> 
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel