[ome-devel] Who's who in Pixels?

[Josh Moore]

On Dec 16, 2011, at 1:07 PM, Harri Jäälinoja wrote:

> 
> Hi Josh,
> 
> one more question/comment about the possibility to have OMERO use original images:
> 
>>> This is still sketchy, and not the optimal situation (as pointed out by Josh in his reply on the mailing list) where one of the replicas of image1.jpg is known to OMERO. If OMERO were to understand image1.jpg as such, without the conversion to the internal format, I think we would have the optimal situation. And now that I think about it, maybe also programmatic access to OMERO will be useful here, not just snooping around in the repository.
>>> 
>>> Any comments?
>> 
>> There's been some recent work on making OMERO understand images without conversion, once that API is ready for general consumption, we'll contact the lists. It's possible that that'll be one of the 4.4 releases.
>> 
> 
> Do you think in this case it will be possible for OMERO to use the original images regardless of the directory where they are? In this way, it would for sure be possible to have one iRODS replica known by OMERO.

I don't foresee it becoming possible (or advisable) to have all local filesystems hooked into OMERO, but we'd definitely like there to be more flexibility in what OMERO knows about.

> I guess what I am after is the possibility to register, as opposed to import, data into OMERO. I picked this up from the iRODS system which has this distinction. When data is imported, for example via a GUI, the original stays behind on the remote client machine. Registering can be used when the data is already on the server file system. The data can be registered into iRODS as long as it is readable by the iRODS server user, and from then on, iRODS will maintain metadata about it, just the same way as it does for data that resides in dedicated iRODS vault directories.

There's been some work on registering images into OMERO while working on FS, but one critical complication for iRODs is that in the OMERO implementation we know which files all belong together thanks to Bio-Formats. What type of data are you looking to do this with?

> I can make a dedicated vault directory a Dropbox into OMERO. When new data arrives, OMERO will create another copy in its own binary storage, and forgets about the original copy. So this is still an import operation. Would it be possible for OMERO just to use the file in the Dropbox for all subsequent work? Or maybe add a command to the command line tool, something like:
> 
> bin/omero register --project="p1" --dataset "d1" /data/iRODS/Vault1/home/user1/Data/Screen1
> 
> What do you think? If this were possible, it would be easy to use iRODS for backing up the image data. This might be useful also if the OMERO server is used for image processing tasks and has a limited number of fast disk where it is not possible to store all data. Then to make room for active projects, we could simply remove the iRODS replicas on the fast disk for data that has not been accessed recently. Trying to access them in OMERO would result in an error, which would indicate to the user that s/he should ask the administrator to recover the data. This could be done in iRODS by making a fresh replica on the fast disk.

This would certainly be feasible in the FS system that we are working towards, but at the moment I don't think all the pieces are in place to let you do this.

> Cheers,
> Harri

Cheers,
~Josh