[ome-users] Question about archiving OMERO files

[josh.moore at gmx.de]

Hi Robbert,

the solution we had discussed in Paris was adding a flag to the
omero.model.Permissions object. This has not been implemented, but
something you could make use of, if you wanted. Then any
OriginalFile/Pixels object which has the flag shouldn't be touched, to
prevent the black image files being created.

Alternatively, if you keep the sha1's for your pixels and original
files up-to-date, you could check for any discrepancies.

Other than that, the database doesn't really have much information on
what the binary data looks like, and certainly doesn't know about your
removing the files unless you tell it so (via a flag or an
annotation).

However, your idea of changing the group of an entire graph actually
seems quite powerful, since it makes use of the security system which
is something each individual client then doesn't have to worry
about.

Regards,
~josh.

Robbert vd Zon writes:
 > Hi Josh,
 > 
 > No, I did not get a reply and did not find a solution for this
 > problem, but I did find a way to work around this.  My current
 > workaround is that when we 'purge' a project (removing all image
 > files), we optionally also change the user and group of this
 > project in the database. This way, the users do not see the project
 > anymore and therefor can also not try to access the file which
 > would make the new empty file.  With our user interface, users can
 > search their archived and purged projects and restores them.  When
 > we restore a project, we restore all files and also restore the
 > username and group.
 > 
 > Thanks for getting back to me about this.
 > 
 > Regards,
 > 
 > Robbert
 > 
 > 
 > 
 > josh.moore at gmx.de wrote:
 > > (offlist)
 > > Hi Robbert,
 > >
 > > sorry for letting this email slip through the cracks. (It came while I
 > > was on vacation.) Did you get an answer that worked for you?
 > >
 > > ~J
 > >
 > > Robbert vd Zon writes:
 > >  > Hi All,
 > >  > 
 > >  > I am new to the user group and am developing an archive
 > >  > solution for OMERO together with Perkin Elmer.
 > >  > 
 > >  > I saw that Bram Gerritsen had recently came across a problem
 > >  > of OMERO creating a black image file when a user tries to
 > >  > access an image from which the file does not exists anymore
 > >  > (because it is archived). I am having the same problem when
 > >  > developing our archive system.
 > >  > 
 > >  > When a project is archived, we copy all files that belong to
 > >  > this project to an archive location.  After that, we can
 > >  > 'purge' this project, which means that we remove the files
 > >  > from the data location.  When we need to restore a project it
 > >  > can happen that some (or all) image files already exists in
 > >  > the data location. This can either be because the files are
 > >  > recreated with real data. If that is the case, we should NOT
 > >  > override these files.  However, when these files were black
 > >  > images that were created on the fly because users were opening
 > >  > the project while the files were archive, we would need to
 > >  > overwrite the files with the original ones.
 > >  > 
 > >  > I am searching for a way to know if a file is a black image
 > >  > created on the fly, or a real image with correct data.  Does
 > >  > anybody know if there is a way of finding that out?
 > >  > 
 > >  > Can I find this information in the database? I could check the
 > >  > events or compare the file-timestamp with something in the
 > >  > database.  Or is there a way that I can recognize it from the
 > >  > file itself (by its size or the content)?
 > >  > 
 > >  > We are trying to create a solution which works both with OMERO
 > >  > 3 and OMERO 4 without modifying OMERO.  So modifying the
 > >  > source is something we would rather not do.
 > >  > 
 > >  > Any help is highly appreciated
 > >  > 
 > >  > Thanks,
 > >  > 
 > >  > Robbert vd Zon
 > >  > DAX Archiving Solutions
 > >