[ome-users] Why is the original image data

[Ilya Goldberg]

I'll add a third too because also "in real life" users often have 
several copies of the same image (usually without realizing it).  I've 
seen this basically everywhere I've bothered to look.  In extreme 
cases, I've seen nearly all of the files have at least one and 
sometimes several copies elsewhere on the fileserver.

Hence, 3) OMEIS enforces unique constraints on all the files/pixels it 
maintains.  So if you maintain your files on OMEIS, you are guaranteed 
to always have exactly one copy - even if you rename the files.  Since 
one OMEIS can service multiple OME "back ends", you can effectively 
enforce the uniqueness constraint across your entire site, including 
people running separate OME back-ends on their desk-tops, for instance. 
  The uniqueness is enforced transparently - in other words its not an 
error to send OMEIS multiple copies of the same file.  If the filename 
and contents are the same as one already stored, you'll just get back 
the old OMEIS ID.  If the filename is different, you'll get a new OMEIS 
ID, but it will be an alias to the original file, and its contents will 
not be duplicated.

So that's another benefit of using managed storage for "original data". 
  In most cases, we have found it very difficult to convince people that 
they don't actually need their original data anymore.  There's 
something magical about that original file, and people are just not 
willing to part with it.  If you can convince them that the file is 
actually stored for them exactly as it was and show them how easy it is 
to get it back, only then are they willing to forgo their personal 
copies of their images and the endless stacks of CDs and DVDs they keep 
them on.

-Ilya

On Sep 13, 2005, at 3:36 PM, Ilya Goldberg wrote:

> Basically, what Mike said.
> I would add that we have taken some steps to mitigate the additional 
> space usage:
> 1.  Image data is compressed unless its actively used (both the 
> original data and the OME Format pixels).  In "real life" scenarios, 
> this results in a recovery of 1/2 to 2/3 of disk space.  That's disk 
> space assuming no OMEIS at all.  In other words, using OMEIS with its 
> duplication of pixels uses 1/2 - 1/3 of the disk space compared to 
> storing just the original data without OMEIS (and without 
> compression).  Images are compressed after 30-90 days depending on 
> settings, so this is over a significant time frame (many months).
> 2.  The pixels in "OME Format" (i.e. not "original data") are 
> basically a cache.  Also depending on settings, they get purged so 
> that the "original data" is really the only repository of the pixels.  
> This only applies to cases where the OME Format pixels *are* the 
> original pixels.  If you have pixels that are derived (from analysis), 
> then they are the original pixels, and will therefore never be purged 
> unless you delete the MEXes that created them.
>
> Both of these things are completely transparent to the user, except 
> that a delay will be incurred when inflating pixels or recovering from 
> a "cache miss".
>
> So in short, its not as bad as it initially seems, and in reality its 
> often a lot better than what people have presently.
>
> -Ilya
>
> On Sep 14, 2005, at 12:46 PM, gerhard wrote:
>
>> Can anyone tell me why the original image data is also stored on 
>> OMEIS.
>> regarding the space images need, does it make sense?
>> _______________________________________________
>> ome-users mailing list
>> ome-users at lists.openmicroscopy.org.uk
>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>>
>
> _______________________________________________
> ome-users mailing list
> ome-users at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>