[ome-users] OME and offline archiving

[Ilya Goldberg]

I would totally agree with Ian.  We recently bought 10 750 GB SATA  
"ES" drives from Seagate for $265 each (7.5 tb for $2650).  Of course  
these were made into 2 4x750 raid 5s with two spares, so we "only"  
got 4 tb of formatted space out of that, but a fair amount of  
redundancy and performance for less than $3k.  You can put your 9tb  
live at all times with decent redundancy and performance for under  
$10k including the CPUs, cabinets, etc.  If you build it as you need  
it instead of all at once, it will be much cheaper still.

Of course, there are still people who insist on SCSI or fiberchannel  
disks, and don't mind paying 10x the cost.  Honestly, I think that  
SATA disks are so cheap, and software raid so efficient, that scsi  
and fiberchannel no longer justify their costs.  Its much more cost- 
effective to design a storage solution around a 10% annual disk  
failure rate than to pay 10x the cost for a marginal improvement in  
reliability (see the Google study on disk failure), and a doubtful  
improvement on end-to-end performance and throughput.  Of course,  
others in the "enterprise class" data storage arena would disagree,  
but my feeling is that its only a matter of time before even they see  
the light.  Some already have...

As Tom mentioned, if you really do want to get more complicated with  
the hardware, the software is "almost there" in terms of letting you  
do it this way - basically because its already set up to take  
intermediate steps to prepare a file to properly service a request.

-Ilya


On Jul 16, 2007, at 11:24 AM, Ian Dobbie wrote:

> Mario Valle <mvalle at cscs.ch> writes:
>
>> The project for which I'm evaluating OME plans to produce 9 TB
>> (uncompressed) of images during its lifetime.
>>
>> Obviously I cannot leave all the images on-line. The idea is to have
>> one or two image stacks online and put the rest on our tape
>> archive. Obviously all metadata continue to be online.
>
> I don't see how leaving all the images online is going to be a serious
> problem. I would not be surprised if buying disks for this size of
> storage is cheaper than tapes. A 750GB disk now costs less than 300
> euro, so with a bit of redundancy 9TB will be less than 20 disks, so
> less than 6,000 euro.
>
> The other thing to consider is what kind of time scale are your
> talking over. If this is, say, a 5 year project with the first 3 years
> being getting equipment, workflow etc working. Then by the time you
> are collecting serious data a 2TB disk will probably only be 300-500
> euro. How expensive is the tape going to be? How much hassle will the
> off-line storage be?
>
> If it was my setup I would build an image server, put the pixel data
> there and stuff it with disks. Keep the tapes for actual backups not
> for data that might be in use.
>
> Ian
> _______________________________________________
> ome-users mailing list
> ome-users at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>