[ome-devel] bioformats performance over network shares

[Curtis Rueden]

Hi everyone,

Bio-Formats has classes known as RandomAccessInputStream and
RandomAccessOutputStream, which are intended to allow random access behavior
in a variety of contexts, including streams (with the StreamHandle class).
All of this logic lives in the loci.common package of loci-common.jar.
Unfortunately, StreamHandle is an abstract class and there is not a
general-purpose concrete implementation for streams. If there were, we could
try using a StreamHandle with a BufferedOutputStream of sufficient buffer
size, and see if it made any difference for Rubén.

Melissa, any thoughts on how difficult it would be to make StreamHandle into
a concrete class, with a constructor that accepts a DataInputStream as
argument, and one that accepts a DataOutputStream?

-Curtis

On Mon, Jun 20, 2011 at 2:37 PM, Gregory Jefferis <
jefferis at mrc-lmb.cam.ac.uk> wrote:

> **
> On 2011-06-20 18:02, Johannes Schindelin wrote:
>
> Maybe the random access could be buffered with a configurable window size?
> That could improve performance and be a more general solution..
>
>  This seems like a good idea. For comparison I remember a case for a Fiji
> plugin I wrote (NrrdReader) that when reading compressed files from network
> drives (SMB) adding a fairly small buffer (50k) could result in a several
> fold increase in read speed.
>
> Also was curious about the ratio of bulk write speed to OME tiff write
> speed in OP (Ruben's) case.
>
> Best,
>
> Greg.
>
>
> --
> Gregory Jefferis, PhD                      jefferis at mrc-lmb.cam.ac.uk
> Division of Neurobiology                   LMB Lab:   +44 (0)1223 252943
> MRC Laboratory of Molecular Biology,       LMB Office:+44 (0)1223 252944
> Hills Road,                                LMB Fax:   +44 (0)1223 402310
> Cambridge, CB2 0QH, UK.
>
> http://www2.mrc-lmb.cam.ac.uk/group-leaders/h-to-m/g-jefferis
> http://www.neuroscience.cam.ac.uk/directory/profile.php?gsxej2
> http://flybrain.stanford.edu
>
>
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20110622/1b1d7104/attachment.html>