[ome-users] performances of retrieving data within matlab from omero server

Jacques Pecreaux jacques.pecreaux at googlemail.com
Fri Jun 24 18:40:15 BST 2011 

Dear List,

I use matlab code to process images stored in Omero and I noticed that a
major bottleneck of the process is the time required to retrieve images from
omero to memory using omero matlab client and I am wondering if other people
experienced the same issue and more importantly if they are aware of
possibilities of improvements.

To be more detailed, I use matlab 2011a on a debian 6 computer (dual iXon
quad core 2.6 Ghz, 24 GB RAM, details below), data are located on a drobo
pro conected to the computer via iSCSI Gigabit interface. Within matlab,
using getPlaneFromImageId in the omero matlab toolbox, I read a 4086 frame
tif file (512x512, 16 bits) in roughtly 200 s. Using omero insight, I
downloaded the original tiff file into the drobo volume and opened it (not
as virtual stack of course) with Fiji and it took about 50 seconds. Copying
the data between two folders on the drobo takes 25 s.

I currently run image processing on data stored in Omero and when profiling
the algorithm, 35 % of the time is spent on getPlaneFromImageId. Thus, a few
fold of speed increase would be a strong advantage considering that the
processing take in toto a few days. Most of the reading are sequential (or
every n-th images).

I would be very grateful if anyone would share suggestions on the way of
optimizing this.

Many thanks in advance

Best regards

Jacques Pécréaux

------------------
System information report, generated by Sysinfo: 6/24/2011 12:29:48 PM
http://sourceforge.net/projects/gsysinfo

SYSTEM INFORMATION
    Running Debian Linux, the 6.0.1 release.
    GNOME: 2.30.2 (Debian 2010-11-12)
    Kernel version: 2.6.32-5-amd64 (#1 SMP Mon Mar 7 21:35:22 UTC 2011)
    GCC: 4.3.5 (x86_64-linux-gnu)
    Xorg: unknown (18 February 2011  08:27:24PM) (18 February 2011
08:27:24PM)
    Hostname: cedre-5a.med.univ-rennes1.fr
    Uptime: 35 days 3 h 4 min

CPU INFORMATION
    GenuineIntel, Intel(R) Xeon(R) CPU           E5620  @ 2.40GHz
    Number of CPUs: 16
    CPU clock currently at 1596.000 MHz with 12288 KB cache
    Numbering: family(6) model(44) stepping(2)
    Bogomips: 4787.70
    Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat
pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb
rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc
aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16
xtpr pdcm dca sse4_1 sse4_2 popcnt aes lahf_lm ida arat tpr_shadow vnmi
flexpriority ept vpid

MEMORY INFORMATION
    Total memory: 24148 MB
    Total swap: 166892 MB

STORAGE INFORMATION

HARDWARE INFORMATION
MOTHERBOARD
    Host bridge
        Intel Corporation Xeon 5600 Series Integrated Memory Controller
Channel 2 Thermal Control (rev 02)
        Subsystem: Hewlett-Packard Company Device 130b
    PCI bridge(s)
        Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 1 (rev
13) (prog-if 00 [Normal decode])
        Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 3 (rev
13) (prog-if 00 [Normal decode])
        Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 7 (rev
13) (prog-if 00 [Normal decode])
        Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 1
(prog-if 00 [Normal decode])
        Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 5
(prog-if 00 [Normal decode])
        Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 6
(prog-if 00 [Normal decode])
        Intel Corporation 82801 PCI Bridge (rev 90) (prog-if 01 [Subtractive
decode])
        Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 3 (rev
13) (prog-if 00 [Normal decode])
        Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 7 (rev
13) (prog-if 00 [Normal decode])
        Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 9 (rev
13) (prog-if 00 [Normal decode])
        Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 1 (rev
13) (prog-if 00 [Normal decode])
        Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 3 (rev
13) (prog-if 00 [Normal decode])
        Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 7 (rev
13) (prog-if 00 [Normal decode])
        Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 1
(prog-if 00 [Normal decode])
        Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 5
(prog-if 00 [Normal decode])
        Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 6
(prog-if 00 [Normal decode])
        Intel Corporation 82801 PCI Bridge (rev 90) (prog-if 01 [Subtractive
decode])
        Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 3 (rev
13) (prog-if 00 [Normal decode])
        Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 7 (rev
13) (prog-if 00 [Normal decode])
        Intel Corporation 5520/5500/X58 I/O Hub PCI Express Root Port 9 (rev
13) (prog-if 00 [Normal decode])
    USB controller(s)
        Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4
(prog-if 00 [UHCI])
        Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5
(prog-if 00 [UHCI])
        Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6
(prog-if 00 [UHCI])
        Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2
(prog-if 20 [EHCI])
        Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1
(prog-if 00 [UHCI])
        Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2
(prog-if 00 [UHCI])
        Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3
(prog-if 00 [UHCI])
        Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1
(prog-if 20 [EHCI])
        Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4
(prog-if 00 [UHCI])
        Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5
(prog-if 00 [UHCI])
        Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6
(prog-if 00 [UHCI])
        Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2
(prog-if 20 [EHCI])
        Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1
(prog-if 00 [UHCI])
        Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2
(prog-if 00 [UHCI])
        Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3
(prog-if 00 [UHCI])
        Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1
(prog-if 20 [EHCI])
    ISA bridge
        Intel Corporation 82801JIR (ICH10R) LPC Interface Controller
        Subsystem: Hewlett-Packard Company Device 130b

GRAPHIC CARD
    VGA controller
        nVidia Corporation G96 [Quadro FX 580] (rev a1) (prog-if 00 [VGA
controller])
        Subsystem: nVidia Corporation Device 063a

SOUND CARD
    Multimedia controller
        Intel Corporation 82801JI (ICH10 Family) HD Audio Controller
        Subsystem: Hewlett-Packard Company Device 130b

NETWORK
    Ethernet controller
        Broadcom Corporation NetXtreme BCM5764M Gigabit Ethernet PCIe (rev
10)
        Subsystem: Hewlett-Packard Company Device 130b

NVIDIA GRAPHIC CARD INFORMATION
    Model name: Quadro FX 580
    Card Type: PCI-E
    Video RAM: unknown
    GPU Frequency: unknown
    Driver version: NVIDIA UNIX x86_64 Kernel Module  260.19.44  Sun Feb 27
22:41:03 PST 2011
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20110624/a6753dd4/attachment.html>

More information about the ome-users mailing list