[ome-users] ImageProcessorReader.setId performance

[Balaji Ramalingam]

Hi,

Your problem seems to be related to the search for Images by bio-formats(loci-tools), within your target directory (imagePath).

This is how the algorithm works in bio-formats,

  1.

The target directory(imagePath) is searched for all ‘bioformats-readable’ images.

-This is the very first step and this is where you are facing problems, I am assuming that you’re ‘imagePath’, is a top directory which has images at various depth’s. The default folder-depth at which bio-formats searches is 4 (I.e, it might still be searching for 4 sub-level folder’s within the top directory, unless specified).

    - The easy alternative is to pre list all the RVG format files within the ‘imagePath' (using file-system search), and read them one at a time      (iteratively). I.e imagePath == “Path to an individual image”.
       This basically removes the bio-formats intelligence of grouping similar files (files part of a stack,etc). Am assuming that should not be  a     problem in your case .

2.  The other alternative is to use the ImagePlusReader instead of the ImageProcessor as suggested in the code, but this needs testing      and playing around.

Quoting from the “ImageProcessorReader.java" code ,

  *

A low-level reader for {@link ij.process.ImageProcessor} objects.
 * For a higher-level reader that returns {@link ij.ImagePlus} objects,
 * see {@link loci.plugins.in.ImagePlusReader} instead.

Please do let us know, how it goes.

Best,
Balaji


From: Mohammad Samarah <isamarah at mac.com<mailto:isamarah at mac.com>>
Reply-To: "msamarah at alumni.fit.edu<mailto:msamarah at alumni.fit.edu>" <msamarah at alumni.fit.edu<mailto:msamarah at alumni.fit.edu>>
Date: Saturday, 22 March 2014 03:36
To: "ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>" <ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>>
Subject: [ome-users] ImageProcessorReader.setId performance

We are using loci_tools.jar to read radiographs in RVG format and extract a thumbnail JPEG image. All works as expected, however if the image in question is in a directory whose parent directory contains several thousands subfolders, then the ImageProcesserReader takes minutes to complete the setId() method. Please see attached code. The same image placed in a folder with its parent directory having much fewer subfolders works with no delay. This behavior can also be seen when using loci_tools.jar UI to open the same image. The issue does not seem to be related to the image itself but rather the fact that the containing folder is a child of a directory with thousands of subfolders. Could you please point us to how we can resolve this issue.

Many thanks for your help,

--
M. Samarah
msamarah at alumni.fit.edu


private ImagePlus readDicomImage(String imagePath)
    {
        ImagePlus imp            = null;
        ImagePlus colorizedImage = null;

        ImageProcessorReader r = new ImageProcessorReader(new ChannelSeparator(LociPrefs.makeImageReader()));
        try
        {
            log_message("INFO: Examining file " + imagePath);

            // the next line takes 10 to 20 minutes to complete when parent folder contains 4000+ subfolders. It takes no time when parent folder has 100 subfolders
            r.setId(imagePath);

            log_message("INFO: Setting file path " + imagePath);
            int num = r.getImageCount();
            log_message("INFO: Image count is" + num);
            int width = r.getSizeX();
            int height = r.getSizeY();
            log_message("INFO: Image Width %d Height %d", width, height);
            ImageStack stack = new ImageStack(width, height);
            log_message("INFO: After constructing image stack");
            byte[][][] lookupTable = new byte[r.getSizeC()][][];
            log_message("INFO: After Allocating memmory for image");
            for (int i=0; i<num; i++)
            {
                ImageProcessor ip = r.openProcessors(i)[0];
                stack.addSlice("" + (i + 1), ip);
                int channel = r.getZCTCoords(i)[1];
                lookupTable[channel] = r.get8BitLookupTable();
            }
            log_message("INFO: After constructing lookup table");
            log_message("INFO:Constructing image");
            imp = new ImagePlus("Dicom", stack);

            colorizedImage = applyLookupTables(r, imp, lookupTable);
            r.close();

        }
        catch(Exception exc)
        {
           log_message("ERROR an error occurred: " + exc.getMessage());
        }
       return colorizedImage;
    }

Example: Takes 10 to 20 minutes on an 8 core machine with 24GB of memory to complete setId method
Data <folder>
0001 <folder>
0002 <folder>
0003 <folder>
...
1000 <folder>
1001 <folder>
File R1.RVG
...
4000 <folder>

Example: Works with no delay
Data <folder>
0001 <folder>
0002 <folder>
0003 <folder>
...
0050 <folder>
0051 <folder>
File R1.RVG
...
00080 <folder>



The University of Dundee is a registered Scottish Charity, No: SC015096
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20140324/e05270dd/attachment.html>