[ome-users] ImageProcessorReader.setId performance
Mohammad Samarah
isamarah at mac.com
Thu Mar 27 23:35:20 GMT 2014
Hi Balaji,
Calling setGroupFiles(false) prior to invoking setId() solved the issue.
Many thanks for your help.
Mohammad
From: Balaji Ramalingam <b.ramalingam at dundee.ac.uk>
Date: Tue, 25 Mar 2014 15:55:58 +0000
To: "msamarah at alumni.fit.edu" <msamarah at alumni.fit.edu>,
"ome-users at lists.openmicroscopy.org.uk"
<ome-users at lists.openmicroscopy.org.uk>
Subject: Re: [ome-users] ImageProcessorReader.setId performance
Hi,
I missed a point in my previous message, turns out that the reader searches
and Œgroups¹ similar images (this is helpful in the case of stacks, etc). In
your case turning that option Œoff¹ might do the trick.
If you could add this call,
r.setGroupFiles(false)
Before the call,
R.setId(ImagePath)
Then that should do the trick.
Best,
Balaji
From: Mohammad Samarah <isamarah at mac.com>
Reply-To: "msamarah at alumni.fit.edu" <msamarah at alumni.fit.edu>
Date: Tuesday, 25 March 2014 14:19
To: Balaji Ramalingam <b.ramalingam at dundee.ac.uk>,
"ome-users at lists.openmicroscopy.org.uk"
<ome-users at lists.openmicroscopy.org.uk>
Subject: Re: [ome-users] ImageProcessorReader.setId performance
Re: [ome-users] ImageProcessorReader.setId performance
Hi Balaji,
Thanks for your insights. We examined the code more closely and through the
debugger. We are passing a single full file path to the
ImageProcesserReader.setId method. The case that fails specifies
/images/001234/R1.RVG while the images folder has more than 4000
sub-folders. We are able to make a workaround by means of copying the same
file to /temp/images/001234/R1.RVG. Temp/images folder contains a small
number of sub-folders. However this is not ideal as the copy takes time to
complete given the size of the images being processed.
You mentioned that the default folder-depth is 4. How do we change it to 1?
Is there an API method to disable this behavior?
Many thanks for your help,
--
M. Samarah
msamarah at alumni.fit.edu
From: Balaji Ramalingam <b.ramalingam at dundee.ac.uk>
Date: Mon, 24 Mar 2014 10:42:02 +0000
To: "msamarah at alumni.fit.edu" <msamarah at alumni.fit.edu>,
"ome-users at lists.openmicroscopy.org.uk"
<ome-users at lists.openmicroscopy.org.uk>
Subject: Re: [ome-users] ImageProcessorReader.setId performance
Hi,
Your problem seems to be related to the search for Images by
bio-formats(loci-tools), within your target directory (imagePath).
This is how the algorithm works in bio-formats,
1. The target directory(imagePath) is searched for all Œbioformats-readable¹
images.
2. -This is the very first step and this is where you are facing problems, I
am assuming that you¹re ŒimagePath¹, is a top directory which has images at
various depth¹s. The default folder-depth at which bio-formats searches is 4
(I.e, it might still be searching for 4 sub-level folder¹s within the top
directory, unless specified).
- The easy alternative is to pre list all the RVG format files within the
ŒimagePath' (using file-system search), and read them one at a time
(iteratively). I.e imagePath == ³Path to an individual image².
This basically removes the bio-formats intelligence of grouping
similar files (files part of a stack,etc). Am assuming that should not be a
problem in your case .
2. The other alternative is to use the ImagePlusReader instead of the
ImageProcessor as suggested in the code, but this needs testing and
playing around.
Quoting from the ³ImageProcessorReader.java" code ,
* A low-level reader for {@link ij.process.ImageProcessor} objects.
* * For a higher-level reader that returns {@link ij.ImagePlus} objects,
* * see {@link loci.plugins.in.ImagePlusReader} instead.
Please do let us know, how it goes.
Best,
Balaji
From: Mohammad Samarah <isamarah at mac.com>
Reply-To: "msamarah at alumni.fit.edu" <msamarah at alumni.fit.edu>
Date: Saturday, 22 March 2014 03:36
To: "ome-users at lists.openmicroscopy.org.uk"
<ome-users at lists.openmicroscopy.org.uk>
Subject: [ome-users] ImageProcessorReader.setId performance
ImageProcessorReader.setId performance
We are using loci_tools.jar to read radiographs in RVG format and extract a
thumbnail JPEG image. All works as expected, however if the image in
question is in a directory whose parent directory contains several thousands
subfolders, then the ImageProcesserReader takes minutes to complete the
setId() method. Please see attached code. The same image placed in a folder
with its parent directory having much fewer subfolders works with no delay.
This behavior can also be seen when using loci_tools.jar UI to open the same
image. The issue does not seem to be related to the image itself but rather
the fact that the containing folder is a child of a directory with thousands
of subfolders. Could you please point us to how we can resolve this issue.
Many thanks for your help,
--
M. Samarah
msamarah at alumni.fit.edu
private ImagePlus readDicomImage(String imagePath)
{
ImagePlus imp = null;
ImagePlus colorizedImage = null;
ImageProcessorReader r = new ImageProcessorReader(new
ChannelSeparator(LociPrefs.makeImageReader()));
try
{
log_message("INFO: Examining file " + imagePath);
// the next line takes 10 to 20 minutes to complete when parent
folder contains 4000+ subfolders. It takes no time when parent folder has
100 subfolders
r.setId(imagePath);
log_message("INFO: Setting file path " + imagePath);
int num = r.getImageCount();
log_message("INFO: Image count is" + num);
int width = r.getSizeX();
int height = r.getSizeY();
log_message("INFO: Image Width %d Height %d", width, height);
ImageStack stack = new ImageStack(width, height);
log_message("INFO: After constructing image stack");
byte[][][] lookupTable = new byte[r.getSizeC()][][];
log_message("INFO: After Allocating memmory for image");
for (int i=0; i<num; i++)
{
ImageProcessor ip = r.openProcessors(i)[0];
stack.addSlice("" + (i + 1), ip);
int channel = r.getZCTCoords(i)[1];
lookupTable[channel] = r.get8BitLookupTable();
}
log_message("INFO: After constructing lookup table");
log_message("INFO:Constructing image");
imp = new ImagePlus("Dicom", stack);
colorizedImage = applyLookupTables(r, imp, lookupTable);
r.close();
}
catch(Exception exc)
{
log_message("ERROR an error occurred: " + exc.getMessage());
}
return colorizedImage;
}
Example: Takes 10 to 20 minutes on an 8 core machine with 24GB of memory to
complete setId method
> Data <folder>
>> 0001 <folder>
>> 0002 <folder>
>> 0003 <folder>
>> ...
>> 1000 <folder>
>> 1001 <folder>
>>> File R1.RVG
>> ...
>> 4000 <folder>
Example: Works with no delay
> Data <folder>
>> 0001 <folder>
>> 0002 <folder>
>> 0003 <folder>
>> ...
>> 0050 <folder>
>> 0051 <folder>
>>> File R1.RVG
>> ...
>> 00080 <folder>
The University of Dundee is a registered Scottish Charity, No: SC015096
The University of Dundee is a registered Scottish Charity, No: SC015096
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20140327/cbefa6e3/attachment.html>
More information about the ome-users
mailing list