<HTML>
<HEAD>
<TITLE>Re: [ome-users] ImageProcessorReader.setId performance</TITLE>
</HEAD>
<BODY>
<FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'>Hi Balaji,<BR>
<BR>
Thanks for your insights. We examined the code more closely and through the debugger. We are passing a single full file path to the ImageProcesserReader.setId method. The case that fails specifies /images/001234/R1.RVG while the images folder has more than 4000 sub-folders. We are able to make a workaround by means of copying the same file to /temp/images/001234/R1.RVG. Temp/images folder contains a small number of sub-folders. However this is not ideal as the copy takes time to complete given the size of the images being processed.<BR>
<BR>
You mentioned that the default folder-depth is 4. How do we change it to 1? Is there an API method to disable this behavior? <BR>
<BR>
Many thanks for your help,<BR>
<BR>
--<BR>
M. Samarah<BR>
<a href="msamarah@alumni.fit.edu">msamarah@alumni.fit.edu</a><BR>
<BR>
<HR ALIGN=CENTER SIZE="3" WIDTH="95%"><B>From: </B>Balaji Ramalingam <<a href="b.ramalingam@dundee.ac.uk">b.ramalingam@dundee.ac.uk</a>><BR>
<B>Date: </B>Mon, 24 Mar 2014 10:42:02 +0000<BR>
<B>To: </B>"<a href="msamarah@alumni.fit.edu">msamarah@alumni.fit.edu</a>" <<a href="msamarah@alumni.fit.edu">msamarah@alumni.fit.edu</a>>, "<a href="ome-users@lists.openmicroscopy.org.uk">ome-users@lists.openmicroscopy.org.uk</a>" <<a href="ome-users@lists.openmicroscopy.org.uk">ome-users@lists.openmicroscopy.org.uk</a>><BR>
<B>Subject: </B>Re: [ome-users] ImageProcessorReader.setId performance<BR>
<BR>
Hi,<BR>
<BR>
Your problem seems to be related to the search for Images by bio-formats(loci-tools), within your target directory (imagePath).<BR>
This is how the algorithm works in bio-formats,<BR>
</SPAN></FONT><OL><LI><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'>The target directory(imagePath) is searched for all ‘bioformats-readable’ images.
</SPAN></FONT><LI><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'>-This is the very first step and this is where you are facing problems, I am assuming that you’re ‘</SPAN><SPAN STYLE='font-size:11.5pt'>imagePath’, is a top directory which has images at various depth’s. The default folder-depth at which bio-formats searches is 4 (I.e, it might still be searching for 4 sub-level folder’s within the top directory, unless specified).<BR>
</SPAN></FONT></OL><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:10.5pt'> - The easy alternative is to pre list all the <B>RVG</B> format files within the ‘imagePath' (using file-system search), and read them one at a time (iteratively). I.e imagePath == “Path to an individual image”.<BR>
This basically removes the bio-formats intelligence of grouping similar files (files part of a stack,etc). Am assuming that should not be a problem in your case .<BR>
<BR>
2. </SPAN><SPAN STYLE='font-size:11.5pt'>The other alternative is to use the ImagePlusReader instead of the ImageProcessor as suggested in the code, but this needs testing and playing around. <BR>
<BR>
</SPAN><SPAN STYLE='font-size:10.5pt'>Quoting from the “ImageProcessorReader.java" code ,<BR>
</SPAN></FONT><UL><LI><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:10.5pt'>A low-level reader for {@link ij.process.ImageProcessor} objects.
</SPAN></FONT><LI><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:10.5pt'> * For a higher-level reader that returns {@link ij.ImagePlus} objects,
</SPAN></FONT><LI><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:10.5pt'> * see {@link loci.plugins.in.ImagePlusReader} instead.<BR>
</SPAN></FONT></UL><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11.5pt'><BR>
Please do let us know, how it goes.<BR>
<BR>
Best,<BR>
Balaji<BR>
</SPAN><SPAN STYLE='font-size:10.5pt'><BR>
<BR>
<B>From: </B>Mohammad Samarah <<a href="isamarah@mac.com">isamarah@mac.com</a>><BR>
<B>Reply-To: </B>"<a href="msamarah@alumni.fit.edu">msamarah@alumni.fit.edu</a>" <<a href="msamarah@alumni.fit.edu">msamarah@alumni.fit.edu</a>><BR>
<B>Date: </B>Saturday, 22 March 2014 03:36<BR>
<B>To: </B>"<a href="ome-users@lists.openmicroscopy.org.uk">ome-users@lists.openmicroscopy.org.uk</a>" <<a href="ome-users@lists.openmicroscopy.org.uk">ome-users@lists.openmicroscopy.org.uk</a>><BR>
<B>Subject: </B>[ome-users] ImageProcessorReader.setId performance<BR>
<BR>
ImageProcessorReader.setId performance <BR>
</SPAN><FONT SIZE="2"><SPAN STYLE='font-size:10pt'>We are using loci_tools.jar to read radiographs in RVG format and extract a thumbnail JPEG image. All works as expected, however if the image in question is in a directory whose parent directory contains several thousands subfolders, then the ImageProcesserReader takes minutes to complete the setId() method. Please see attached code. The same image placed in a folder with its parent directory having much fewer subfolders works with no delay. This behavior can also be seen when using loci_tools.jar UI to open the same image. The issue does not seem to be related to the image itself but rather the fact that the containing folder is a child of a directory with thousands of subfolders. Could you please point us to how we can resolve this issue.<BR>
<BR>
Many thanks for your help,<BR>
<BR>
--<BR>
M. Samarah<BR>
<a href="msamarah@alumni.fit.edu">msamarah@alumni.fit.edu</a><BR>
</SPAN></FONT><SPAN STYLE='font-size:11pt'><BR>
<BR>
private ImagePlus readDicomImage(String imagePath)<BR>
{<BR>
ImagePlus imp = null;<BR>
ImagePlus colorizedImage = null;<BR>
<BR>
ImageProcessorReader r = new ImageProcessorReader(new ChannelSeparator(LociPrefs.makeImageReader()));<BR>
try<BR>
{<BR>
log_message("INFO: Examining file " + imagePath);<BR>
<BR>
// the next line takes 10 to 20 minutes to complete when parent folder contains 4000+ subfolders. It takes no time when parent folder has 100 subfolders<BR>
<FONT COLOR="#FF0000">r.setId(imagePath);<BR>
</FONT><BR>
log_message("INFO: Setting file path " + imagePath);<BR>
int num = r.getImageCount();<BR>
log_message("INFO: Image count is" + num);<BR>
int width = r.getSizeX();<BR>
int height = r.getSizeY();<BR>
log_message("INFO: Image Width %d Height %d", width, height);<BR>
ImageStack stack = new ImageStack(width, height);<BR>
log_message("INFO: After constructing image stack");<BR>
byte[][][] lookupTable = new byte[r.getSizeC()][][];<BR>
log_message("INFO: After Allocating memmory for image");<BR>
for (int i=0; i<num; i++) <BR>
{<BR>
ImageProcessor ip = r.openProcessors(i)[0];<BR>
stack.addSlice("" + (i + 1), ip);<BR>
int channel = r.getZCTCoords(i)[1];<BR>
lookupTable[channel] = r.get8BitLookupTable();<BR>
}<BR>
log_message("INFO: After constructing lookup table");<BR>
log_message("INFO:Constructing image");<BR>
imp = new ImagePlus("Dicom", stack);<BR>
<BR>
colorizedImage = applyLookupTables(r, imp, lookupTable);<BR>
r.close();<BR>
<BR>
}<BR>
catch(Exception exc)<BR>
{<BR>
log_message("ERROR an error occurred: " + exc.getMessage());<BR>
}<BR>
return colorizedImage; <BR>
}<BR>
<BR>
<FONT COLOR="#FF0000">Example: Takes 10 to 20 minutes on an 8 core machine with 24GB of memory to complete setId method<BR>
</FONT></SPAN></FONT><BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'><FONT COLOR="#FF0000">Data <folder><BR>
</FONT></SPAN></FONT><BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'><FONT COLOR="#FF0000">0001 <folder><BR>
0002 <folder><BR>
0003 <folder><BR>
...<BR>
1000 <folder><BR>
1001 <folder><BR>
</FONT></SPAN></FONT><BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'><FONT COLOR="#FF0000">File R1.RVG<BR>
</FONT></SPAN></FONT></BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'><FONT COLOR="#FF0000">...<BR>
4000 <folder><BR>
</FONT></SPAN></FONT></BLOCKQUOTE></BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'><BR>
<FONT COLOR="#008000">Example: Works with no delay<BR>
</FONT></SPAN></FONT><BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'><FONT COLOR="#008000">Data <folder><BR>
</FONT></SPAN></FONT><BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'><FONT COLOR="#008000">0001 <folder><BR>
0002 <folder><BR>
0003 <folder><BR>
...<BR>
0050 <folder><BR>
0051 <folder><BR>
</FONT></SPAN></FONT><BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'><FONT COLOR="#008000">File R1.RVG<BR>
</FONT></SPAN></FONT></BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'><FONT COLOR="#008000">...<BR>
00080 <folder><BR>
</FONT></SPAN></FONT></BLOCKQUOTE></BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'><BR>
<BR>
<BR>
</SPAN><FONT SIZE="2"><SPAN STYLE='font-size:10pt'>The University of Dundee is a registered Scottish Charity, No: SC015096</SPAN></FONT><SPAN STYLE='font-size:11pt'><BR>
</SPAN></FONT>
</BODY>
</HTML>