[ome-users] Network drive slowness
Melissa Linkert
melissa at glencoesoftware.com
Thu Dec 11 09:11:56 GMT 2014
Hi Paul and Niko,
> >> Would this bug also explain slowness in the OMERO importer while
> >> importing files which are located on a network drive (in Windows)?
It's very possible - the easiest way to know for sure is to compare the
import time for the same file on a network drive vs. on a local drive.
> > I doubt this is related - as I understand it, the (current) import process
> > using OMERO.insight just copies the image file(s) onto the server before it
> > is processing them. So the first time the BioFormats kicks in there is
> > *after* the actual transfer and therefore the network drive issue should
> > not apply here.
That's mostly correct, but Bio-Formats is still used client-side at
import time send metadata to the server and to determine which files
need to be uploaded. Bio-Formats definitely would be accessing the
files as they are on the network drive.
> You're probably right that the File.isHidden() in BioFormats isn't called
> in the OMERO importer at the client side. However, from quickly grepping
> through the source of openmicroscopy/components/insight I found several
> calls to File.isHidden(). I was wondering if these are also meant to be
> optimized as part of this issue.
We weren't initially considering the OMERO code base, but once this
issue is resolved in Bio-Formats it likely makes sense to use the same
fix in OMERO.
Regards,
-Melissa
P.S. Niko, you are now CC'd on the ticket as well.
On Thu, Dec 11, 2014 at 09:07:41PM +0100, Paul van Schayck wrote:
> Hi Niko,
>
> You're probably right that the File.isHidden() in BioFormats isn't called
> in the OMERO importer at the client side. However, from quickly grepping
> through the source of openmicroscopy/components/insight I found several
> calls to File.isHidden(). I was wondering if these are also meant to be
> optimized as part of this issue.
>
> What we experience in the OMERO importer is for very low latency network
> drives that the import is relatively slow (compared to file copy). For
> higher latency network drives the import is slow and the importer UI
> becomes unresponsive (although the import continues). If you want me to
> quantify these experiences, I could do that.
>
> Thanks,
>
> Paul
>
>
> On Thu, Dec 11, 2014 at 8:22 PM, Niko Ehrenfeuchter <
> nikolaus.ehrenfeuchter at unibas.ch> wrote:
>
> > Hi Paul,
> >
> > I doubt this is related - as I understand it, the (current) import process
> > using OMERO.insight just copies the image file(s) onto the server before it
> > is processing them. So the first time the BioFormats kicks in there is
> > *after* the actual transfer and therefore the network drive issue should
> > not apply here.
> >
> > I might be wrong, though...
> >
> > Cheers
> > Niko
> >
> > On 11.12.2014 17:29, Paul van Schayck wrote:
> >
> >> Hi Melissa,
> >>
> >> Would this bug also explain slowness in the OMERO importer while
> >> importing files which are located on a network drive (in Windows)?
> >>
> >> Thanks,
> >>
> >> Paul
> >>
> >> On Thu, Dec 11, 2014 at 3:26 PM, Niko Ehrenfeuchter
> >> <nikolaus.ehrenfeuchter at unibas.ch
> >> <mailto:nikolaus.ehrenfeuchter at unibas.ch>> wrote:
> >>
> >> Hi Melissa,
> >>
> >> could you please add me to the CC list on that ticket as well? We're
> >> also suffering from the performance issue when opening files from
> >> network shares, especially on Windows.
> >>
> >> Cheers
> >> Niko
> >>
> >>
> >> On 11.12.2014 03:34, Melissa Linkert wrote:
> >>
> >> Hi Rafael and Lorenz,
> >>
> >> You are both now CC'd on ticket 6586, and will receive an
> >> automated
> >> email with "[OME] #6586" in the subject line each time the
> >> ticket is updated.
> >>
> >> Regards,
> >> -Melissa
> >>
> >> On Thu, Dec 11, 2014 at 12:55:24PM +0100, Maier, Lorenz wrote:
> >>
> >> Hi Rafael,
> >>
> >> probably Zen Blue reads only the planes and zoom levels that
> >> it actually displays.
> >>
> >> Best,
> >> Lorenz
> >>
> >> PS: please CC me on that ticket!
> >>
> >> -----Ursprüngliche Nachricht-----
> >> Von: ome-users
> >> [mailto:ome-users-bounces at __lists.openmicroscopy.org.uk
> >> <mailto:ome-users-bounces at lists.openmicroscopy.org.uk>] Im
> >> Auftrag von r b
> >> Gesendet: Donnerstag, 11. Dezember 2014 01:20
> >> An: Melissa Linkert
> >> Cc: ome-users at lists.__openmicroscopy.org.uk
> >> <mailto:ome-users at lists.openmicroscopy.org.uk>
> >> Betreff: Re: [ome-users] Network drive slowness
> >>
> >> Hi Melissa and list,
> >>
> >> I would like to be automatically notified of updates about
> >> that ticket.
> >>
> >> Is there an obvious reason why the same file opens so much
> >> faster in Zen Blue 2012?
> >>
> >> Thank you,
> >>
> >> Rafael Buono
> >>
> >> On Wed, Dec 10, 2014 at 5:45 PM, Melissa Linkert
> >> <melissa at glencoesoftware.com
> >> <mailto:melissa at glencoesoftware.com>> wrote:
> >>
> >> Hi Rafael,
> >>
> >> I did the test with a 460MB czi file.
> >>
> >> If I copy it to the desktop and then open it, it
> >> takes ~20 seconds in Fiji.
> >> If I open it from the network drive:
> >> 1st time: 10 minutes
> >> 2nd time: 2 minutes
> >> 3rd time: ~20 seconds
> >> 4th time: ~20 seconds
> >>
> >> Opening the same file from the network storage or
> >> directly from the
> >> desktop takes less than 4 seconds with Zen Blue 2012.
> >>
> >>
> >> Thank you for the additional information.
> >>
> >> 20 seconds to open a 460MB file is within the range of
> >> what I would
> >> expect - even more so if the images are compressed
> >> within the file.
> >> The fact that only the first opening from the network
> >> drive takes 10
> >> minutes suggests that this is not related to file format
> >> itself. I
> >> strongly suspect that this ticket on our issue tracking
> >> system is the
> >> problem that you are experiencing:
> >>
> >> http://trac.openmicroscopy.__org/ome/ticket/6586
> >>
> >> <http://trac.openmicroscopy.org/ome/ticket/6586>
> >>
> >> If you would like to be automatically notified of
> >> updates to that
> >> ticket, please let me know.
> >>
> >> Regards,
> >> -Melissa
> >>
> >> On Mon, Dec 08, 2014 at 06:33:51PM -0600, r b wrote:
> >>
> >> Hi Melissa and list,
> >>
> >> I did the test with a 460MB czi file.
> >>
> >> If I copy it to the desktop and then open it, it
> >> takes ~20 seconds in Fiji.
> >> If I open it from the network drive:
> >> 1st time: 10 minutes
> >> 2nd time: 2 minutes
> >> 3rd time: ~20 seconds
> >> 4th time: ~20 seconds
> >>
> >> Opening the same file from the network storage or
> >> directly from the
> >> desktop takes less than 4 seconds with Zen Blue 2012.
> >>
> >>
> >> About the "Bio-Formats Import Options" window. I
> >> don't go through it. I go:
> >> File>Open...> then I double click the file and it
> >> opens it without
> >> asking anything else..
> >> If I use the Bio-Formats Importer route to open the
> >> file from the
> >> desktop, selecting:
> >> -View stack with: Hyperstack
> >> -Stack order: XYCZT
> >> -Display metadata
> >> -Display OME-XML metadata
> >> -Display ROIs
> >>
> >> I still get ~20 seconds for it to open.
> >>
> >>
> >> Thank you
> >>
> >> Rafael Buono
> >>
> >> On Mon, Dec 8, 2014 at 12:27 PM, Melissa Linkert
> >> <melissa at glencoesoftware.com
> >> <mailto:melissa at glencoesoftware.com>> wrote:
> >>
> >> Hi Rafael,
> >>
> >> First, as an example, I have a .czi of
> >> ~210MB that opens:
> >> - ZEN Blue Lite 2012 loading from network
> >> drive: about 2 seconds
> >> - Copying from network drive and pasting to
> >> the Desktop: about 4
> >> seconds
> >> - Fiji loading from network drive: 30 seconds
> >>
> >> In the ImageJ list I was informed that
> >> taking longer to load czi
> >> files might be a widespread issue with czi
> >> files.
> >>
> >>
> >> How long does it take to read the file using
> >> Fiji, after it has
> >> been copied to the Desktop? That would help us
> >> to know if this is
> >> strictly due to the network file system, or if
> >> the procedure for
> >> reading the file is too slow.
> >>
> >> It would also be useful to know which options
> >> you have selected in
> >> the "Bio-Formats Import Options" window.
> >>
> >> The second problem was found while testing
> >> the speeds above. If I
> >> put the same file one folder deeper and try
> >> to open it in Fiji,
> >> the time grows from 30 seconds to 12 minutes.
> >>
> >> I am one of those people who organizes
> >> things by creating folder,
> >> inside folder, inside folder.
> >> In the following structure:
> >> O:\Folder1\Folder2\Folder3\__
> >> Folder4\Folder5\Folder6\__Folder7
> >>
> >> If I have the czi file in Folder6, it takes
> >> 30 seconds to open. If
> >> I move it to Folder7, it takes 12 minutes to
> >> open.
> >>
> >> Is there an obvious reason for this increase
> >> in time?
> >>
> >>
> >> Having more folders can cause subtle performance
> >> degradation, but
> >> typically not as much as you describe. If you
> >> open the file in
> >> Folder7 twice in a row, does it take 12 minutes
> >> both times?
> >>
> >> Regards,
> >> -Melissa
> >>
> >> On Fri, Dec 05, 2014 at 11:05:21AM -0600, r b
> >> wrote:
> >>
> >> Hi all,
> >>
> >> I recently got our lab workstation connected
> >> through the 1Gbps
> >> network to the building's server. Among
> >> other reasons, I was
> >> hoping to process images directly from the
> >> network drive. However,
> >> even with the upgrade to 1Gbps, loading
> >> images in Fiji is still
> >> slow. I have encountered two problems and I
> >> am not sure they are related.
> >>
> >> First, as an example, I have a .czi of
> >> ~210MB that opens:
> >> - ZEN Blue Lite 2012 loading from network
> >> drive: about 2 seconds
> >> - Copying from network drive and pasting to
> >> the Desktop: about 4
> >> seconds
> >> - Fiji loading from network drive: 30 seconds
> >>
> >> In the ImageJ list I was informed that
> >> taking longer to load czi
> >> files might be a widespread issue with czi
> >> files.
> >>
> >> The second problem was found while testing
> >> the speeds above. If I
> >> put the same file one folder deeper and try
> >> to open it in Fiji,
> >> the time grows from 30 seconds to 12 minutes.
> >>
> >> I am one of those people who organizes
> >> things by creating folder,
> >> inside folder, inside folder.
> >> In the following structure:
> >> O:\Folder1\Folder2\Folder3\__
> >> Folder4\Folder5\Folder6\__Folder7
> >> If I have the czi file in Folder6, it takes
> >> 30 seconds to open. If
> >> I move it to Folder7, it takes 12 minutes to
> >> open.
> >>
> >> Is there an obvious reason for this increase
> >> in time?
> >>
> >> Thank you
> >>
> >> Rafael Buono
> >>
> >> PS:
> >> Some info about my system below:
> >> Java information:
> >> os.arch => amd64
> >> os.name <http://os.name> => Windows 7
> >> os.version => 6.1
> >> java.version => 1.6.0_24
> >> java.vendor => Sun Microsystems Inc.
> >> java.runtime.name <http://java.runtime.name>
> >> => Java(TM) SE Runtime Environment
> >> java.runtime.version => 1.6.0_24-b07
> >> java.vm.name <http://java.vm.name> => Java
> >> HotSpot(TM) 64-Bit Server VM
> >> java.vm.version => 19.1-b02
> >> java.vm.vendor => Sun Microsystems Inc.
> >> java.vm.info <http://java.vm.info> => mixed
> >> mode
> >> java.awt.graphicsenv =>
> >> sun.awt.__Win32GraphicsEnvironment
> >> java.specification.name
> >> <http://java.specification.name> => Java
> >> Platform API Specification
> >> java.specification.version => 1.6
> >> sun.cpu.endian => little
> >> sun.desktop => windows
> >> file.separator => \
> >>
> >> The up-to-date check says: UP_TO_DATE
> >>
> >> Information relevant to JAVA_HOME related
> >> problems:
> >>
> >> JAVA_HOME is set to:
> >> C:\Users\ADMINI~1\Desktop\__
> >> Fiji.app/java/win64/jdk1.6.0___24//jre
> >> imagej.dir =>
> >> C:\Users\ADMINI~1\Desktop\__Fiji.app
> >>
> >> Information about the version of each plugin:
> >>
> >> Activated update sites:
> >> ImageJ: http://update.imagej.net/ (last
> >> check:20141126151327)
> >> Fiji: http://fiji.sc/update/ (last
> >> check:20141125101145) MOSAIC
> >> ToolSuite:
> >> http://mosaic.mpi-cbg.de/__
> >> Downloads/update/Fiji/__MosaicToolsuite/
> >> <http://mosaic.mpi-cbg.de/
> >> Downloads/update/Fiji/MosaicToolsuite/>
> >> (last
> >> check:20141105181755)
> >>
> >>
> >>
> >> Bio-Formats information:
> >> Release 5.0.6, revision 33b1150, built 10
> >> November 2014.
> >> ______________________________
> >> ___________________
> >> ome-users mailing list
> >> ome-users at lists.__openmicroscopy.org.uk
> >> <mailto:ome-users at lists.openmicroscopy.org.uk
> >> >
> >> http://lists.openmicroscopy.__
> >> org.uk/mailman/listinfo/ome-__users
> >> <http://lists.openmicroscopy.
> >> org.uk/mailman/listinfo/ome-users>
> >>
> >> _________________________________________________
> >> ome-users mailing list
> >> ome-users at lists.__openmicroscopy.org.uk
> >> <mailto:ome-users at lists.openmicroscopy.org.uk>
> >> http://lists.openmicroscopy.__org.uk/mailman/listinfo/ome-__
> >> users
> >> <http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-
> >> users>
> >> _________________________________________________
> >> ome-users mailing list
> >> ome-users at lists.__openmicroscopy.org.uk
> >> <mailto:ome-users at lists.openmicroscopy.org.uk>
> >> http://lists.openmicroscopy.__org.uk/mailman/listinfo/ome-__
> >> users
> >> <http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-
> >> users>
> >>
> >> _________________________________________________
> >> ome-users mailing list
> >> ome-users at lists.__openmicroscopy.org.uk
> >> <mailto:ome-users at lists.openmicroscopy.org.uk>
> >> http://lists.openmicroscopy.__org.uk/mailman/listinfo/ome-__users
> >> <http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users>
> >>
> >>
> >> --
> >> Niko Ehrenfeuchter | Image Analysis Specialist | Biozentrum,
> >> University of Basel | Klingelbergstr. 50/70 | CH-4056 Basel
> >> Phone: +41 (61) 26 72673 | nikolaus.ehrenfeuchter at unibas.__ch
> >> <mailto:nikolaus.ehrenfeuchter at unibas.ch> | www.biozentrum.unibas.ch
> >> <http://www.biozentrum.unibas.ch> |
> >> www.microscopynetwork.unibas.__ch
> >> <http://www.microscopynetwork.unibas.ch>
> >>
> >> _________________________________________________
> >> ome-users mailing list
> >> ome-users at lists.__openmicroscopy.org.uk
> >> <mailto:ome-users at lists.openmicroscopy.org.uk>
> >> http://lists.openmicroscopy.__org.uk/mailman/listinfo/ome-__users
> >> <http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users>
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> ome-users mailing list
> >> ome-users at lists.openmicroscopy.org.uk
> >> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
> >>
> >>
> > --
> > Niko Ehrenfeuchter | Image Analysis Specialist | Biozentrum, University of
> > Basel | Klingelbergstr. 50/70 | CH-4056 Basel
> > Phone: +41 (61) 26 72673 | nikolaus.ehrenfeuchter at unibas.ch |
> > www.biozentrum.unibas.ch | www.microscopynetwork.unibas.ch
> >
More information about the ome-users
mailing list