[ome-users] Big File questions

[Wood, Christopher]

Will,

Thanks for the response.

Comparing the ome-tiff and the tif in Insight, moving to the next z-plane/time point it seems (I haven't measured it) that the tiff, imported as a big image is 2-3 times slower than the ome-tiff imported normally. The images take a long time because of their size, but the difference is noticeable.

I also forgot to mention that the ome-tiff imported normally does not need the long time to render the thumbnails.

Thanks again,

Chris

From: Will Moore [mailto:will at lifesci.dundee.ac.uk]
Sent: Thursday, July 07, 2011 4:00 PM
To: Wood, Christopher
Cc: ome-users at lists.openmicroscopy.org.uk
Subject: Re: [ome-users] Big File questions

Hi Chris,

 I'll let others handle the more tricky questions, but I can help with some points:

The "Big Image" cutoff is currently a plane size of 3000 x 3000 pixels (or equivalent total number of pixels). This is the cut-off for clients, when deciding whether to use a Big Image (Google-Maps) viewer.
However, the import process that is used for "Big Image" file formats is always used for those formats, regardless of size.
These formats include tiff, png, svs etc. But not ome-tiff.
We use this strategy for various "Big Image" formats (svs etc) as well as other "graphics" formats (png etc) to avoid the increase in file size when these compressed formats are expanded to raw pixel data. Instead we create a Pyramid of jpeg-2000 compressed data.

I assume the delay when Insight first accesses the thumbnails is due to the creation of thumbnails, which is quite expensive from pyramids.

Hope this helps,

   Will.



On 7 Jul 2011, at 21:32, Wood, Christopher wrote:


Hi (again),

Our omero server (4.3) is experiencing some odd behavior that seems to be related to big images and importing/rendering them.

Some large tif images, around 2560x2160x100z,  where imported into a dataset - the pyramid/big image strategy is used. It takes a while to calculate the pyramids, but finishes and it seems fine.

When trying to view these afterword (a day later), if the dataset is clicked in Insight (on a client computer) and insight tries to show the thumbnails, the cpu on the server goes to 100% and the logs show lines like:
2011-07-07 11:14:17,675 INFO  [                 org.perf4j.TimingLogger] (l.Server-1) start[1310055257626] time[49] tag[omero.loadPixelsByImage]
2011-07-07 11:14:17,721 INFO  [                 org.perf4j.TimingLogger] (l.Server-1) start[1310055257675] time[46] tag[omero.loadRenderingSettingsByUser]
2011-07-07 11:14:17,744 INFO  [             ome.io.nio.FilePathResolver] (l.Server-1) Metadata only file, resulting path: /OMERO/Files/Dir-003/3311
2011-07-07 11:14:17,744 INFO  [                ome.io.nio.PixelsService] (l.Server-1) Using Pyramid BfPixelBuffer: /OMERO/Pixels/Dir-001/1788_pyramid
2011-07-07 11:14:17,802 INFO  [       loci.formats.in.MinimalTiffReader] (l.Server-1) Reading IFDs
2011-07-07 11:14:18,899 INFO  [       loci.formats.in.MinimalTiffReader] (l.Server-1) Populating metadata
2011-07-07 11:15:00,019 INFO  [ ome.services.blitz.fire.SessionManagerI] (3-thread-4) Performing requestHeartbeats
2011-07-07 11:18:00,016 INFO  [ ome.services.blitz.fire.SessionManagerI] (3-thread-1) Performing requestHeartbeats

(see attached log for more)

The dataset contains 12 images, 6 of those are big images. The logs show this happens for each large image and it is a very long process. After this process if insight is closed and reopened, the thumbnails display fine. Is this the expected behavior?  If insight would give some feedback, about what was going on in the background that would be good.  Does this processing need to performed for each user that looks at the images?

Insight does not display these images in the "Google Maps" way; it is not tiled and the overview is not shown in the corner. I have importer larger (xy) files that do use the "Google Maps" way.

I saved the original tif file as an ome-tiff using ImageJ/Bio-Formats and imported it. The regular import strategy was used for the ome-tiff. Why the difference with the same size of file?

I can upload the images (> 1GB) if that would be helpful.

Thanks
Chris
<B.log>
_______________________________________________
ome-users mailing list
ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users

William Moore
Wellcome Trust Centre for Gene Regulation & Expression
College of Life Sciences
MSI/WTB/JBC Complex
University of Dundee
Dow Street
Dundee  DD1 5EH
United Kingdom

Phone 01382 386364
http://openmicroscopy.org





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20110707/ad7dd72b/attachment.html>