[ome-users] Error when converting large .svs or .vsi images

Wed Nov 7 10:14:42 GMT 2018

Hi Lawrence,

It will very much depend on the particular files. From some quick tests Ive found that JPEG seemed to come out slightly ahead for the SVS files I tested while LZW performed better at reducing size on VSI files.

If it helps you can find out the original compression and tile sizes using :

showinf -nopix /path/to/inputFile.vsi

You can also set the tile size to be used during conversion using:

bfconvert -compression JPEG -tilex 512 -tiley 512 /path/to/inputFile.vsi output-512x512-tiles.tiff

With Thanks,
David Gault

On 6 Nov 2018, at 17:21, Tarbox, Lawrence <LRTarbox at uams.edu<mailto:LRTarbox at uams.edu>> wrote:

These errors came up while using a compression codec.  Here is the command that I was using:

/home/ltarbox/bftools/bfconvert  -no-upgrade -overwrite -compression JPEG \
  "${FILES[MOAB_JOBARRAYINDEX-1]}" \
  /home/ltarbox/CIR-scans/out/WSI-${MOAB_JOBARRAYINDEX}_S%s.ome.tif

Are you thinking that switching from JPEG to JPEG-2000 or LZW would work better?

The original .svs images use JPEG compression (I think) with a tile size of 256x256

The original .vsi images use JPEG compression with a tile size of 512x512.

I was somewhat hoping that if I requested -compression JPEG bfconvert would simply copy the already-JPEG-compressed tiles from the original images into the .ome.tif images.  But I’m game to try a different compression scheme.

From: David Gault (Staff) <d.gault at dundee.ac.uk<mailto:d.gault at dundee.ac.uk>>
Sent: Tuesday, November 6, 2018 9:14 AM
To: OME User Support List <ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>>; Tarbox, Lawrence <LRTarbox at uams.edu<mailto:LRTarbox at uams.edu>>
Subject: Re: [ome-users] Error when converting large .svs or .vsi images

Hi Lawrence,

It does look like it is related to the file size of the converted file. Setting a compression codec should reduce the required size and might allow for easier conversion.

For example using:
bfconvert -compression JPEG-2000 /path/to/originalFile.svs /path/to/convertedFile.ome.tiff
bfconvert -compression LZW /path/to/originalFile.svs /path/to/convertedFile.ome.tiff

With Thanks,
David Gault

On 5 Nov 2018, at 20:19, Tarbox, Lawrence <LRTarbox at uams.edu<mailto:LRTarbox at uams.edu>> wrote:

I am using 5.9.2, from 3 Sept 2018.

The file sizes that are not converting range from 1.7 GB to 4.2 GB.  The .svs images that were smaller than about 1.3 GB converted without problem.

From: ome-users <ome-users-bounces at lists.openmicroscopy.org.uk<mailto:ome-users-bounces at lists.openmicroscopy.org.uk>> On Behalf Of David Gault (Staff)
Sent: Monday, November 5, 2018 7:10 AM
To: OME User Support List <ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>>
Subject: Re: [ome-users] Error when converting large .svs or .vsi images

Hi Lawrence,

That error looks as though the buffer has been unable to successfully increase its limit for some reason.
What size are the original files you are trying to convert? Also what version of Bio-Formats are you currently using?

With Thanks,
David Gault

On 2 Nov 2018, at 15:21, Tarbox, Lawrence <LRTarbox at uams.edu<mailto:LRTarbox at uams.edu>> wrote:

In converting large .svs or .vsi images, I get the following error:

/home/ltarbox/CIR-scans
54680.svs
SVSReader initializing 54680.svs
Reading IFDs
Populating metadata
Populating OME metadata
[Aperio SVS] -> /home/ltarbox/CIR-scans/out/WSI-2_S%s.ome.tif [OME-TIFF]
Switching to BigTIFF (by file size)
Tile size = 256 x 256
Exception in thread "main" java.lang.IllegalArgumentException
        at java.nio.Buffer.limit(Buffer.java:275)
        at loci.common.NIOFileHandle.buffer(NIOFileHandle.java:636)
        at loci.common.NIOFileHandle.writeSetup(NIOFileHandle.java:642)
        at loci.common.NIOFileHandle.write(NIOFileHandle.java:505)
        at loci.common.NIOFileHandle.write(NIOFileHandle.java:499)
        at loci.common.NIOFileHandle.write(NIOFileHandle.java:487)
        at loci.common.RandomAccessOutputStream.write(RandomAccessOutputStream.java:217)
        at loci.formats.tiff.TiffSaver.writeIFDStrips(TiffSaver.java:987)
        at loci.formats.tiff.TiffSaver.writeImageIFD(TiffSaver.java:451)
        at loci.formats.tiff.TiffSaver.writeImage(TiffSaver.java:401)
        at loci.formats.tiff.TiffSaver.writeImage(TiffSaver.java:277)
        at loci.formats.out.TiffWriter.saveBytes(TiffWriter.java:275)
        at loci.formats.out.OMETiffWriter.saveBytes(OMETiffWriter.java:218)
        at loci.formats.tools.ImageConverter.convertTilePlane(ImageConverter.java:748)
        at loci.formats.tools.ImageConverter.convertPlane(ImageConverter.java:634)
        at loci.formats.tools.ImageConverter.testConvert(ImageConverter.java:563)
        at loci.formats.tools.ImageConverter.main(ImageConverter.java:884)

Smaller images convert fine.

I tripled the memory size (to 40 GB), but it stll fails

Ideas?

Lawrence Tarbox, Ph.D., Dept. of Biomedical Informatics, Univ. of Arkansas for Medical Sciences
mailto:LTarbox at uams.edu +1.314.681-2752

________________________________
Confidentiality Notice: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message.
_______________________________________________
ome-users mailing list
ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openmicroscopy.org.uk_mailman_listinfo_ome-2Dusers&d=DwMFAg&c=27AKQ-AFTMvLXtgZ7shZqsfSXu-Fwzpqk4BoASshREk&r=7PfGBb2SHdaGpGKqCizhwg&m=r40EMcblVZoAtg0jtYbiDCk9hOS9PhHfHWfyTgcvMjE&s=AaJ76JDcXs17ypTw9JmpTKnUrJxLimF9uMYHNayZeuw&e=>

The University of Dundee is a registered Scottish Charity, No: SC015096
_______________________________________________
ome-users mailing list
ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openmicroscopy.org.uk_mailman_listinfo_ome-2Dusers&d=DwMFAg&c=27AKQ-AFTMvLXtgZ7shZqsfSXu-Fwzpqk4BoASshREk&r=FQmnf-oLIEICrzyIHywOlUOTxJWIZf3tvJ8LUzIG0qM&m=3m3AIgFE_QbUasB7kt9_RCytADWjS30QJ6Al7TrUAQA&s=kfZIGmhU35jS7P0cZDCo9Svf9RFFIpFrGtbxDCbc37Y&e=>

The University of Dundee is a registered Scottish Charity, No: SC015096

The University of Dundee is a registered Scottish Charity, No: SC015096
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20181107/72eb2b02/attachment.html>