[ome-devel] Auto-detecting whether to export as BigTIFF

[Curtis Rueden]

Hi everyone,

Below is a recent discussion on fiji-users about exporting very large TIFF
files (i.e., BigTIFF needed).

I looked at the source code for TiffWriter and think it would be doable to
automatically turn on the isBigTiff flag if the total size of all planes to
be written exceeds the 4GB threshold. This check would happen once, just
prior to calling tiffSaver.writeHeader(), so the correct header type gets
written. The only tricky bit is at the edge case where the IFD metadata
might push the file over the limit. One (hacky) way around this issue would
be to switch to BigTIFF when the pixel data is even close: say, over 3.5GB.

Do you agree this would be a good idea? If so, can we file a ticket for it?

Thanks,
Curtis


---------- Forwarded message ----------
From: Curtis Rueden <ctrueden at wisc.edu>
Date: 2012/4/10
Subject: Re: [fiji-users] Large tiff files
To: Sébastien Tosi <sebastien.tosi at gmail.com>
Cc: Fiji-users <fiji-users at googlegroups.com>


Hi Sébastien,



Thank you Curtis, I tried the Bio-formats exporter but it failed with an
> error message telling that the BigTiff flag should be enabled. Do you know
> how to configure it? Anyway I decided to export the stack as a sequence of
> images to import it in the other program.
>

Hmm, sorry about that. I thought the Exporter was smart enough to switch to
BigTIFF as needed. I will talk to the other Bio-Formats developers about
making it smarter (should be easy).

Regards,
Curtis



2012/4/10 Sébastien Tosi <sebastien.tosi at gmail.com>

> Thank you Curtis, I tried the Bio-formats exporter but it failed with an
> error message telling that the BigTiff flag should be enabled. Do you know
> how to configure it? Anyway I decided to export the stack as a sequence of
> images to import it in the other program.
>
> Sébastien
>
>
> On 5 April 2012 18:36, Curtis Rueden <ctrueden at wisc.edu> wrote:
>
>> Hi Sébastien,
>>
>> It seems the limit is around 4.2 GB, all
>>> file above this limit suffering the problem.
>>>
>>
>> TIFFs have an inherent 32-bit limit (~4GB as you note). Try using the
>> Bio-Formats Exporter to write out your TIFFs, which will use the 64-bit
>> BigTIFF format as needed when exporting.
>>
>> Regards,
>> Curtis
>>
>>
>> 2012/4/5 Sébastien Tosi <sebastien.tosi at gmail.com>
>>
>>> Hello,
>>>
>>> Large z-stack are saved improperly as tiff files by the built-in
>>> ImageJ tiff filter. The stacks can be read by ImageJ but any other
>>> software fail to open them... inspecting the header of the files (in
>>> Matlab) it shows that only a single plane is present in the file. I am
>>> working with single frame 8-bit series with around 500 z planes and
>>> 3000x3000 pixels per plane. It seems the limit is around 4.2 GB, all
>>> file above this limit suffering the problem.
>>>
>>> Sébastien
>>>
>>> --
>>> Please avoid top-posting, and please make sure to reply-to-all!
>>>
>>> Mailing list web interface: http://groups.google.com/group/fiji-users
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20120410/54f6d420/attachment.html>