[ome-devel] Auto-detecting whether to export as BigTIFF

[Melissa Linkert]

Hi Curtis,

> Below is a recent discussion on fiji-users about exporting very large TIFF
> files (i.e., BigTIFF needed).
> 
> I looked at the source code for TiffWriter and think it would be doable to
> automatically turn on the isBigTiff flag if the total size of all planes to
> be written exceeds the 4GB threshold. This check would happen once, just
> prior to calling tiffSaver.writeHeader(), so the correct header type gets
> written. The only tricky bit is at the edge case where the IFD metadata
> might push the file over the limit. One (hacky) way around this issue would
> be to switch to BigTIFF when the pixel data is even close: say, over 3.5GB.
> 
> Do you agree this would be a good idea? If so, can we file a ticket for it?

There has been a ticket for this for a while:

http://trac.openmicroscopy.org.uk/ome/ticket/6589

...I just haven't had the time to finish it.

Regards,
-Melissa

On Tue, Apr 10, 2012 at 04:42:16PM -0500, Curtis Rueden wrote:
> Hi everyone,
> 
> Below is a recent discussion on fiji-users about exporting very large TIFF
> files (i.e., BigTIFF needed).
> 
> I looked at the source code for TiffWriter and think it would be doable to
> automatically turn on the isBigTiff flag if the total size of all planes to
> be written exceeds the 4GB threshold. This check would happen once, just
> prior to calling tiffSaver.writeHeader(), so the correct header type gets
> written. The only tricky bit is at the edge case where the IFD metadata
> might push the file over the limit. One (hacky) way around this issue would
> be to switch to BigTIFF when the pixel data is even close: say, over 3.5GB.
> 
> Do you agree this would be a good idea? If so, can we file a ticket for it?
> 
> Thanks,
> Curtis
> 
> 
> ---------- Forwarded message ----------
> From: Curtis Rueden <ctrueden at wisc.edu>
> Date: 2012/4/10
> Subject: Re: [fiji-users] Large tiff files
> To: Sébastien Tosi <sebastien.tosi at gmail.com>
> Cc: Fiji-users <fiji-users at googlegroups.com>
> 
> 
> Hi Sébastien,
> 
> 
> 
> Thank you Curtis, I tried the Bio-formats exporter but it failed with an
> > error message telling that the BigTiff flag should be enabled. Do you know
> > how to configure it? Anyway I decided to export the stack as a sequence of
> > images to import it in the other program.
> >
> 
> Hmm, sorry about that. I thought the Exporter was smart enough to switch to
> BigTIFF as needed. I will talk to the other Bio-Formats developers about
> making it smarter (should be easy).
> 
> Regards,
> Curtis
> 
> 
> 
> 2012/4/10 Sébastien Tosi <sebastien.tosi at gmail.com>
> 
> > Thank you Curtis, I tried the Bio-formats exporter but it failed with an
> > error message telling that the BigTiff flag should be enabled. Do you know
> > how to configure it? Anyway I decided to export the stack as a sequence of
> > images to import it in the other program.
> >
> > Sébastien
> >
> >
> > On 5 April 2012 18:36, Curtis Rueden <ctrueden at wisc.edu> wrote:
> >
> >> Hi Sébastien,
> >>
> >> It seems the limit is around 4.2 GB, all
> >>> file above this limit suffering the problem.
> >>>
> >>
> >> TIFFs have an inherent 32-bit limit (~4GB as you note). Try using the
> >> Bio-Formats Exporter to write out your TIFFs, which will use the 64-bit
> >> BigTIFF format as needed when exporting.
> >>
> >> Regards,
> >> Curtis
> >>
> >>
> >> 2012/4/5 Sébastien Tosi <sebastien.tosi at gmail.com>
> >>
> >>> Hello,
> >>>
> >>> Large z-stack are saved improperly as tiff files by the built-in
> >>> ImageJ tiff filter. The stacks can be read by ImageJ but any other
> >>> software fail to open them... inspecting the header of the files (in
> >>> Matlab) it shows that only a single plane is present in the file. I am
> >>> working with single frame 8-bit series with around 500 z planes and
> >>> 3000x3000 pixels per plane. It seems the limit is around 4.2 GB, all
> >>> file above this limit suffering the problem.
> >>>
> >>> Sébastien
> >>>
> >>> --
> >>> Please avoid top-posting, and please make sure to reply-to-all!
> >>>
> >>> Mailing list web interface: http://groups.google.com/group/fiji-users
> >>>
> >>
> >>
> >

> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel