[ome-users] [SCIFIO] How to fix bug: BigTiff not handled well by Bio-Formats or SCIFIO

[Curtis Rueden]

Hi Adam,

> I'll try out the new SCIFIO and let you know what happens.

Great! Just to be clear: SCIFIO is not used by default in Fiji... yet.
Coming soon. In the meantime, you can either:

Try with ImageJ2 (experimental!):
- Help > Switch to Modern Mode
- File > Save As...

Write a program which uses SCIFIO's ImgSaver API:
-
https://github.com/scifio/scifio/blob/scifio-0.11.0/scifio/src/main/java/io/scif/img/ImgSaver.java

Unfortunately there isn't a great tutorial yet for me to point you at --
this stuff is still a bit bleeding edge. But we're happy to answer any
questions you have if you reply back to the list!

Regards,
Curtis


On Mon, Apr 21, 2014 at 4:15 PM, Adam Packer <a.packer at ucl.ac.uk> wrote:

> Hi Curtis,
>
> Thanks so much for replying to my message in such detail!  I'll try out
> the new SCIFIO and let you know what happens.
>
> Best
> Adam
>
>
> On Mon, Apr 21, 2014 at 10:07 PM, Curtis Rueden <ctrueden at wisc.edu> wrote:
>
>> Hi Adam,
>>
>> [CCing the SCIFIO and OME users mailing lists, as this discussion will be
>> of interest to others.]
>>
>> > This bug:
>> > https://groups.google.com/forum/#!topic/fiji-users/7qLsi-V-jvs
>> > (Bio-Formats does not export a tiff using the BigTIFF library
>> > automatically when a file will be bigger than 4GB) is still a bug in
>> > the latest version of FIJI and the latest build of Bio-Formats
>> > installed separately in ImageJ, though you hoped to fix it in 2012.
>>
>> Right. My apologies that that problem has continued to exist for so long.
>> IMO, Bio-Formats should auto-detect whether the output file might be large
>> enough to be need BigTIFF, and act accordingly. But the current 5.0.1
>> release does not.
>>
>> > You mention in late 2013 on stackoverflow that SCIFIO should solve
>> > this problem:
>> >
>> http://stackoverflow.com/questions/18986368/java-how-to-write-a-very-large-20-000x20-000-px-or-larger-tif-image/19034370#19034370
>> > but that does not yet seem to be the case as FIJI, for which SCIFIO
>> > claims to do the I/O, still has this bug when trying to write out a
>> > tiff >4GB.
>>
>>  Actually, that SO question is about a different issue: huge planes. The
>> BigTIFF format is actually needed even for more modest size planes, when
>> you simply have many of them, which result in a TIFF file of more than 4GB
>> in size. Regular TIFF uses 32-bit offsets, so as soon as the TIFF file gets
>> over 4GB, there is no way to indicate that an offset should go deeper into
>> the file than that. BigTIFF solves this problem by using 64-bit offsets.
>>
>> > Changing Line 132 in TIffWriter.JAVA to say: isBigTiff = true; would
>> > simply force writing BigTIFFs all the time, which would solve the
>> > problem for me (although I realize cause other problems for others!)
>>
>> Well, there is also an API method in TiffWriter: setBigTiff(true). But of
>> course you have to write a little program in order to use it; there is no
>> built-in method in the ImageJ user interface to enable that option. And it
>> is off by default.
>>
>> > However, I cannot find a way to change this java file and propagate
>> > the change to within ImageJ/FIJI.  When I compile the new
>> > TiffWriter.java in ImageJ I get hundreds of errors (see attached log
>> > from ImageJ).
>>
>> I am not sure why you are seeing those errors. I would need to know more
>> about how you were trying to build the system. The best ways to build
>> Bio-Formats are with either Ant ("ant jars" from top level works) or Maven
>> (e.g., "mvn" from top level).
>>
>> > What am I missing and/or is there any simple way to solve this problem
>> > and/or fix the bug?
>>
>> Earlier today, Mark Hiner released a new version of SCIFIO that
>> automatically enables BigTIFF as needed:
>>     http://scif.io/pipermail/scifio/2014-April/000067.html
>>     http://maven.imagej.net/content/groups/public/io/scif/scifio/0.11.0/
>>
>> We are close to having Fiji use SCIFIO for its image I/O by default; once
>> that happens, opening and saving large images should "just work" even when
>> they are larger than 4GB in size.
>>
>> Regards,
>> Curtis
>>
>>
>> On Wed, Apr 16, 2014 at 4:03 PM, Adam Packer <a.packer at ucl.ac.uk> wrote:
>>
>>> Dear Curtis
>>>
>>> This bug:
>>> https://groups.google.com/forum/#!topic/fiji-users/7qLsi-V-jvs
>>>  (Bio-Formats does not export a tiff using the BigTIFF library
>>> automatically when a file will be bigger than 4GB) is still a bug in the
>>> latest version of FIJI and the latest build of Bio-Formats installed
>>> separately in ImageJ, though you hoped to fix it in 2012.
>>>
>>> You mention in late 2013 on stackoverflow that SCIFIO should solve this
>>> problem:
>>>
>>> http://stackoverflow.com/questions/18986368/java-how-to-write-a-very-large-20-000x20-000-px-or-larger-tif-image/19034370#19034370
>>> but that does not yet seem to be the case as FIJI, for which SCIFIO
>>> claims to do the I/O, still has this bug when trying to write out a tiff
>>> >4GB.
>>>
>>> Changing Line 132 in TIffWriter.JAVA to say:
>>> isBigTiff = true;
>>> would simply force writing BigTIFFs all the time, which would solve the
>>> problem for me (although I realize cause other problems for others!)
>>>
>>> However, I cannot find a way to change this java file and propagate the
>>> change to within ImageJ/FIJI.  When I compile the new TiffWriter.java in
>>> ImageJ I get hundreds of errors (see attached log from ImageJ).
>>>
>>> What am I missing and/or is there any simple way to solve this problem
>>> and/or fix the bug?
>>>
>>> All the best,
>>> Adam Packer
>>> University College London
>>>
>>
>>
>> _______________________________________________
>> SCIFIO mailing list
>> SCIFIO at scif.io
>> http://scif.io/mailman/listinfo/scifio
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20140421/2968f2d8/attachment.html>