[ome-users] Support for .FLI files from Capture software (HiCAM FLUO high-speed camera)

Mon Jan 14 19:17:30 GMT 2019

Hello Sebastien,

The GB file size problem persisted, and we think the following pull request
will fix it: https://github.com/openmicroscopy/bioformats/pull/3303

If you have any reconmondations please let me know.

Regarding implementing the unsigned 12 bit data format, please let me know
if I can be of any further help.

Many thabks for your support.

Best,
Johan

Op ma 14 jan. 2019 10:57 schreef Sebastien Besson (Staff) <
s.besson at dundee.ac.uk:

> Hi Johan and Aaron,
>
> Happy New Year to you guys too. I am reincluding the Users mailing lists
> in this
> conversation.
>
> On 11 Jan 2019, at 08:11, Johan Herz <johan at lambertinstruments.com> wrote:
>
> Hello Sebastien and Ponti,
>
> Thank you for the follow-up, all to you too a very good 2019!
>
> @Aaron, yes I will be available for testing, my pleasure.
>
> @Sebastien,  The FLI format for the Hi-speed is not yet very well
> documented, it is basically based on the original FLI format with a few new
> data types. We also support color cameras. But from our experience, if we
> added a header entry that the bio-formats FLI reader did not recognize, it
> would just ignore it. Unless it makes a collision with another key entry.
>
> So basically the main issues currently are the Data type and the 2gb File
> size max.
> It will become possible to open very big files virtually, right? Because
> we can easily make files that are 50GB or bigger so it can most of the time
> never fit the RAM memory.
>
>
>
> About 2GB, I assume you are referring to the issue described in this
> section of the
> technical documentation [1]? Briefly, we are routinely dealing with
> datasets in the
> GB-TB magnitude so file size per se should not be a problem.
>
> Am I correct in assuming you are loading data using the ImageJ/Fiji
> Bio-Formats
> Plugin? In that case opening as a virtual stack is definitely the first
> thing to do. For
> images where Individual planes exceed 2GB limit (due to dimensions or
> pixel type),
> you might need to use the Crop option to load images in tiles rather than
> whole planes.
>
> [1]
> https://docs.openmicroscopy.org/bio-formats/5.9.2/about/bug-reporting.html
>
>
> Regarding compiling Bio-formats, I probably should have given it more
> time, which I did not had, unfortunately, to have given the manual a better
> read. What IDE are you using or are you compiling command line?
>
>
> Generally this documentation page is the landing page for everything
> related to building
> Bio-Formats from source [2]. In general, people will use either the
> command-line or an IDE,
> Eclipse being probably the most popular one. We should be able to offer
> guidance for both
> paths.
>
> Best,
> Sebastien
>
> [2]
> https://docs.openmicroscopy.org/bio-formats/5.9.2/developers/building-bioformats.html
>
>
> I am looking forward to receiving the fix
>
> Best regards,
>
> Johan Herz, Sales Engineer
>
> Please visit FLIM.camera <http://flim.camera/> for more information about
> the Toggel.
>
> Tel: +31-50-501-8461 | Skype: lambert-johan
> Lambert Instruments BV|Leonard Springerlaan 19 (5th floor)|9727 KB
> Groningen|The Netherlands
> Dutch Chamber of Commerce nr.: 52396940 | www.lambertinstruments.com
>
>
> Op di 8 jan. 2019 om 18:04 schreef Sebastien Besson (Staff) <
> s.besson at dundee.ac.uk>:
>
>> Hi Johan and Aaron,
>>
>> Thanks for making example images available, we will copy them internally
>> for testing purposes.
>> Is an amended version of the technical documentation described the format
>> changes also available?
>>
>> So far we have captured this new variant of the .fli format in the
>> general Bio-Formats backlog [1].
>> In terms of priority, the academically-funded team will primarily focus
>> on next-generation open formats
>> In 2019, starting with the release of Bio-Formats 6 [2]. Although we will
>> not be able to prioritise such work,
>> we will happily review any direct community contribution to update the
>> reader. An alternate possibility solution
>> would be to get in touch with Glencoe Software and commission the reader
>> update.
>>
>> Trying to work in incremental steps and solve the immediate blockers,
>> Johan, could you give us more details
>> into the type of issues you ran into while compiling Bio-Formats?
>>
>> Best,
>> Sebastien
>>
>>
>> [1] https://trello.com/c/OGGEzj7X/305-li-flim-12-bit-format
>> [2]
>> https://blog.openmicroscopy.org/file-formats/community/2018/11/29/ometiffpyramid/
>>
>>
>>
>>
>> On 7 Jan 2019, at 09:28, Ponti Aaron <aaron.ponti at bsse.ethz.ch> wrote:
>>
>> Hi,
>>
>> Happy new year, everyone! Sorry for the long silence about the FMI issue.
>> I don’t think I will soon find the time to implement support for 12-but FLI
>> files in bio-formats myself. I see that Johan replied to the thread and
>> provided example images. Would you guys work together to sort out the
>> issue? That’d be great!
>>
>> Thanks a lot!
>> a2
>>
>> ----
>> Dr. Aaron Ponti
>> Software and Data Management Engineer
>> Image Analysis Specialist
>> Single Cell Facility
>> Department of Biosystems Science and Engineering (D-BSSE)
>> ETH Zürich
>> Office 2.30
>> Mattenstrasse 26
>> 4058 Basel, Switzerland
>> aaron.ponti at bsse.ethz.ch
>> T: +41 61 387 33 74
>> F: +41 61 387 39 93
>>
>> *From:* Johan Herz <johan at lambertinstruments.com>
>> *Sent:* Mittwoch, 19. Dezember 2018 17:19
>> *To:* OME User Support List <ome-users at lists.openmicroscopy.org.uk>
>> *Cc:* Lummen Tom <tom.lummen at bsse.ethz.ch>; Ponti Aaron <
>> aaron.ponti at bsse.ethz.ch>
>> *Subject:* Re: [ome-users] Support for .FLI files from Capture software
>> (HiCAM FLUO high-speed camera)
>>
>> Hello Sebastien,
>>
>> Thank you for your reply.
>>
>> I have 2 files, 6 GB each.
>> They are made with 2 different versions of Capture our Hi-speed recording
>> software.
>> We make use of the .fli format and we tried to be as compatible with the
>> Bio-formats as possible.
>> Unfortunately, we could not work around the 12 bit, hence the unsupported
>> data format.
>>
>> Personally, I have tried to get bio-formats to compile on my one machine,
>> I made a checkout, but I ran in too many problems. It seems that you are
>> planning to have a new release.
>> Because I wanted to fix the problem of not being able to open files
>> having a filesize bigger than 2 GB.
>>
>> So could you take this in to account as well? To open our files
>> virtual, FiJi supports this, right?
>>
>> Is it okay for you if I share the files through Google Drive, as they are
>> too big to upload to your server?
>>
>>
>> * Recording_Capture_1.1.fli
>> <https://drive.google.com/a/lambertinstruments.com/file/d/1xw43lsucOssXb97ia0JJCCZ9467hV3iO/view?usp=drive_web>*
>>
>> * Recording_Capture_1.0.4.fli
>> <https://drive.google.com/a/lambertinstruments.com/file/d/19NwEUveiphMxUf9QSjtSHSMBthlaPxqW/view?usp=drive_web>*
>>
>> Also have attached a Recording with our New FLIM system, because
>> originally the .fli was for our FLIM system.
>> Those files are still compatible. So for testing purposes, I included
>> that such that you can make sure it still works.
>> See attachment.
>>
>> Please let me know if you have any further questions.
>>
>>
>>
>> Best regards,
>>
>>
>> Johan Herz, Sales Engineer
>> Please visit FLIM.camera <http://flim.camera/> for more information
>> about the Toggel.
>>
>> Tel: +31-50-501-8461 | Skype: lambert-johan
>> Lambert Instruments BV|Leonard Springerlaan 19 (5th floor)|9727 KB
>> Groningen|The Netherlands
>> Dutch Chamber of Commerce nr.: 52396940 | www.lambertinstruments.com
>>
>>
>> Op ma 17 dec. 2018 om 15:41 schreef Sebastien Besson (Staff) <
>> s.besson at dundee.ac.uk>:
>>
>> Hi Aaron,
>>
>>
>> On 14 Dec 2018, at 10:00, Ponti Aaron <aaron.ponti at bsse.ethz.ch> wrote:
>>
>> Hello,
>>
>> we have .FLI files from Lambert Instruments' Capture software (version
>> 1.0.4.0) for the high-speed camera HiCAM FLUO. Images are streamed at high
>> speed and stored as UINT12 (2 pixels in 3 bytes) or optionally as UINT8.
>> The FLIM reader in bio-formats (
>> https://github.com/openmicroscopy/bioformats/blob/develop/components/formats-gpl/src/loci/formats/in/LiFlimReader.java)
>> does not seem to support 12-bit intensities.
>>
>>
>> From the internal specification of the LI-FLIM file format which we used
>> for the reader [1]:
>>
>> "The datatype key can have one of the following 8 values: UINT8, INT8,
>> UINT16, INT16,
>> UINT32, INT32, REAL32 or REAL64.”
>>
>> A file that would use UINT12 would probably be a new variant of the file
>> format for which we
>> have currently neither a format specification nor representative samples
>>
>>
>> Also, when asking bio-formats to return the metadata, the whole file is
>> returned (which suggests that the metadata of standard FLIM .FLI files is
>> organized -- or terminated -- differently).
>>
>>
>> Understood. This would be consistent with the pixel type issue reported
>> above i.e. that could
>> indeed reflect an internal layout different from the one Bio-Formats is
>> expecting..
>>
>>
>>  The file is binary with the first bytes containing metadata information
>> in readable text format. One field is 'hasDarkImage'. If hasDarkImage == 1,
>> the very last frame is a dark image that is subtracted on-the-fly in the
>> software when displaying the frames (but the images are stored raw in the
>> file). The 'exposureTime' is in micro seconds. Finally, the software seems
>> to support 'compression', although there is no setting for turning it on in
>> the software. The software supports binning.
>>
>>
>>
>>  Would you be interested in adding support for these files to
>> bio-formats? We have example files that we could upload.
>>
>>
>> Over the past years, we have been working on ways to have more
>> sustainable support for new file
>> formats including any variants of existing file formats. We published a
>> blog post summarizing our
>> experience a few years ago [2] and since then more manufacturers have
>> been engaging in this
>> process.
>>
>> Have you been in touch with Lambert Instruments to report this issue? Are
>> they aware that the latest files
>> produced by their software are no longer readable by Bio-Formats and any
>> software based upon it?
>>
>> Representative samples allowing to reproduce the issue are always
>> welcome. Ideally public samples licensed
>> under CC-BY license [3] are the most effective way to help spontaneous or
>> sponsored contributions to the
>> project and facilitate both the immediate software work as well as the
>> long-term maintenance.
>> You can upload datasets under 2GB to our QA system [4] and we can provide
>> FTP credentials for large samples.
>>
>> Best,
>> Sebastien
>>
>>
>> [1]
>> https://docs.openmicroscopy.org/bio-formats/5.9.2/formats/lambert-instruments-flim.html
>> [2]
>> https://blog.openmicroscopy.org/file-formats/community/2016/08/31/bf-partnerships/
>> [3] https://downloads.openmicroscopy.org/images/
>> [4] http://qa.openmicroscopy.org.uk/qa/upload/
>>
>>
>>
>>  Thanks a lot,
>> a2
>>
>>
>>
>>
>> ----
>> Dr. Aaron Ponti
>> Software and Data Management Engineer
>> Image Analysis Specialist
>> Single Cell Facility
>> Department of Biosystems Science and Engineering (D-BSSE)
>> ETH Zürich
>> Office 2.30
>> Mattenstrasse 26
>> 4058 Basel, Switzerland
>> aaron.ponti at bsse.ethz.ch
>> T: +41 61 387 33 74
>> F: +41 61 387 39 93
>>
>> _______________________________________________
>> ome-users mailing list
>> ome-users at lists.openmicroscopy.org.uk
>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>>
>>
>>
>> The University of Dundee is a registered Scottish Charity, No: SC015096
>> _______________________________________________
>> ome-users mailing list
>> ome-users at lists.openmicroscopy.org.uk
>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>>
>> _______________________________________________
>> ome-users mailing list
>> ome-users at lists.openmicroscopy.org.uk
>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>>
>>
>>
>> The University of Dundee is a registered Scottish Charity, No: SC015096
>>
>
>
> The University of Dundee is a registered Scottish Charity, No: SC015096
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20190114/7a8847a9/attachment.html>