[ome-users] Support for .FLI files from Capture software (HiCAM FLUO high-speed camera)

Mon Jan 14 09:57:19 GMT 2019

Hi Johan and Aaron,

Happy New Year to you guys too. I am reincluding the Users mailing lists in this
conversation.

On 11 Jan 2019, at 08:11, Johan Herz <johan at lambertinstruments.com<mailto:johan at lambertinstruments.com>> wrote:

Hello Sebastien and Ponti,

Thank you for the follow-up, all to you too a very good 2019!

@Aaron, yes I will be available for testing, my pleasure.

@Sebastien,  The FLI format for the Hi-speed is not yet very well documented, it is basically based on the original FLI format with a few new data types. We also support color cameras. But from our experience, if we added a header entry that the bio-formats FLI reader did not recognize, it would just ignore it. Unless it makes a collision with another key entry.

So basically the main issues currently are the Data type and the 2gb File size max.
It will become possible to open very big files virtually, right? Because we can easily make files that are 50GB or bigger so it can most of the time never fit the RAM memory.

About 2GB, I assume you are referring to the issue described in this section of the
technical documentation [1]? Briefly, we are routinely dealing with datasets in the
GB-TB magnitude so file size per se should not be a problem.

Am I correct in assuming you are loading data using the ImageJ/Fiji Bio-Formats
Plugin? In that case opening as a virtual stack is definitely the first thing to do. For
images where Individual planes exceed 2GB limit (due to dimensions or pixel type),
you might need to use the Crop option to load images in tiles rather than whole planes.

[1] https://docs.openmicroscopy.org/bio-formats/5.9.2/about/bug-reporting.html

Regarding compiling Bio-formats, I probably should have given it more time, which I did not had, unfortunately, to have given the manual a better read. What IDE are you using or are you compiling command line?

Generally this documentation page is the landing page for everything related to building
Bio-Formats from source [2]. In general, people will use either the command-line or an IDE,
Eclipse being probably the most popular one. We should be able to offer guidance for both
paths.

Best,
Sebastien

[2] https://docs.openmicroscopy.org/bio-formats/5.9.2/developers/building-bioformats.html

I am looking forward to receiving the fix

Best regards,

Johan Herz, Sales Engineer

Please visit FLIM.camera<http://flim.camera/> for more information about the Toggel.

Tel: +31-50-501-8461 | Skype: lambert-johan
Lambert Instruments BV|Leonard Springerlaan 19 (5th floor)|9727 KB Groningen|The Netherlands
Dutch Chamber of Commerce nr.: 52396940 | www.lambertinstruments.com<http://www.lambertinstruments.com/>

Op di 8 jan. 2019 om 18:04 schreef Sebastien Besson (Staff) <s.besson at dundee.ac.uk<mailto:s.besson at dundee.ac.uk>>:
Hi Johan and Aaron,

Thanks for making example images available, we will copy them internally for testing purposes.
Is an amended version of the technical documentation described the format changes also available?

So far we have captured this new variant of the .fli format in the general Bio-Formats backlog [1].
In terms of priority, the academically-funded team will primarily focus on next-generation open formats
In 2019, starting with the release of Bio-Formats 6 [2]. Although we will not be able to prioritise such work,
we will happily review any direct community contribution to update the reader. An alternate possibility solution
would be to get in touch with Glencoe Software and commission the reader update.

Trying to work in incremental steps and solve the immediate blockers, Johan, could you give us more details
into the type of issues you ran into while compiling Bio-Formats?

Best,
Sebastien

[1] https://trello.com/c/OGGEzj7X/305-li-flim-12-bit-format
[2] https://blog.openmicroscopy.org/file-formats/community/2018/11/29/ometiffpyramid/

On 7 Jan 2019, at 09:28, Ponti Aaron <aaron.ponti at bsse.ethz.ch<mailto:aaron.ponti at bsse.ethz.ch>> wrote:

Hi,

Happy new year, everyone! Sorry for the long silence about the FMI issue. I don’t think I will soon find the time to implement support for 12-but FLI files in bio-formats myself. I see that Johan replied to the thread and provided example images. Would you guys work together to sort out the issue? That’d be great!

Thanks a lot!
a2

----
Dr. Aaron Ponti
Software and Data Management Engineer
Image Analysis Specialist
Single Cell Facility
Department of Biosystems Science and Engineering (D-BSSE)
ETH Zürich
Office 2.30
Mattenstrasse 26
4058 Basel, Switzerland
aaron.ponti at bsse.ethz.ch<mailto:aaron.ponti at bsse.ethz.ch>
T: +41 61 387 33 74
F: +41 61 387 39 93

From: Johan Herz <johan at lambertinstruments.com<mailto:johan at lambertinstruments.com>>
Sent: Mittwoch, 19. Dezember 2018 17:19
To: OME User Support List <ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>>
Cc: Lummen Tom <tom.lummen at bsse.ethz.ch<mailto:tom.lummen at bsse.ethz.ch>>; Ponti Aaron <aaron.ponti at bsse.ethz.ch<mailto:aaron.ponti at bsse.ethz.ch>>
Subject: Re: [ome-users] Support for .FLI files from Capture software (HiCAM FLUO high-speed camera)

Hello Sebastien,

Thank you for your reply.

I have 2 files, 6 GB each.
They are made with 2 different versions of Capture our Hi-speed recording software.
We make use of the .fli format and we tried to be as compatible with the Bio-formats as possible.
Unfortunately, we could not work around the 12 bit, hence the unsupported data format.

Personally, I have tried to get bio-formats to compile on my one machine, I made a checkout, but I ran in too many problems. It seems that you are planning to have a new release.
Because I wanted to fix the problem of not being able to open files having a filesize bigger than 2 GB.

So could you take this in to account as well? To open our files virtual, FiJi supports this, right?

Is it okay for you if I share the files through Google Drive, as they are too big to upload to your server?

[https://ssl.gstatic.com/docs/doclist/images/icon_10_generic_list.png] Recording_Capture_1.1.fli<https://drive.google.com/a/lambertinstruments.com/file/d/1xw43lsucOssXb97ia0JJCCZ9467hV3iO/view?usp=drive_web>

[https://ssl.gstatic.com/docs/doclist/images/icon_10_generic_list.png] Recording_Capture_1.0.4.fli<https://drive.google.com/a/lambertinstruments.com/file/d/19NwEUveiphMxUf9QSjtSHSMBthlaPxqW/view?usp=drive_web>

Also have attached a Recording with our New FLIM system, because originally the .fli was for our FLIM system.
Those files are still compatible. So for testing purposes, I included that such that you can make sure it still works.
See attachment.

Please let me know if you have any further questions.

Best regards,

Johan Herz, Sales Engineer
Please visit FLIM.camera<http://flim.camera/> for more information about the Toggel.

Tel: +31-50-501-8461 | Skype: lambert-johan
Lambert Instruments BV|Leonard Springerlaan 19 (5th floor)|9727 KB Groningen|The Netherlands
Dutch Chamber of Commerce nr.: 52396940 | www.lambertinstruments.com<http://www.lambertinstruments.com/>

Op ma 17 dec. 2018 om 15:41 schreef Sebastien Besson (Staff) <s.besson at dundee.ac.uk<mailto:s.besson at dundee.ac.uk>>:
Hi Aaron,

On 14 Dec 2018, at 10:00, Ponti Aaron <aaron.ponti at bsse.ethz.ch<mailto:aaron.ponti at bsse.ethz.ch>> wrote:

Hello,

we have .FLI files from Lambert Instruments' Capture software (version 1.0.4.0) for the high-speed camera HiCAM FLUO. Images are streamed at high speed and stored as UINT12 (2 pixels in 3 bytes) or optionally as UINT8. The FLIM reader in bio-formats (https://github.com/openmicroscopy/bioformats/blob/develop/components/formats-gpl/src/loci/formats/in/LiFlimReader.java) does not seem to support 12-bit intensities.

From the internal specification of the LI-FLIM file format which we used for the reader [1]:

"The datatype key can have one of the following 8 values: UINT8, INT8, UINT16, INT16,
UINT32, INT32, REAL32 or REAL64.”

A file that would use UINT12 would probably be a new variant of the file format for which we
have currently neither a format specification nor representative samples

Also, when asking bio-formats to return the metadata, the whole file is returned (which suggests that the metadata of standard FLIM .FLI files is organized -- or terminated -- differently).

Understood. This would be consistent with the pixel type issue reported above i.e. that could
indeed reflect an internal layout different from the one Bio-Formats is expecting..

 The file is binary with the first bytes containing metadata information in readable text format. One field is 'hasDarkImage'. If hasDarkImage == 1, the very last frame is a dark image that is subtracted on-the-fly in the software when displaying the frames (but the images are stored raw in the file). The 'exposureTime' is in micro seconds. Finally, the software seems to support 'compression', although there is no setting for turning it on in the software. The software supports binning.

 Would you be interested in adding support for these files to bio-formats? We have example files that we could upload.

Over the past years, we have been working on ways to have more sustainable support for new file
formats including any variants of existing file formats. We published a blog post summarizing our
experience a few years ago [2] and since then more manufacturers have been engaging in this
process.

Have you been in touch with Lambert Instruments to report this issue? Are they aware that the latest files
produced by their software are no longer readable by Bio-Formats and any software based upon it?

Representative samples allowing to reproduce the issue are always welcome. Ideally public samples licensed
under CC-BY license [3] are the most effective way to help spontaneous or sponsored contributions to the
project and facilitate both the immediate software work as well as the long-term maintenance.
You can upload datasets under 2GB to our QA system [4] and we can provide FTP credentials for large samples.

Best,
Sebastien

[1] https://docs.openmicroscopy.org/bio-formats/5.9.2/formats/lambert-instruments-flim.html
[2] https://blog.openmicroscopy.org/file-formats/community/2016/08/31/bf-partnerships/
[3] https://downloads.openmicroscopy.org/images/
[4] http://qa.openmicroscopy.org.uk/qa/upload/

 Thanks a lot,
a2

----
Dr. Aaron Ponti
Software and Data Management Engineer
Image Analysis Specialist
Single Cell Facility
Department of Biosystems Science and Engineering (D-BSSE)
ETH Zürich
Office 2.30
Mattenstrasse 26
4058 Basel, Switzerland
aaron.ponti at bsse.ethz.ch<mailto:aaron.ponti at bsse.ethz.ch>
T: +41 61 387 33 74
F: +41 61 387 39 93

_______________________________________________
ome-users mailing list
ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users

The University of Dundee is a registered Scottish Charity, No: SC015096
_______________________________________________
ome-users mailing list
ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
_______________________________________________
ome-users mailing list
ome-users at lists.openmicroscopy.org.uk<mailto:ome-users at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users

The University of Dundee is a registered Scottish Charity, No: SC015096

The University of Dundee is a registered Scottish Charity, No: SC015096
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20190114/5d4f984a/attachment.html>