[ome-users] Flex File Format
Jason Swedlow
jason at lifesci.dundee.ac.uk
Tue May 5 15:46:25 BST 2009
Hi Matthew-
On 4 May 2009, at 11:08, <Matthew.Smicker at sanofi-aventis.com> <Matthew.Smicker at sanofi-aventis.com
> wrote:
> Does OMERO convert to OME-TIFF upon import of a non-OME-TIFF file?
> My
> impression was that it preserved the original format and just used
> Bio-formats to read it?
OMERO uses Bio-Formats to read proprietary files. As discussed before
on this list, we turn on formats in OMERO once we are sure metadata
support is correct and we have a good set of test data. Even then,
we have problems with different versions of formats, specific metadata
types.
>
> I wanted to comment that the issue touched upon with vendor
> compatibility is a significant one. We have cases where usage of the
> vendor software is critical, but the development of that software to
> support OME-TIFF is not feasible. Conversion from proprietary formats
> would in most cases be possible, but not necessarily the other way.
> It
> also takes time and extra logic.
>
Converting files is certainly inconvenient, especially when dealing
with large data sets.
A solution is to use an application like OMERO, that provides a common
interface to many different data types.
> Ghislain's idea of the 'chimeric' OME-TIFF/Flex-file sounds nifty
> since
> the FLEX file is essentially TIFF. His other suggestion of having a
> pointer to the pixel data within the original file could be more
> flexible (e.g. non-TIFF formats as well).
Yes, but think this through, and consider the difficulty of
maintaining metadata synchrony between the two, or more, files.
The idea of the pointer to the pixel data is already in OME-TIFF.
In general, this discussion demands a solution to this problem, now.
We sympathise-- our own science and discovery also suffers from the
difficulties imposed by lack of interoperability. So our work towards
standardised formats and interfaces is a work in progress. We are far
advanced over where we were one year ago. Given that progress and
momentum, we'd prefer to keep pushing the concepts of interoperability
for all developers. As a customer, you have alot of power to demand
this type of standardization from your vendors. OME's job is to
ensure that implementing interoperability is easy, cost effective for
commercial entities, and most importantly, enabling.
>
>
> Rather than conversion, I could imagine a metadata translation to
> OME-XML with pointers to the image file/index. After translation, the
> reader would start with OME-XML parsing, but use a specific native
> image
> reader for the pixels. (basically focus on the metadata standard as a
> file rather than pixels)
Interesting. You should come work with OME and see how much fun we
have just tracking the metadata! Pixels too?
If:
1. someone out there can provide funding for this
2. get all the vendors to guarantee that they will update us with
correct specifications of new formats BEFORE they are released
we'll consider it.
>
> ...that or hopefully things progress more quickly with OME-TIFF
> adoption!
>
Exactly. That seems the simplest, and ultimately the most effective
strategy.
Thanks very much for your feedback. This is a great discussion, and
covers many of the issues that we have worked through many years.
Cheers,
Jason
>
> Matt
>
>
>
> -----Original Message-----
> From: ome-users-bounces at lists.openmicroscopy.org.uk
> [mailto:ome-users-bounces at lists.openmicroscopy.org.uk] On Behalf Of
> Ghislain Bonamy
> Sent: Sunday, May 03, 2009 4:04 PM
> To: Jason Swedlow
> Cc: ome-users at lists.openmicroscopy.org.uk; John Moffat
> Subject: Re: [ome-users] Flex File Format
>
> Hi Jason,
>
>
>
> Thanks for the feedback! Let me try to clarify a few points:
>
>
> Unfortunately the flex file format is not up to date in the
> bioformats,
> although some of the new metadata is handled. In particular,
> bioformats
> is unable to retrieve the number of fields/z-section/channels/time
> point, which is critical point.
>
>
>
> Thanks for this feedback. This is news to us, as we correctly report
> this metadata from the test files we have. If you have had problems
> using Bio-Formats, the best thing to do is to send us details, and if
> possible, the offending files, and we'll certainly work to we fix any
> problems we find.
>
>
>
> GB: What I meant here is that many new tags are supported; however
> some
> of the critical one are not correctly handled. I do not believe that
> that I am using any special files, but rather that the way to extract
> the number of z-sections, fields etc. has not been implemented. I
> talked
> about his with Melissa a few months ago, and perhaps we can take this
> discussion off-line with users interested in FLEX file reader
> improvements in bioformats.
>
>
> I was hoping to work on this and add this capacity to the format but
> have not had a chance to it. I have contacted P&E about maybe helping
> out since they are using Omro a base for their Columbus platform,
> but I
> don't think, oddly enough, that they are interested to help.
>
>
>
> On the contrary, PerkinElmer (I assume that is who you are referring
> to)
> have been very helpful supplying test files. Again, if you have files
> that might expose something we don't know about, sending us specific
> details and the data is the best thing to do.
>
>
>
> GB: Yes I was referring to PerkinElmer. I am glad to hear that they
> provided file examples, although I still hope that perhaps they will
> provide some actual coding/testing support to allow full handling of
> their FLEX file format. As mentioned above, I do not think that the
> flex
> files that I am using differ from the one they have provided. I
> believe
> it is more a matter of parsing out the full information contained in
> their metadata.
>
>
>
> Perhaps if enough customer request that they help improving bioformats
> they might do it. I will try to work on it soon and will keep the
> Opera
> users posted. Furthermore, a nice feature would be to be able to keep
> the metadata from the flex file in addition to the metadata from
> OME, so
> that the flex file remains backward compatible with the Epochal
> (becomes
> both an ome.tiff and remains a flex file). This feature which could be
> extended to all TIFF derivative format, may require some serious
> modifications of bioformats, but I will let the expert (i.e.. Jason)
> see
> what they can do about t.
>
>
>
> Can you explain why we should do this? Why doesn't the metadata in
> OME-TIFF satisfy your needs? Note that we NEVER directly write
> proprietary formats-- we have enough problems supporting reads of all
> the variants. Writing as well would be a minefield, and more
> importantly, breaks the whole concept of supporting interoperability.
>
>
>
> GB: I am not suggesting writing a proprietary file format, although I
> see why it may sound like it. One of the limitations with using the
> OME-TIFF model is that the file becomes incompatible with the vendors
> tools. It seems that the format selected by P&E is very good in the
> sense that it builds on a format logic closely related to OME-TIFF.
> Thus, I though enabling creation of chimeric files could be useful for
> formats in which it can be implemented (manly with Tiff derivative
> formats). I wrote this in relation to importing files into OMERO,
> which
> was the purpose of the original thread. If Flex flex files can be
> converted as OME.TIFF, but retain the flex file characteristics (ie.
> flex extension + correct IFID), this would make it very convenient. I
> suppose that an easier (and perhaps better) alternative would be to
> generate an OME-TIFF file were the pixel data points to the flex file?
> Could this work and allow users to import there flex files into OMERO?
>
> On another note bioformats does support the luraware compression,
> granted hat you provide a license code for it. Logic would suggest
> that
> a key would be required to compress with luraware but not to
> decompress
> to make the forma at least partially open. This however is not the
> case.
> Perhaps P&E should in my opinion provide such license code since the
> capacity of a user to read an image is obviously a inherent part of
> their system. However, this may not be part of the agreement they
> reach
> with Luraware. In the mean time, saving as uncompressed tiff is the
> better solution.
>
>
>
> We can't comment on PerkinElmer's plans for file formats; we have no
> knowledge of their plans and agreements and any discussions of this
> with
> the community and users is up to them. We do however agree that
> saving
> data in uncompressed form often leaves the user with many more options
> for data handling and analysis.
>
>
>
> GB: Agreed, just brought it up for users employing Luraware
> compression
> and hoping to get a response from people at P&E, to see if it would be
> feasible within their licensing agreement with Luraware to provide a
> license key to their Opera users. In particular for working with OMERO
> with compressed FLEX files.
>
> Let me know if my points remain unclear.
>
> Best,
>
> Ghislain Bonamy, PhD
> _______________________________
> Genomic Institute of the
> Novartis Research
> Foundation
> Functional Genomics, G214
> +1 (858) 812-1534 (W & F)
> +1 (858) 354-7388 (C)
> www.gnf.org <http://www.gnf.org/ <http://www.gnf.org/> >
>
> HudsonAlpha Instiute for Biotechnology
> www.hudsonalpha.org <http://www.hudsonalpha.org/> <http://
> www.gnf.org/>
>
> ________________________________
>
> From: ome-users-bounces at lists.openmicroscopy.org.uk on behalf of John
> Moffat
> Sent: Fri 5/1/2009 11:40 PM
> To: ome-users at lists.openmicroscopy.org.uk
> Subject: [ome-users] Flex File Format
>
>
>
> Hi,
> I was just about to ask the exact same question as the previous
> poster.
> Since I'm not using the Lurawave compression (not worth the hassle, in
> my opinion), is there a way we can activate the PerkinElmer .flex
> format
> in the bioformats library?
>
> Does anyone know if the flex file bioformat is up to date with the
> newer
> (Acapella 2) version of the format? Either way, I'd like to test it.
>
> Thanks
> John
>
>
> _______________________________________________
> ome-users mailing list
> ome-users at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>
>
> _______________________________________________
> ome-users mailing list
> ome-users at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
>
>
>
>
>
> **************************
>
> Wellcome Trust Centre for Gene Regulation & Expression
>
> College of Life Sciences
>
> MSI/WTB/JBC Complex
>
> University of Dundee
>
> Dow Street
>
> Dundee DD1 5EH
>
> United Kingdom
>
>
>
> phone (01382) 385819
>
> Intl phone: 44 1382 385819
>
> FAX (01382) 388072
>
> email: jason at lifesci.dundee.ac.uk <mailto:jason at lifesci.dundee.ac.uk>
>
>
>
> Lab Page: http://www.dundee.ac.uk/lifesciences/swedlow/
> <http://www.dundee.ac.uk/lifesciences/swedlow/>
>
> Open Microscopy Environment: http://openmicroscopy.org
> <http://openmicroscopy.org/>
>
> **************************
>
>
>
> The University of Dundee is a Scottish Registered Charity, No.
> SC015096.
>
>
>
>
>
>
>
>
>
> _______________________________________________
> ome-users mailing list
> ome-users at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
> _______________________________________________
> ome-users mailing list
> ome-users at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users
**************************
Wellcome Trust Centre for Gene Regulation & Expression
College of Life Sciences
MSI/WTB/JBC Complex
University of Dundee
Dow Street
Dundee DD1 5EH
United Kingdom
phone (01382) 385819
Intl phone: 44 1382 385819
FAX (01382) 388072
email: jason at lifesci.dundee.ac.uk
Lab Page: http://www.dundee.ac.uk/lifesciences/swedlow/
Open Microscopy Environment: http://openmicroscopy.org
**************************
The University of Dundee is a Scottish Registered Charity, No. SC015096.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20090505/5d898c27/attachment.html>
More information about the ome-users
mailing list