[ome-devel] (no subject)
Jason Swedlow
jason at lifesci.dundee.ac.uk
Sat Jun 26 20:25:49 BST 2004
Hello-
The attached message describes a proposal by Bitplane AG, submitted to
EAMNET at the recent European Light Microscopy Initiative Meeting in
Goteborg, Sweden, to use OME XML as a data exchange format of light
microscopy. OME XML is the file format developed by the Open
Microscopy Environment, an open source image software project devoted
to developing software tools for biological microscopy.
A word about where the OME XML file is coming from:
The Open Microscopy Environment provides a system for storing and
accessing binary image data, metadata, and analytic results. The
project has been running since 1998. The best site for documentation
about OME is at http://docs.openmicroscopy.org.uk. The ideas behind
the project are most recently described in Swedlow et al (2003)
Informatics and quantitative analysis in biological imaging. Science.
300:100-2. The project is open source, licensed under LGPL. OME was
founded in 1998 by Peter Sorger (MIT), Ilya Goldberg (NIH), and myself
(Univ of Dundee).
One of the major design requirements for OME is that the database be
locally extensible-- your science will be different than mine, so your
data model will differ too. Your instance of OME must adapt to your
changing needs. To solve this problem, we build the OME database
schema from an XML specification. The easiest way to view this is at
http://docs.openmicroscopy.org.uk/api/xml/OME/index.html. However the
full description of the namespace is available for download at
http://www.openmicroscopy.org/XMLschemas/OME/FC/ome.xsd If you want an
HTML view of this, go to
http://www.openmicroscopy.org/XMLschemas/OME/latest/ome_xsd/ How this
is gets implemented is at
http://docs.openmicroscopy.org.uk/newbie/db-def.html
Bitplane was commissioned a number of years ago by EAMNET to propose a
file exchange format for microscopy. They have examined OME XML and
are proposing it as a possible solution. The OME project supports this
proposal insomuch as it means more people might use it and help improve
it. We are reluctant to call it a "standard" yet. If people look at
it, use it, find its faults, and help us improve it, then maybe it can
one day become a de facto standard. We think it is ready for real
world use, but we expect we have not included everything the community
will need. Finally, I note that the OME XML file is distinct from the
larger goals of OME, namely a database and interface system for
biological microscopy. See our docs site for more info.
Thanks again for your support.
Cheers,
Jason
Begin forwarded message:
> From: messerli at bitplane.com
> Date: 14 May 2004 08:29:41 BST
> To: "j.swedlow at dundee.ac.uk" <j.swedlow at dundee.ac.uk>
> Subject: EAMNET: Image Exchange File Format
> Reply-To: messerli at bitplane.com
>
> To: EAMNET Partners, Microscope Manufacturers, Software Manufacturers,
> OME Team, AIC Team
>
> At the first ELMI meeting in 2001 in Chieti, Italy, Bitplane was
> assigned the task of evaluating and proposing an image exchange file
> format for the community. The main purpose is to facilitate data
> exchange between commercial instruments and/or software manufacturers
> and scientists. This task also became a specific aim of workpackage
> no. 8 of EAMNET, a EU funded network to assist scientists in
> exploiting the power of imaging.
>
> At the upcoming ELMI meeting in Gothenburg I will present and most
> importantly discuss our choice. In order to allow those of you
> interested in this topic to prepare for the discussion I have included
> the key elements of my presentation below.
>
> I am looking forward to receiving feedback from you and to meet with
> you in two weeks in Gothenburg to discuss how you want this work
> package of EAMNET to continue.
>
> With best regards,
>
> Marius
>
> Marius Messerli, Ph.D.
> CEO Bitplane AG
> President Bitplane Inc.
> http://www.bitplane.com
>
>
> Advantages of an Exchange File Format
> • Support different types and brands of instruments within one lab.
> • Facilitate exchange of data between scientists.
> • Facilitate archiving and management of images due common image
> file format.
> • Increasing the availability of 3rd party image analysis software.
> • Facilitating evaluation of equipment and software due to direct
> comparison.
>
> Choice of Format
> We have chosen to adopt the XML schema which was developed as part of
> the Open Microscopy Environment referred to as OME below. OME is a
> leading project on the definition and development of an open source,
> manufacturer independent, scientific image repository and management
> system and has been founded by Peter Sorger (MIT) and Jason Swedlow
> (Wellcome Trust, Dundee) in 1998.
>
> These are the reasons why we have chosen this format:
> • It is a vendor independent project.
> • The format specifically addresses the needs of microscopists.
> • It is a well-funded project that shows impressive momentum by
> itself.
> • ...plus all the arguments that speak for XML itself.
> • And last but not least compatibility with and access to all the
> other functions of OME.
> I am happy to announce that Jason Swedlow will also be present at the
> ELMI meeting and that he has volunteered to make a short presentation
> on OME-XML and answer questions.
>
> FAQ
> When discussing the format some of you have expressed concerns
> regarding various aspects of the OME-XML format. We have compiled
> these questions and discussed them with Ilya Goldberg, chief of the
> Image Informatics and Computational Biology Unit at NIA/NIH and a
> leading scientist behind OME who has provided many of the answers
> below.
>
> How does the XML Schema Look Like?
> The various pieces of the OME XML infrastructure (of which the schema
> is but one important component) is documented here.
>
> Why do we need a new format? Why can we not just take TIFF?
> In the standard application of the TIFF format, all meta-data is
> lost. The only thing you have left is the pixels themselves. TIFF
> provides a mechanism (custom tags) to add meta-data, but there would
> have to be agreement on the meta data, the tags, and their structure
> which is essentially what we've accomplished in XML.
>
> The file format itself isn't new - XML is used everywhere, and there
> is a lot of generic off-the-shelf software that does things with XML
> that you can't do with existing formats (parsers, XML databases, query
> tools, general data mining, etc). All we've really done is used an
> existing format and defined a set of metadata that should be included
> to describe microscopy images. Much in the same spirit as MIAME/MAGE
> for microarrays (also XML).
>
> In other words, this isn't a new file format - its a new set of
> meta-data. If the same information was presented in a completely
> different form, if it was still in XML, you could do an
> inter-conversion with a stylesheet.
>
> Who supports OME-XML format today?
> This list is almost certainly incomplete and one of the goals of our
> upcoming ELMI meeting is to get a better overview; please upate me
> prior to the meeting if you are already implementing OME-XML or have
> decided to do so. The information below is feedback we have received
> from the OME team:
> • Applied Precision: Softworx Eexplorer
> • Bitplane: Imaris
> • Perkin Elmer is discussing the implementation on technical level.
> • Carl Zeiss and 3i have expressed interest at top levels of
> management
> Besides commercial support there are a number of grants for some
> fairly major imaging infrastructure in evaluation for which at least
> one of the specific aims in is to implement OME-XML interoperability.
>
> Can OME-XML store large files?
> OME-XML files contain the voxel data in compressed form. This
> mechanism reduces the file size in the first place. The format also
> supports links to other files (or resources) containing the voxel data
> so that one potentially large 4D image can be split into several files
> (or resources).
>
> How about the the cost of parsing an XML document?
> The pixel data in OME-XML is plane-addressable. This way one can
> efficiently read sub-sections. An important point to remember is that
> OME XML is an interoperability standard. Its not necessarily primarily
> meant to be how you store your data natively or internally.
>
> The modern XML libraries (libxml-2.xx) are extremely fast. The
> process is completely I/O bound and will only become more so as drive
> performance lags behind CPU. Just like with any other format, the
> bottleneck is waiting for the bits to fly under the read head in your
> drive. Because we specify a compression standard, in practice our
> documents are smaller than their equivalent counterparts in other
> formats. This theoretically allows you to offload more of the reading
> task to the CPU, further decreasing read/access times.
>
> Lots of software is turning up that uses XML as its native format -
> even soon MS Office. With XML becoming (being?) a broadly used format
> it is likely that possible performance issues will quickly be
> resolved.
>
> Are there plans in OME to provide (c++/java/perl/...) libraries to
> support reading and writing OME XML files?
> More recently there has been considerable interest in parsing
> software in Java. The OME team is currently evaluating a couple of
> different options for implementation. In the near future there will
> probably be open-source Java libraries to parse OME XML.
>
> On a basic level there are very good libraries available for parsing
> XML, and they have standard APIs, so the calls look identical in all
> languages regardless of underlying implementation. Using a DOM parser,
> its completely trivial to extract the meta-data out of an XML file and
> do what you will with it. SAX parsers are more memory efficient, but
> they are more complex to deal with because its a stream parser with
> call-backs. What we do is pre-process the XML document using a SAX
> parser written in C (based on libxml2), stripping out all of the
> pixels but leaving the original document otherwise identical. Once the
> pixels are stripped, we read the resulting XML document into a DOM
> parser implemented in Perl (also based on libxml2). The code is all
> there on CVS - its extremely straight forward to do. If some of these
> utilities are useful in commercial software, we will definitely
> consider switching licenses to help adoption.
>
>
>
**************************
MSI/WTB Complex
The University of Dundee
Dow Street
Dundee DD1 5EH
United Kingdom
phone (01382) 345819
Intl phone: 44 1382 345819
FAX (01382) 348072
email: j.swedlow at dundee.ac.uk
Lab Page: http://www.dundee.ac.uk/lifesciences/swedlow/
Open Microscopy Environment: http://www.openmicroscopy.org
**************************
NOTE NEW EMAIL ADDRESS: jason at lifesci.dundee.ac.uk
**************************
MSI/WTB Complex
The University of Dundee
Dow Street
Dundee DD1 5EH
United Kingdom
phone (01382) 345819
Intl phone: 44 1382 345819
FAX (01382) 348072
email: jason at lifesci.dundee.ac.uk
Lab Page: http://www.dundee.ac.uk/lifesciences/swedlow/
Open Microscopy Environment: http://www.openmicroscopy.org
**************************
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/enriched
Size: 12448 bytes
Desc: not available
Url : http://necromancer.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20040626/7591f4ef/attachment.bin
More information about the ome-devel
mailing list