[ome-devel] Proposed tweak to µManager data files in 2.0

[Hadrien Mary]

Hello guys,

Sorry for the very naive question Chris but why can't you just use the OME
XML format as the core format for MM instead of creating another format ?

I don't understand the disadvantages of doing that and if there is
limitations with the current OME XML format why not trying to improve it
with the OME folks ?

Thank you



--
Hadrien Mary
Postdoc

Brouhard's lab
University of McGill
Montreal, Canada

On Thu, Apr 14, 2016 at 2:11 PM, Chris Weisiger <cweisiger at msg.ucsf.edu>
wrote:

> Thank you, everyone, for your feedback so far. Keep it coming! I do want
> to highlight a few points and provide some clarifications.
>
> There seems to be fairly broad support for the idea of generating an
> external OME XML file. It sounds in general like you would prefer us to
> either *always* generate the OME XML externally (i.e. as a separate
> dedicated file), or *never* do so, which seems fair enough. Also, if the
> file is externally-generated, then it should ideally have a name tied to
> the TIFF filenames, in case users mix datasets together.
>
> Note for those considering parsing these files themselves: we also
> currently have the option to always generate a metadata.txt file, which is
> a JSON file that contains complete metadata (including the state of every
> device in the scope) for every image in the dataset. It will be much more
> complete than the OME XML. Of course, if you already have code for parsing
> OME XML, or want to use the OME libraries, then parsing the XML instead
> would be easier.
>
> We are not at this time considering making major structural changes to our
> data files. That is definitely a project we want to tackle in the future,
> but it is going to require serious thinking, because we want to have a file
> format that we can continue to use for the next decade at least. Every time
> we change our file format, we break users' workflows, so it's not something
> we would do lightly (of course, we will continue to support existing file
> formats as well). When the New µManager File Format project (NµFF :)) does
> start, you can expect there to be some significant discussion on these
> forums to make certain that we're not missing important use cases.
>
> That said, feel free to continue to brainstorm! I'm listening to your
> suggestions.
>
> -Chris
>
> On Wed, Apr 13, 2016 at 10:26 AM, Chris Weisiger <cweisiger at msg.ucsf.edu>
> wrote:
>
>> Hello all,
>>
>> After discussion with some of our users and with the OME developers, I'm
>> creating this thread to invite comments on a proposed minor change to
>> µManager's "image stack file" storage format, in µManager 2.0 only (1.4
>> would not be changed). The "image stack file" format (which is the default
>> format) stores multiple image planes in the same TIFF file. This change
>> would be to the situation where multiple of these multi-plane TIFF files
>> are generated in a single dataset.
>>
>> µManager writes multiple TIFF files in two situations: first, if there is
>> more than 4GB of data to store (as our TIFFs don't yet support BigTIFF),
>> and second, if the user has elected to store data for each stage position
>> in a separate file. Currently, the OME XML is written to a single one of
>> these files, and the other files are "pointed" to that file so they know
>> where to look to load the data in the XML. Thus in order to load any file
>> in the dataset, you must have not only that file, but also the TIFF file
>> that contains the OME XML.
>>
>> The proposal is to instead write the XML to a newly-created file
>> (tentatively named "OMEXMLMetadata.companion.ome") that would exist in the
>> same directory as the TIFF files, display settings, comments, and
>> metadata.txt, as appropriate. As before, the TIFF files would be "pointed"
>> to this file; it would need to be present to use the BioFormats Importer or
>> any other OME library loader.
>>
>> In situations where there is only one TIFF file, the OME XML would still
>> be stored in that TIFF file; the "OMEXMLMetadata.companion.ome" file would
>> not be generated. For reference, the XML would look something like this:
>>
>> http://pastebin.com/cTzcQwEZ
>>
>> I've tested this change locally and haven't detected any problems, but
>> I'm not familiar with the kinds of workflows that our users may want to
>> perform, so any comments are welcome. Thanks for your time.
>>
>> -Chris Weisiger and the µManager dev team
>>
>
>
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20160414/1f0c71dc/attachment.html>