[ome-users] Missing metadata for Micro-Manager datasets

[Thomas Julou]

Dear Melissa,

Great to read you’re already on it!
I attach a small test dataset acquired with the demo config of MM 1.4.22. tifutill tell me it has all tags you need:
50838 (0xc696) LONG (4) 5<28 5496 32 768 768>
50839 (0xc697) BYTE (1) 7092<0x4a 0x49 ...>
51123 (0xc7b3) ASCII (2) 3329<{"Objective-Name":"DObje ...>

If it’s not enough, id happily send you an experimental dataset but would rather acquire a small one so as to keep it small… 

Best,
Thomas




> On 25 Jan 2016, at 21:39, Melissa Linkert <melissa at glencoesoftware.com> wrote:
> 
> Hi Thomas,
> 
>> As far as I understand, the issue comes from the fact that MM stores 2 types of metadata: a minimalist set of general metadata are stored as OME-metadata (time, pixel size, camera name, etc) and configuration-specific metadata are saved as a JSON string using a dedicated tiff tag (51123). At the moment these are not read by MM’s bioformats reader in the first place, so that crucial metadata (e.g. illumination wavelength, intensity) cannot be retrieved at all (neither in omero nor in fiji using bioformats import). Unfortunately there is no simple way to populate more OME-metadata on the MM side because the required ontology is not defined (nothing says that a given combination of property defines a given metadata). However, it should be doable to read the JSON string of configuration-specific metadata and store it in some generic slot. Even better:
>> - the JSON might be parsed so that each field is stored in a given slot.
>> - since most fields have the same value for all frames, they could be added only once; those which change could be shown in a nice scrollable table (as e.g. exposure).
>> From a user perspective, those configuration-specific metadata should be displayed in the “Original metadata” section of omero. However I don’t know what it means in technical terms on the bioformats/omero side.
> 
> We're testing the first stage of a solution now:
> 
> https://github.com/openmicroscopy/bioformats/pull/2213
> 
> Note though that we do not have any datasets which use the
> MicroManagerMetadata tag (51123), so for the moment only the IJMetadata
> tag (50839, see https://micro-manager.org/wiki/Micro-Manager_File_Formats) is used.
> If you are able to send a small dataset that has the
> MicroManagerMetadata tag populated, that would be much appreciated.
> 
> Regards,
> -Melissa
> 
> On Sun, Jan 24, 2016 at 01:26:41PM +0100, Thomas Julou wrote:
>> Hello,
>> 
>> I decided some time ago to use omero for microscope images archiving and browsing in our lab. This has been facilitated by the fact that people at our imaging facility already run an omero server and were kind enough to configure a customised storage so that we can access the data also from the command line on our high performance cluster. This to say that I’m already rather committed to omero ;) Our images are almost exclusively acquired using micro-manager and I recently realised a major limitation of the omero reader (after slightly changing our microscope configuration, in fact), namely the fact that several important metadata are not visible in omero. 
>> 
>> As far as I understand, the issue comes from the fact that MM stores 2 types of metadata: a minimalist set of general metadata are stored as OME-metadata (time, pixel size, camera name, etc) and configuration-specific metadata are saved as a JSON string using a dedicated tiff tag (51123). At the moment these are not read by MM’s bioformats reader in the first place, so that crucial metadata (e.g. illumination wavelength, intensity) cannot be retrieved at all (neither in omero nor in fiji using bioformats import). Unfortunately there is no simple way to populate more OME-metadata on the MM side because the required ontology is not defined (nothing says that a given combination of property defines a given metadata). However, it should be doable to read the JSON string of configuration-specific metadata and store it in some generic slot. Even better:
>> - the JSON might be parsed so that each field is stored in a given slot.
>> - since most fields have the same value for all frames, they could be added only once; those which change could be shown in a nice scrollable table (as e.g. exposure).
>> From a user perspective, those configuration-specific metadata should be displayed in the “Original metadata” section of omero. However I don’t know what it means in technical terms on the bioformats/omero side.
>> 
>> I hope that this request makes sense and can be addressed without major trouble. 
>> Best regards,
>> 
>> --
>> Thomas Julou  |  Computational & Systems Biology  |  Biozentrum – University of Basel  |  Klingelbergstrasse 50/70 CH-4056 Basel  |  +41 (0)61 267 16 21
>> 
>> _______________________________________________
>> ome-users mailing list
>> ome-users at lists.openmicroscopy.org.uk
>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20160125/9b605df9/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: MM_test_OME_1.zip
Type: application/zip
Size: 99869 bytes
Desc: not available
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20160125/9b605df9/attachment.zip>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20160125/9b605df9/attachment-0001.html>