[ome-devel] Is OME-TIF file with 2 image series of different pixel types supported?
Michael Ellis
michael.ellis at dsuk.biz
Fri Dec 2 16:57:20 GMT 2016
Curtis,
Many thanks. We may dump the captured frames to disk in whatever format allows for the fastest time to save, perhaps raw data, since it’s possible that the streaming acquisition may run quickly and we need to release the memory used for holding the images as soon as possible. Our image acquisition approach allows for image frames to be returned to the camera for refuse to avoid memory reallocation.
But we certainly should plan as you have outlined to accumulate the metadata during acquisition.
Then we can replay the frames out of raw storage into the OME.TIFF in our case we might be writing the image data twice but increasingly we are working on computers with flash drives so it’s probably not too big a hit.
Being able to capture the data at highest possible frame rate is probably more important for us.
— ME
> On 2 Dec 2016, at 16:45, Curtis Rueden <ctrueden at WISC.EDU> wrote:
>
> Hi Michael,
>
> > Since I will need the ability to stream frames, and all my frames will
> > be of same size and type, I’ll write the frame data to disk as it is
> > collected and the populate the OME.TIFF file by recovering the frames
> > from disk. Not ideal but not too much work.
>
> For what it's worth, here is how we do it at LOCI with our WiscScan laser scanning software:
>
> During the run:
>
> - Acquire TIFF frames, writing them to individual files on disk.
>
> - Keep an OME-XML data structure in memory which is continuously updated as the acquisition proceeds.
>
> - Write a snapshot of the OME-XML block to a temporary file on disk, in case the run crashes.
>
> At the completion of the run:
>
> - Rewrite all TIFF frames into the final desired organization—in our case, one TIFF file per time point and channel, with all Z positions in the same file.
>
> - Generate the final OME-XML block from the OME-XML data structure, injecting it into every 10th TIFF file.
> -- The other 9 TIFFs receive a "partial OME-XML metadata" stub as described in the OME-TIFF specification [1].
> -- This provides some redundancy of metadata without bloating the size of all TIFF files.
>
> It is also feasible not to rewrite the TIFFs ever, and instead efficiently inject the final OME-XML into each existing TIFF file. The Bio-Formats library provides convenience methods for doing this [2], as long as you write a stub ImageDescription initially.
>
> In this way, it is possible to do e.g. streaming acquisition, and ultimately end up with OME-TIFF.
>
> Regards,
> Curtis
>
> [1] http://www.openmicroscopy.org/site/support/ome-model/ome-tiff/specification.html#partial-ome-xml-metadata <http://www.openmicroscopy.org/site/support/ome-model/ome-tiff/specification.html#partial-ome-xml-metadata>
> [2] https://github.com/openmicroscopy/bioformats/blob/v5.2.4/components/formats-bsd/src/loci/formats/tiff/TiffSaver.java#L892-L897 <https://github.com/openmicroscopy/bioformats/blob/v5.2.4/components/formats-bsd/src/loci/formats/tiff/TiffSaver.java#L892-L897>
>
> --
> Curtis Rueden
> LOCI software architect - http://loci.wisc.edu/software <http://loci.wisc.edu/software>
> ImageJ2 lead, Fiji maintainer - http://imagej.net/User:Rueden <http://imagej.net/User:Rueden>
> Did you know ImageJ has a forum? http://forum.imagej.net/ <http://forum.imagej.net/>
>
>
> On Fri, Dec 2, 2016 at 10:06 AM, Michael Ellis <michael.ellis at dsuk.biz <mailto:michael.ellis at dsuk.biz>> wrote:
>
>> On 2 Dec 2016, at 15:40, Roger Leigh <rleigh at dundee.ac.uk <mailto:rleigh at dundee.ac.uk>> wrote:
>>
>> On 01/12/16 15:26, Michael Ellis wrote:
>>> Dear Roger
>>>
>>> The main body of my code can be seen
>>> here: https://cloud.karyotyper.com/s/K9SoEKgLOcbgwTg <https://cloud.karyotyper.com/s/K9SoEKgLOcbgwTg>
>>>
>>> CaptureImages is essential a sorted ArrayList of CaptureImage
>>> CapturedImage is a BufferImage that has a Map<String, Object> associated
>>> with it for additional image properties such as channel name, colour,
>>> stage position, exposure time etc.
>>>
>>> My BioFormatsWriter class interrogates those extra image properties to
>>> build the extra metadata.
>>>
>>> Because BioFormats has to have all metadata constructed before writing
>>> the pixel data, this BioFormatsWriter simple implementation
>>> construct all the metadata as each image series is added with the
>>> addSeries() method, and then actually writes the images data when it
>>> gets to the close() method.
>>
>> I checked through the OMETiffWriter code, and unfortunately it does
>> require all the image series' to be present up-front.
>>
>> https://github.com/openmicroscopy/bioformats/blob/develop/components/formats-bsd/src/loci/formats/out/OMETiffWriter.java#L218 <https://github.com/openmicroscopy/bioformats/blob/develop/components/formats-bsd/src/loci/formats/out/OMETiffWriter.java#L218>
>>
>> This could potentially be made more general, but will require more
>> thought about its implications. The writer is currently oblivious to
>> the subsequently added image series', which is why it's unfortunately
>> not working properly as it stands.
>>
>
> Since I will need the ability to stream frames, and all my frames will be of same size and type, I’ll write the frame data to disk as it is collected and the populate the OME.TIFF file by recovering the frames from disk. Not ideal but not too much work.
>
>>> Some pointers in regards to what “is not facet-valid with respect to
>>> pattern '(urn:lsid:([\w\-\.]+\.[\w\-\.]+)+:\S+:\S+)|(\S+:\S+)’” would be
>>> much appreciated too.
>>
>> Here the above regular expression is used to validate the "ID"
>> attributes for various elements. The default is to match the (\S+:\S+)
>> pattern by using identifiers like "Instrument:0", "Channel:1",
>> "FilterSet:2" and so on. I think use of "Fitc" and "TxRed" as Filter
>> IDs is not matching; you could use e.g. "Model" here as a description,
>> though the most user-visible place is probably on Channel "Name" if you
>> want it visible in user interfaces. The same applies to
>> "EmulatedCamera_0" on Detector; this is expecting Detector:n as the ID;
>> "Model" is likely the most appropriate attribute here for the camera
>> name. A shorthand Name attribute would perhaps be a convenient addition
>> to the model for these components, as a shorthand reference, since the
>> ID is essentially a primary key with a strict naming requirement.
> Yes, that is a terrific help and I will look to make my code comply.
>>
>>
>> I hope that helps, please let me know if there's anything you'd like
>> explaining further.
>
> I am very appreciative of the help you have offered. I am writing a computer assisted microscope application and storage of multiple 5D images with associated metadata has to be tackled one way or another. Rolling my own was not an attractive option especially since ImageJ will be one popular end points and if the ImageJ BioFormats Importer plugin does a half decent job, then I have a real benefit for zero (extra) effort.
>
> Again many thanks!
>
>
>>
>>
>> Regards,
>> Roger
>>
>> --
>> Dr Roger Leigh -- Open Microscopy Environment
>> Wellcome Trust Centre for Gene Regulation and Expression,
>> College of Life Sciences, University of Dundee, Dow Street,
>> Dundee DD1 5EH Scotland UK Tel: (01382) 386364
>>
>> The University of Dundee is a registered Scottish Charity, No: SC015096
>> _______________________________________________
>> ome-devel mailing list
>> ome-devel at lists.openmicroscopy.org.uk <mailto:ome-devel at lists.openmicroscopy.org.uk>
>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel <http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel>
> Michael Ellis (Managing Director)
> Digital Scientific UK Ltd.
> http://www.dsuk.biz <http://www.dsuk.biz/>
> michael.ellis at dsuk.biz <mailto:michael.ellis at dsuk.biz>
> tel: +44(0)1223 911215 <tel:+44%201223%20911215>
>
> The Commercial Centre
> 6 Green End
> Cambridge
> CB23 7DY
>
>
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk <mailto:ome-devel at lists.openmicroscopy.org.uk>
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel <http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel>
>
>
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
Michael Ellis (Managing Director)
Digital Scientific UK Ltd.
http://www.dsuk.biz <http://www.dsuk.biz/>
michael.ellis at dsuk.biz
tel: +44(0)1223 911215
The Commercial Centre
6 Green End
Cambridge
CB23 7DY
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20161202/7c89568b/attachment.html>
More information about the ome-devel
mailing list