[ome-devel] Zeiss 710

Rubén Muñoz ruben.munoz at embl.de
Tue Oct 12 19:32:31 BST 2010 

Hi Curtis and Melissa,

On Oct 12, 2010, at 5:41 PM, Curtis Rueden <ctrueden at wisc.edu> wrote:

> Hi Rubén & Melissa,
> 
> > Is there a way of excluding the image/pixel list in all the OME.TIF with bfconvert?
> 
> Perhaps I am misunderstanding the question, but are you asking about
> excluding the Image and Pixels elements from the OME-XML metadata?  

Yes that's my question.

> If
> so, then I'm afraid that that is really not possible - both elements
> provide critical metadata for reading the pixel data.
> 

Thats how I understand the specification. All the files are cross-referenced in the OME.TIF headers by Pixel elements, independently of the number of files in the set...

Note: We store all the planes & timepoins & channels in different files for convenience, that's not a matter of discussion anymore here, since our software doesn't always operate on stacks.

> This could conceivably happen with extremely small image planes. For example, an 8x8 image at 8-bit resolution only occupies 64 bytes, but the metadata would be at least ~180 bytes; e.g.:
> 
> <Image ID="Image:xxxx" Name="Image xxxx"><Pixels DimensionOrder="XYCZT" ID="Pixels:xxxx" SizeC="1" SizeT="1" SizeX="8" SizeY="8" SizeZ="1" Type="uint8"><TiffData/></Pixels></Image>
> 
> I can't think of any other case where the metadata size exceeds the pixel size. Rubén, can you elaborate on what is going on? What are the dimensions of your images? How many screens, plates, wells, and what is the image resolution?

@Curtis: Actually we face this problem with very large datasets instead. One of our time-lapse acquisitions has 23000 ome.tif files, each is 8M = 2.5M + 5.5 M headers. This kind of storage can not be afforded by some groups.

My colleagues plan to make bigger experiments, and I we are facing storage issues. And bioformats needs more memory to convert large datasets. 

To split each serie in different ome.tif datasets seems like our solution. Do you think its convenient? Are there tools for this? I am working in one already.

@Melissa: Do you think that a tool that could group or split datasets could also deal with the Leica Matrix Screener OME.TIF. My tool will assume that all the headers are the same, but I know that is not like that. Therefore I think that unification of the headers is a good idea, in the future.

Thanks for your replies,

Rubén

> 
> -Curtis
> 
> On Tue, Oct 12, 2010 at 10:23 AM, Melissa Linkert <melissa at glencoesoftware.com> wrote:
> Hi Rubén,
> 
> > Is there a way of excluding the image/pixel list in all the OME.TIF with bfconvert?
> 
> Perhaps I am misunderstanding the question, but are you asking about
> excluding the Image and Pixels elements from the OME-XML metadata?  If
> so, then I'm afraid that that is really not possible - both elements
> provide critical metadata for reading the pixel data.
> 
> > For large multi-file sets the metadata is bigger than the pixel values and this prevents us to use OME.TIF in High Throughput Screens.
> 
> That is a bit strange, but I have a few ideas as to what might be
> happening there.  I imagine the OME-TIFF file is prohibitively large,
> but if you could send just the OME-XML metadata that would be a huge
> help.  You can extract the metadata like this:
> 
>  # if all of the Bio-Formats command-line tools are installed
>  $ tiffcomment /path/to/file.ome.tiff
> 
>  # if 'tiffcomment' is not installed
>  $ java loci.formats.tools.TiffComment /path/to/file.ome.tiff
> 
> > I'd like to help with the Leica Matrix Screener and ZEN importers as much as possible.
> 
> Thank you for the offer.  At the moment, there is not much that can be
> done with the Zeiss LSM/Zen reader.  It will be at least a few weeks
> before I can do anything with the Leica Matrix reader, but you are more
> than welcome to start on a reader in the meantime if you like.  There
> are three main pieces of work to be done for this reader:
> 
>  1) Make sure that all of the files are grouped together correctly, no
>  matter which file in the dataset is selected by the user.
> 
>  2) Make sure that all of the pixel data and 'core' metadata is read
>  correctly.
> 
>  3) Make sure that all of the OME-XML metadata from each of the
>  constituent OME-TIFFs is parsed and unified.
> 
> My plan is to roughtly model the Leica Matrix reader after
> loci.formats.in.MIASReader and loci.formats.in.InCellReader, with an
> internal loci.formats.in.OMETiffReader doing much of the work for (2)
> and (3).
> 
> Alternatively, you can wait a bit until I've had time to write an initial
> version, and then help with the testing and polishing.
> 
> Regards,
> -Melissa
> 
> On Mon, Oct 11, 2010 at 09:51:56PM +0200, Rubén Muñoz wrote:
> > Hi Melissa,
> > Is there a way of excluding the image/pixel list in all the OME.TIF with bfconvert?
> > For large multi-file sets the metadata is bigger than the pixel values and this prevents us to use OME.TIF in High Throughput Screens. I'd like to help with the Leica Matrix Screener and ZEN importers as much as possible.
> > Best regards,
> > Rubén
> >
> > On Oct 11, 2010, at 8:22 PM, Melissa Linkert <melissa at glencoesoftware.com> wrote:
> >
> > > Hi Rubén,
> > >
> > >> That's good news. I am amazed by how the project evolves from simpler to more sophisticated needs. Actually bfconvert was designed to convert I guess, but I was trying to transform Leica Matrix Screener multiple OME.TIF to one-big OME.TIF and then enrich the metadata.
> > >
> > > If possible, I would recommend waiting for this ticket to be closed:
> > >
> > > http://dev.loci.wisc.edu/trac/software/ticket/563
> > >
> > > ...at which point you can just do 'bfconvert leica-matrix-file.ome.tiff output.ome.tiff' without any additional magic.
> > >
> > >> To give dimensionally to a dataset is my goal. Could one edit the metadata with EditTiffComment for that? Would a tool like EditTiffComment that modifies the metadata of many files accordingly be usefull?
> > >
> > > You could use one of our TIFF comment editing utilities:
> > >
> > > * java loci.formats.tools.TiffComment -edit /path/to/file
> > > * java loci.formats.tools.EditTiffG /path/to/file
> > >
> > > But please be very, very cautious.  Again, the above ticket really is
> > > the best solution, so if you can I would recommend either waiting for
> > > that ticket to be closed or (if you have a lot of free time) helping to implement it.
> > >
> > > Regards,
> > > -Melissa
> > >
> > > On Wed, Oct 06, 2010 at 02:46:50PM +0200, Rubén Muñoz wrote:
> > >> Hi Melissa,
> > >>
> > >> On Oct 6, 2010, at 1:09 AM, Melissa Linkert wrote:
> > >>
> > >>> Hi Rubén,
> > >>>
> > >>>> Therefore tried this with bfconvert, even when appending to existing OME.TIF is not supported yet.
> > >>>
> > >>> Appending to existing OME-TIFF files is, in theory, supported.  However,
> > >>> the MetadataRetrieve object used by OMETiffWriter must be correctly set
> > >>> up, which unfortunately will not happen by repeatedly calling
> > >>> bfconvert.
> > >>
> > >> That's good news. I am amazed by how the project evolves from simpler to more sophisticated needs. Actually bfconvert was designed to convert I guess, but I was trying to transform Leica Matrix Screener multiple OME.TIF to one-big OME.TIF and then enrich the metadata.
> > >>
> > >>>
> > >>>> However wIth the latest revision of bioformats 7034 when I output the bfconvert to the same file consecutively, the file is always getting bigger in size.
> > >>>
> > >>> Yes, I see the same behavior.  It is ticketed here:
> > >>>
> > >>> http://dev.loci.wisc.edu/trac/software/ticket/578
> > >>>
> > >>> And, as usual, you have been CC'd.  I am a bit curious though: why run
> > >>> bfconvert repeatedly with the same output file?  Are you trying to
> > >>> convert multiple datasets to a single OME-TIFF file?  If I know the use
> > >>> case, then maybe there is a work-around until the above ticket is closed.
> > >>
> > >> To give dimensionally to a dataset is my goal. Could one edit the metadata with EditTiffComment for that? Would a tool like EditTiffComment that modifies the metadata of many files accordingly be usefull?
> > >>
> > >> Another topic, I guess you remember about the Zeiss ZEN software,  multiple positions in the same LSM file are now supported.
> > >> I recently found out that it also produces multi-file LSM datasets (without .mdb). I am getting one of these. I wonder how these files now about each other, but could be that they don't.
> > >>
> > >> Best regards,
> > >>
> > >> Rubén
> > >>
> > >>
> > >>
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20101012/d1ac0a21/attachment.html>

More information about the ome-devel mailing list