[ome-devel] Uploading ICS files into OMERO and OMERO client API questions
Jason Swedlow
jason at lifesci.dundee.ac.uk
Wed Aug 26 10:18:31 BST 2009
Nick et al--
The only thing to add to Curtis' detailed response is that supporting
FLIM is a real priority for us. We are actively working through FLIM
in the OME Data Model, Bio-Formats, and OMERO. This is because from a
modeling perspective, it forces us to confront serious limitations in
our existing model, and is a concrete example where we can get good
data to work through concepts. In addition, as you know, FLIM is an
incredibly powerful method, and one that the community needs, and
frankly, our own work depends on. Finally, it's a chance to show all
of our tools doing real work, and producing critical results for
research and discovery.
Please do keep pushing this. We'll have more to say specifically on
FLIM once our 4.1 release is out.
Cheers,
Jason
On 25 Aug 2009, at 23:44, Nick Perry wrote:
> Wow, thank you for that PHENOMENAL response.
>
> On Tue, Aug 25, 2009 at 11:02 PM, Curtis Rueden <ctrueden at wisc.edu>
> wrote:
> Hi Nick,
>
> 1. I noticed that I can successfully import .ics files for FLIM
> data. However, they can't be visualized within the OMERO server
> (understandably since I imagine the algorithm for reconstruction
> images isn't design for FLIM data).
>
> ICS is a flexible file format, capable of representing many
> dimensional organizations. While OMERO (Bio-Formats) is capable of
> importing ICS data, it may not get the dimensional structure right
> in every case. In the case of FLIM data, it is assuredly wrong,
> because the OME data model does not yet support the FLIM dimensions
> of phase & frequency (in the case of frequency domain) and time bin
> (in the case of time domain).
>
> You can use the Bio-Formats command line tools (http://www.loci.wisc.edu/ome/formats-tools.html
> ) to study how metadata is being mapped from the original structure
> into the standardized OME model. Try this command:
>
> showinf myData.ics -nopix -omexml
>
> The -nopix flag instructs Bio-Formats to skip reading any pixel
> data, and the -omexml flag instructs it to standardize the metadata
> into the OME data model.
>
> Towards the end of the output, you will see a block of text
> bracketed by "<OME>" and "</OME>" that describes the standardized
> dimensional structure. You can also check the SizeC, SizeZ and SizeT
> values in the "core metadata" output near the top. My guess is that
> at the moment, Bio-Formats is inappropriately compacting the phase
> and frequency dimensions into one or more of the Z, C and T
> dimensions, or else dropping them completely. Either way, the
> information subsequently pulled into the OMERO database is invalid.
>
> First, in terms of the command line tool: I tried it out earlier
> today trying to see if I could convert ics 1.0 and ics 2.0 files of
> the lifetime data into ome-tifs and just put them in the OMERO
> server (a la 'smart server' concept). However, both times the
> created ome-tif files were created (1 from ics 1.0, the other from
> ics 2.0), both files were uploaded successfully to the server (ie,
> no errors) but neither of them ever showed up in importer. Also, I
> ran the validation tool provided by the command line tool, and
> neither file passed due to xml tags being missing and such. if
> you're interested (perhaps a bug?) i could send you the error
> otherwise i'll assume it's because it was the lifetime FLIM data and
> there are some missing necessary meta-data needed to constitute a
> 'readable' ome-tif file. Note: the file i used contained X, Y, Z, T,
> and Channel. There was no phase/frequency in this file. So i would
> of thought it could be converted correctly. Interestingly, Heygens
> could create a readable ome-xml file from this and i could as least
> see it in OMERO with insight.
>
>
> Now, until FLIM data, which is 7 dimensional, is somehow fit into
> the OME-TIF data model (which would likely require a change in the
> OME-TIF specification itself), I wonder if the FLIM data can just be
> stored on OMERO servers as .ics files?
>
> OMERO is capable of preserving the original data files. However, its
> primary function is to digest the data (both pixels and metadata)
> into a standardized form. Bio-Formats converts the metadata into an
> OME data structure, which is then used to populate the OMERO
> database. It also decodes the pixels and stores them into a raw
> pixels repository on disk, for efficient access later.
>
> Since the OME data model does not yet support the FLIM-specific
> dimensions, this digestion will mangle the dimensional parameters of
> your data
>
> Right. I was thinking more along the lines of skipping the
> bioformats stage at this point until the ome-xml is expanded to
> allow more dimensions (for reasons I discuss below)
>
>
> Does the OMERO server do anything to change these files as it tries
> to interpret them? Or does it just store it like any data storage
> server would? Essentially, if I put the FLIM data which is
> incompatible to any sort of function OMERO uses right now into .ics
> files and upload them (they upload successfully according to
> importer) would OMERO just store them as they are and not tamper
> with the files in any way?
>
> Hopefully the above explanation answers these questions. If not,
> please ask for clarification and we will be happy to explain in more
> detail.
>
>
> Am I correct in my understanding that if skip bioformats my raw data
> is NOT digested, and thus i get the unmodified ics version on OMERO?
>
> If so, could I then access these files through API with analysis
> software acting as an OMERO client and edit the files themselves on
> the server?
>
> With the OMERO API, you can: A) access the digested data for further
> analysis with your software; and B) access the original, untouched
> ICS data.
>
> In the case of A, the digested data is invalid, due to the lack of
> support for FLIM. While your software could theoretically untangle
> any mangling that occurred during import, it would make more sense
> to wait until OMERO properly supports the FLIM dimensions.
>
> In the case of B, your software could process the returned ICS data
> stream exactly like an ICS file on disk. But in that case, it seems
> unnecessary to go through the rigamarole of storing the files in
> OMERO in the first place. Why not use a regular networked file
> system instead?
>
> Because figuring out how to play with the API furthers my goals even
> if I'm working with incorrect data. Essentially, I imagine I could
> get complete access to the LI-FLIM analysis software, which can work
> with both ics AND eventually the new ome-xml/tif files. So I just
> need to get the connection hooked up and that's half of the battle.
>
>
> 2. If I have software available that analyzes the specific FLIM data
> we use, how long would it take (best guess would do here; I just
> need some sort of concept of the scale of this type of project) to
> interface the OMERO server with this software via the OMERO API?
>
> It depends on the software, and the type of integration desired. If
> it is closed source, proprietary software from a vendor, then you
> have no control over its feature set. However, assuming you are
> talking about in-house and/or open source software, you have a way
> forward. One of the cool things about OMERO is its use of Ice (http://www.zeroc.com/ice.html
> ), allowing cross-platform client access to the server from a number
> of different programming paradigms including C++, Java, .NET,
> Python, PHP, Ruby, and Objective-C.
>
> There are different kinds of clients, as well. The server is capable
> of delivering compressed, rendered image planes on demand, reducing
> the processing burden on the client as well as the necessary network
> bandwidth -- this approach is how the OMERO.insight client works.
> But the server can also deliver raw image planes for the client to
> use with completely custom processing.
>
> See above.
>
>
> I would also like to respond to some points you raised in an email
> to Kevin:
>
> Part 1) Get LI-FLIM data inside OMERO (what you called the smart
> file server).
> Part 2) Add support for LI-FLIM analysis be in it the form of a
> module, direct implementation into the source code of OMERO, etc (I
> am more unclear at this point how this portion would work).
>
> I agree that these two steps are what is needed. However,
> accomplishing Part 1 requires an extension to the OMERO data model,
> as described above, since OMERO does more than just storing original
> data.
>
> In an email to Sander, I re-broke-down how I see the problem:
>
> 1. The OME-TIF data model would be fixed to allow reference/sample
> data.
> 2. LI-FLIM software would output reference/sample data as OME-TIF.
> 3. LI-FLIM software would output lifetime data as OME-TIF.
> 4. Visualization software would be implemented inside of OMERO so
> that reference/sample/lifetime images can be correctly visualized
> once connected to the server (internally, this is all the server can
> do; it just shows the picture with basic image editing functions;
> any true image analysis happens outside the server with user built
> clients).
> 5. Use the OMERO server API to connect the EXISTING LI-FLIM software
> to the OMERO server to perform analysis (new idea I just had that
> I'm pretty sure works but need to confirm).
>
> Essentially, I mention that (1) must happen before (2) can happen.
> (3) can happen without (1) but would eventually change as (1)
> changes. However, I don't see working on 3 being a step sideways - i
> imagine coding that solution would expedite getting lifetime data
> into the new ome-xml specification when that happens too, so it'
> just a small step forwards but forwards nonetheless. (4) would
> probably need to happen after 1-3, and (5) is something I think
> could happen now as well (the API exists, I have the LI-FLIM software)
>
>
> Part 1 is ideally what I could contribute to during the rest of my
> stay. Here is a more in-depth explanation of what I'm thinking:
>
> When you use the LI-FLIM system, you take both Sample and Reference
> images (stored in two separate files sample.fli and reference.fli).
> Each of these images is 7 dimensional in terms of the file format.
>
> The first step on our end is to add support for the LI-FLIM (.fli)
> file format. We have the specification and some sample data from
> Sander de Jong, so it is only a matter of time until we have an
> initial implementation. I am planning to work on it later this week,
> actually.
>
> However, Bio-Formats must be able to successfully map the additional
> dimensions into the OME data model before the files can be robustly
> imported into an OMERO server. This means extending the OME-XML data
> model. The OMERO developers have been actively discussing how to do
> so in recent weekly meetings, passing proposals back and forth,
> etc., and will share these with the community as soon as we have
> something reasonable.
>
> Once you have collected the sample and reference files, right now
> you load them into the LI-FLIM analysis software and IMMEDIATELY a
> lifetime image is created. No analysis is done at this point. From
> this point, everything you need for analysis is contained within the
> lifetime file. This lifetime file is also the same file format as
> the sample and reference files were.
>
> On a side-note, just for completeness, it seems that the reason the
> LI-FLIM file format is 7 dimensions is so that the format is broad
> enough to hold reference/sample data and lifetime data, which are
> fundamentally different.
>
> Right; I have not yet seen a file that actually uses all 7
> dimensions simultaneously.
>
> Reference/sample data only use X, Y, Z, T, Phase, and Frequency
> (channel is always 1, at least as far as I can tell with my demo
> software, so Sander should confirm). These files are therefore 6
> dimensional.
>
> In the case of combined spectral lifetime (SLIM) data, where the
> emission data is split across multiple wavelengths, channels could
> be greater than one. Not sure if any Lambert systems are capable of
> this, but we (LOCI) have a home-built system based on the Becker &
> Hickl time domain approach that does SLIM.
>
> I am not sure but I will get back to you on this.
>
>
>
> Lifetime data uses only X, Y, Z, T, Channel (phase and frequency are
> always 1, but again need confirmation). So lifetime.fli files are 5
> dimensional, and use the same dimensions as the OME-TIF. So as far
> as CRUCIAL metadata is concerned, the OME-TIF format suffices.
>
> Right. Fundamentally, "lifetime data" means that the pixel values
> have been computed to match the expected lifetime values.
> Presumably, in the case of multiple lifetime values (e.g., two-
> component), the lifetime data files would have multiple channels?
> Also, what about other parameters such as maximum intensity at each
> pixel, or background at each pixel? (These values are more
> applicable to time domain; I am still getting my head around
> frequency domain so perhaps they are irrelevant to the frequency
> approach.)
>
> Yes. In the lifetime files I received from Sander (and I believe you
> did as well) the extended files had lifetime 1 and lifetime 2, as
> well as some other channels. I don't believe there is background and
> I'm not sure about max intensity, so I will get back to you with
> answers to those as well as soon as I confirm.
>
>
>
> The reason I thought it is useful to be able to put lifetime data
> into OMERO and be able to analyze that, as opposed to reference/
> sample data, is that as far as I can tell, the computer connected to
> the LI-FLIM hardware itself needs to run the LI-FLIM software.
>
> There is no reason you shouldn't be able to perform analysis of the
> reference/sample data after the fact. So ideally we would like to
> support storing the reference/sample data into the OMERO system.
>
> Yeah I completely misunderstood how analysis of FLIM works. Sander
> corrected me :)
>
>
> I don't get the impression OMERO can control microscopes, but
> perhaps I'm wrong.
>
> You are correct. OMERO is not intended as a tool for acquisition,
> though we have made an effort to support data from many different
> acquisition systems, including the open source Micro-Manager software.
>
> As such, because we still need to be able to go through the LI-FLIM
> software at least initially, why not just let that software compile
> the raw sample/reference data into the lifetime data, which can
> readily fit into the existing OME data model since it uses the same
> 5 dimensions? This would save the hassle (at least for now) of re-
> designing the entire datamodel for OME-TIF.
>
> Sure, makes sense.
>
> Once the lifetime image is collected on the LI-FLIM software running
> the hardware, then, that software could just save the lifetime data
> into the OME-TIF format since that's the only important file for
> image analysis at this point (per Sander's confirmation), and save
> the reference/sample data as .ics. Then all 3 files could be put
> into OMERO which would ultimately develop analysis tools (Sander
> could help here?) for analyzing the lifetime file.
>
> Storing the reference/sample data as ICS and into the OMERO server
> is asking for trouble at the moment, as explained above.
>
> BUT, it would also work for now if we have need of storing files
> immediately (right?).
>
>
> That's how I figured the workflow would work anyway - the LI-FLIM
> software connected to the FLIM hardware just outputs the lifetime
> file, which would go into OMERO for storage and analysis, and which
> stores reference/sample .ics files for completeness.
>
> There is an attachment mechanism in OMERO, to store arbitrary binary
> data (such as PDFs, PPTs, etc.) along with an image. If you are
> intent on this workflow, perhaps it would make more sense to store
> the original FLI files as binary attachments, rather than parsing
> them in any way at the moment. Then they would still be recoverable
> for processing with the LI-FLIM software at a later time.
>
> I think this workflow might make sense since at least I would be
> contributing to forward progress.
>
>
> This also solves the metadata storage issue because all the metadata
> from the reference/sample files is in the lifetime file as well. And
> this data can easily just be put into the XML part of the OME-TIF
> (it's hardware stuff only, like something=x, something=y, etc.).
>
> We still need to define a mapping from the FLI hardware metadata
> into the standard OME data model. Best would be if the LI-FLIM
> software could export to OME-TIFF -- but it doesn't do that now,
> does it? We will also want to do the mapping in Bio-Formats, as an
> FLI file format reader, so that OMERO (and other software packages
> that use Bio-Formats) can import the original FLI files properly.
>
> Is there not a CustomAttribute tag? I thought I saw one earlier
> today. I recognize that this is not the permanent solution but if I
> were to work on getting the lifetime files into ome-tif form I could
> just throw LI-FLIM specific metadata for hardware into that tag, yes?
>
> Basically, to summarize all of the above (since it was getting a
> little choppy):
>
> The OME-XML format needs to change to accommodate additional
> dimensions (for FLIM in this case, phase and frequency). However,
> this solution will NOT happen in my timeframe; I leave in 3 weeks.
> To at least do SOMETHING to help, I thought it would be beneficial to:
>
> 1) Help Sander output lifetime FLIM files in OME-TIF as the file
> format is specified currently, since this code would also probably
> be useful in converting lifetime FLIM files into the new OME-XML
> whenever it is released (and thus represents forward progress even
> if it's not a complete solution).
>
> 2) Work on hooking up the LI-FLIM analysis software to the OMERO
> server via the provided API. This would need to happen anyway to
> analyze the LI-FLIM data.
>
> The extra perk of both of these in addition to providing useful code
> is that no matter how we store the FLIM data right now (since the
> OME-XML might take a while) the Plateforme can start using OMERO to
> store data and run analysis off of it. So yes, the reference/sample
> data will be in either ICS/binary, and the lifetime will be in an
> eventually out-dated OME-XML, but we can at least use the benefits
> of the 'smart file server' provided by OMERO while at the same time
> create code that could ultimately be used to help output OME-TIF
> files directly from LI-FLIM software as well as work with the API
> for analysis clients for FLIM.
>
>
>
> HTH,
> Curtis
>
> Again, thank you SO MUCH for the response. It was incredibly helpful!
>
> Nick
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
**************************
Wellcome Trust Centre for Gene Regulation & Expression
College of Life Sciences
MSI/WTB/JBC Complex
University of Dundee
Dow Street
Dundee DD1 5EH
United Kingdom
phone (01382) 385819
Intl phone: 44 1382 385819
FAX (01382) 388072
email: jason at lifesci.dundee.ac.uk
Lab Page: http://www.dundee.ac.uk/lifesciences/swedlow/
Open Microscopy Environment: http://openmicroscopy.org
**************************
The University of Dundee is a Scottish Registered Charity, No. SC015096.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20090826/fa9a1fba/attachment-0001.htm
More information about the ome-devel
mailing list