[ome-devel] Uploading ICS files into OMERO and OMERO client API questions

[Nick Perry]

Wow, thank you for that PHENOMENAL response.

On Tue, Aug 25, 2009 at 11:02 PM, Curtis Rueden <ctrueden at wisc.edu> wrote:

> Hi Nick,
>
> 1. I noticed that I can successfully import .ics files for FLIM data.
>> However, they can't be visualized within the OMERO server (understandably
>> since I imagine the algorithm for reconstruction images isn't design for
>> FLIM data).
>
>
> ICS is a flexible file format, capable of representing many dimensional
> organizations. While OMERO (Bio-Formats) is capable of importing ICS data,
> it may not get the dimensional structure right in every case. In the case of
> FLIM data, it is assuredly wrong, because the OME data model does not yet
> support the FLIM dimensions of phase & frequency (in the case of frequency
> domain) and time bin (in the case of time domain).
>
> You can use the Bio-Formats command line tools (
> http://www.loci.wisc.edu/ome/formats-tools.html) to study how metadata is
> being mapped from the original structure into the standardized OME model.
> Try this command:
>
>   showinf myData.ics -nopix -omexml
>
> The -nopix flag instructs Bio-Formats to skip reading any pixel data, and
> the -omexml flag instructs it to standardize the metadata into the OME data
> model.
>
> Towards the end of the output, you will see a block of text bracketed by
> "<OME>" and "</OME>" that describes the standardized dimensional structure.
> You can also check the SizeC, SizeZ and SizeT values in the "core metadata"
> output near the top. My guess is that at the moment, Bio-Formats is
> inappropriately compacting the phase and frequency dimensions into one or
> more of the Z, C and T dimensions, or else dropping them completely. Either
> way, the information subsequently pulled into the OMERO database is invalid.
>

First, in terms of the command line tool: I tried it out earlier today
trying to see if I could convert ics 1.0 and ics 2.0 files of the lifetime
data into ome-tifs and just put them in the OMERO server (a la 'smart
server' concept). However, both times the created ome-tif files were created
(1 from ics 1.0, the other from ics 2.0), both files were uploaded
successfully to the server (ie, no errors) but neither of them ever showed
up in importer. Also, I ran the validation tool provided by the command line
tool, and neither file passed due to xml tags being missing and such. if
you're interested (perhaps a bug?) i could send you the error otherwise i'll
assume it's because it was the lifetime FLIM data and there are some missing
necessary meta-data needed to constitute a 'readable' ome-tif file. Note:
the file i used contained X, Y, Z, T, and Channel. There was no
phase/frequency in this file. So i would of thought it could be converted
correctly. Interestingly, Heygens could create a readable ome-xml file from
this and i could as least see it in OMERO with insight.


>
>  Now, until FLIM data, which is 7 dimensional, is somehow fit into the
>> OME-TIF data model (which would likely require a change in the OME-TIF
>> specification itself), I wonder if the FLIM data can just be stored on OMERO
>> servers as .ics files?
>
>
> OMERO is capable of preserving the original data files. However, its
> primary function is to digest the data (both pixels and metadata) into a
> standardized form. Bio-Formats converts the metadata into an OME data
> structure, which is then used to populate the OMERO database. It also
> decodes the pixels and stores them into a raw pixels repository on disk, for
> efficient access later.
>
> Since the OME data model does not yet support the FLIM-specific dimensions,
> this digestion will mangle the dimensional parameters of your data
>

Right. I was thinking more along the lines of skipping the bioformats stage
at this point until the ome-xml is expanded to allow more dimensions (for
reasons I discuss below)


>
> Does the OMERO server do anything to change these files as it tries to
>> interpret them? Or does it just store it like any data storage server would?
>> Essentially, if I put the FLIM data which is incompatible to any sort of
>> function OMERO uses right now into .ics files and upload them (they upload
>> successfully according to importer) would OMERO just store them as they are
>> and not tamper with the files in any way?
>
>
> Hopefully the above explanation answers these questions. If not, please ask
> for clarification and we will be happy to explain in more detail.
>
>

Am I correct in my understanding that if skip bioformats my raw data is NOT
digested, and thus i get the unmodified ics version on OMERO?


>  If so, could I then access these files through API with analysis software
>> acting as an OMERO client and edit the files themselves on the server?
>>
>
> With the OMERO API, you can: A) access the digested data for further
> analysis with your software; and B) access the original, untouched ICS data.
>
> In the case of A, the digested data is invalid, due to the lack of support
> for FLIM. While your software could theoretically untangle any mangling that
> occurred during import, it would make more sense to wait until OMERO
> properly supports the FLIM dimensions.
>
> In the case of B, your software could process the returned ICS data stream
> exactly like an ICS file on disk. But in that case, it seems unnecessary to
> go through the rigamarole of storing the files in OMERO in the first place.
> Why not use a regular networked file system instead?
>

Because figuring out how to play with the API furthers my goals even if I'm
working with incorrect data. Essentially, I imagine I could get complete
access to the LI-FLIM analysis software, which can work with both ics AND
eventually the new ome-xml/tif files. So I just need to get the connection
hooked up and that's half of the battle.


>
> 2. If I have software available that analyzes the specific FLIM data we
>> use, how long would it take (best guess would do here; I just need some sort
>> of concept of the scale of this type of project) to interface the OMERO
>> server with this software via the OMERO API?
>>
>
> It depends on the software, and the type of integration desired. If it is
> closed source, proprietary software from a vendor, then you have no control
> over its feature set. However, assuming you are talking about in-house
> and/or open source software, you have a way forward. One of the cool things
> about OMERO is its use of Ice (http://www.zeroc.com/ice.html), allowing
> cross-platform client access to the server from a number of different
> programming paradigms including C++, Java, .NET, Python, PHP, Ruby, and
> Objective-C.
>
> There are different kinds of clients, as well. The server is capable of
> delivering compressed, rendered image planes on demand, reducing the
> processing burden on the client as well as the necessary network bandwidth
> -- this approach is how the OMERO.insight client works. But the server can
> also deliver raw image planes for the client to use with completely custom
> processing.
>

See above.


>
> I would also like to respond to some points you raised in an email to
> Kevin:
>
> Part 1) Get LI-FLIM data inside OMERO (what you called the smart file
>> server).
>> Part 2) Add support for LI-FLIM analysis be in it the form of a module,
>> direct implementation into the source code of OMERO, etc (I am more unclear
>> at this point how this portion would work).
>>
>
> I agree that these two steps are what is needed. However, accomplishing
> Part 1 requires an extension to the OMERO data model, as described above,
> since OMERO does more than just storing original data.
>

In an email to Sander, I re-broke-down how I see the problem:

1. The OME-TIF data model would be fixed to allow reference/sample data.
2. LI-FLIM software would output reference/sample data as OME-TIF.
3. LI-FLIM software would output lifetime data as OME-TIF.
4. Visualization software would be implemented inside of OMERO so that
reference/sample/lifetime images can be correctly visualized once connected
to the server (internally, this is all the server can do; it just shows the
picture with basic image editing functions; any true image analysis happens
outside the server with user built clients).
5. Use the OMERO server API to connect the EXISTING LI-FLIM software to the
OMERO server to perform analysis (new idea I just had that I'm pretty sure
works but need to confirm).

Essentially, I mention that (1) must happen before (2) can happen. (3) can
happen without (1) but would eventually change as (1) changes. However, I
don't see working on 3 being a step sideways - i imagine coding that
solution would expedite getting lifetime data into the new ome-xml
specification when that happens too, so it' just a small step forwards but
forwards nonetheless. (4) would probably need to happen after 1-3, and (5)
is something I think could happen now as well (the API exists, I have the
LI-FLIM software)


>
> Part 1 is ideally what I could contribute to during the rest of my stay.
>> Here is a more in-depth explanation of what I'm thinking:
>>
>> When you use the LI-FLIM system, you take both Sample and Reference images
>> (stored in two separate files sample.fli and reference.fli). Each of these
>> images is 7 dimensional in terms of the file format.
>>
>
> The first step on our end is to add support for the LI-FLIM (.fli) file
> format. We have the specification and some sample data from Sander de Jong,
> so it is only a matter of time until we have an initial implementation. I am
> planning to work on it later this week, actually.
>
> However, Bio-Formats must be able to successfully map the additional
> dimensions into the OME data model before the files can be robustly imported
> into an OMERO server. This means extending the OME-XML data model. The OMERO
> developers have been actively discussing how to do so in recent weekly
> meetings, passing proposals back and forth, etc., and will share these with
> the community as soon as we have something reasonable.
>
> Once you have collected the sample and reference files, right now you load
>> them into the LI-FLIM analysis software and IMMEDIATELY a lifetime image is
>> created. No analysis is done at this point. From this point, everything you
>> need for analysis is contained within the lifetime file. This lifetime file
>> is also the same file format as the sample and reference files were.
>>
>> On a side-note, just for completeness, it seems that the reason the
>> LI-FLIM file format is 7 dimensions is so that the format is broad enough to
>> hold reference/sample data and lifetime data, which are fundamentally
>> different.
>>
>
> Right; I have not yet seen a file that actually uses all 7 dimensions
> simultaneously.
>
> Reference/sample data only use X, Y, Z, T, Phase, and Frequency (channel is
>> always 1, at least as far as I can tell with my demo software, so Sander
>> should confirm). These files are therefore 6 dimensional.
>>
>
> In the case of combined spectral lifetime (SLIM) data, where the emission
> data is split across multiple wavelengths, channels could be greater than
> one. Not sure if any Lambert systems are capable of this, but we (LOCI) have
> a home-built system based on the Becker & Hickl time domain approach that
> does SLIM.
>

I am not sure but I will get back to you on this.


>
>
> Lifetime data uses only X, Y, Z, T, Channel (phase and frequency are always
>> 1, but again need confirmation). So lifetime.fli files are 5 dimensional,
>> and use the same dimensions as the OME-TIF. So as far as CRUCIAL metadata is
>> concerned, the OME-TIF format suffices.
>>
>
> Right. Fundamentally, "lifetime data" means that the pixel values have been
> computed to match the expected lifetime values. Presumably, in the case of
> multiple lifetime values (e.g., two-component), the lifetime data files
> would have multiple channels? Also, what about other parameters such as
> maximum intensity at each pixel, or background at each pixel? (These values
> are more applicable to time domain; I am still getting my head around
> frequency domain so perhaps they are irrelevant to the frequency approach.)
>

Yes. In the lifetime files I received from Sander (and I believe you did as
well) the extended files had lifetime 1 and lifetime 2, as well as some
other channels. I don't believe there is background and I'm not sure about
max intensity, so I will get back to you with answers to those as well as
soon as I confirm.


>
>
> The reason I thought it is useful to be able to put lifetime data into
>> OMERO and be able to analyze that, as opposed to reference/sample data, is
>> that as far as I can tell, the computer connected to the LI-FLIM hardware
>> itself needs to run the LI-FLIM software.
>
>
> There is no reason you shouldn't be able to perform analysis of the
> reference/sample data after the fact. So ideally we would like to support
> storing the reference/sample data into the OMERO system.
>

Yeah I completely misunderstood how analysis of FLIM works. Sander corrected
me :)


>
> I don't get the impression OMERO can control microscopes, but perhaps I'm
>> wrong.
>
>
> You are correct. OMERO is not intended as a tool for acquisition, though we
> have made an effort to support data from many different acquisition systems,
> including the open source Micro-Manager software.
>
> As such, because we still need to be able to go through the LI-FLIM
>> software at least initially, why not just let that software compile the raw
>> sample/reference data into the lifetime data, which can readily fit into the
>> existing OME data model since it uses the same 5 dimensions? This would save
>> the hassle (at least for now) of re-designing the entire datamodel for
>> OME-TIF.
>
>
> Sure, makes sense.
>
>
>> Once the lifetime image is collected on the LI-FLIM software running the
>> hardware, then, that software could just save the lifetime data into the
>> OME-TIF format since that's the only important file for image analysis at
>> this point (per Sander's confirmation), and save the reference/sample data
>> as .ics. Then all 3 files could be put into OMERO which would ultimately
>> develop analysis tools (Sander could help here?) for analyzing the lifetime
>> file.
>
>
> Storing the reference/sample data as ICS and into the OMERO server is
> asking for trouble at the moment, as explained above.
>

BUT, it would also work for now if we have need of storing files immediately
(right?).


>
>  That's how I figured the workflow would work anyway - the LI-FLIM software
>> connected to the FLIM hardware just outputs the lifetime file, which would
>> go into OMERO for storage and analysis, and which stores reference/sample
>> .ics files for completeness.
>
>
> There is an attachment mechanism in OMERO, to store arbitrary binary data
> (such as PDFs, PPTs, etc.) along with an image. If you are intent on this
> workflow, perhaps it would make more sense to store the original FLI files
> as binary attachments, rather than parsing them in any way at the moment.
> Then they would still be recoverable for processing with the LI-FLIM
> software at a later time.
>

I think this workflow might make sense since at least I would be
contributing to forward progress.


>
> This also solves the metadata storage issue because all the metadata from
>> the reference/sample files is in the lifetime file as well. And this data
>> can easily just be put into the XML part of the OME-TIF (it's hardware stuff
>> only, like something=x, something=y, etc.).
>>
>
> We still need to define a mapping from the FLI hardware metadata into the
> standard OME data model. Best would be if the LI-FLIM software could export
> to OME-TIFF -- but it doesn't do that now, does it? We will also want to do
> the mapping in Bio-Formats, as an FLI file format reader, so that OMERO (and
> other software packages that use Bio-Formats) can import the original FLI
> files properly.
>

Is there not a CustomAttribute tag? I thought I saw one earlier today. I
recognize that this is not the permanent solution but if I were to work on
getting the lifetime files into ome-tif form I could just throw LI-FLIM
specific metadata for hardware into that tag, yes?

Basically, to summarize all of the above (since it was getting a little
choppy):

The OME-XML format needs to change to accommodate additional dimensions (for
FLIM in this case, phase and frequency). However, this solution will NOT
happen in my timeframe; I leave in 3 weeks. To at least do SOMETHING to
help, I thought it would be beneficial to:

1) Help Sander output lifetime FLIM files in OME-TIF as the file format is
specified currently, since this code would also probably be useful in
converting lifetime FLIM files into the new OME-XML whenever it is released
(and thus represents forward progress even if it's not a complete solution).

2) Work on hooking up the LI-FLIM analysis software to the OMERO server via
the provided API. This would need to happen anyway to analyze the LI-FLIM
data.

The extra perk of both of these in addition to providing useful code is that
no matter how we store the FLIM data right now (since the OME-XML might take
a while) the Plateforme can start using OMERO to store data and run analysis
off of it. So yes, the reference/sample data will be in either ICS/binary,
and the lifetime will be in an eventually out-dated OME-XML, but we can at
least use the benefits of the 'smart file server' provided by OMERO while at
the same time create code that could ultimately be used to help output
OME-TIF files directly from LI-FLIM software as well as work with the API
for analysis clients for FLIM.


>
>
> HTH,
> Curtis
>

Again, thank you SO MUCH for the response. It was incredibly helpful!

Nick
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20090826/20fea5ad/attachment-0001.htm