[ome-users] Fwd: Request for volunteers interested in integration with CellProfiler 2

[Josh Moore]

Forwarding to the list.
~Josh

Begin forwarded message:

> From: Adam Fraser
> Date: March 10, 2011 5:20:10 PM GMT+01:00
> To: Felix Meyenhofer
> Cc: Josh Moore
> Subject: Re: [ome-users] Request for volunteers interested in integration with CellProfiler 2
> 
> Hi Felix,
> 
> Thank you for your detailed reply, this is exactly the kind of context that
> I'm looking for from OMERO and CP users. It's nice to know what people are
> doing out there, and where they're running into problems. I've forwarded
> your email to some people in my group to get some feedback on your idea or
> writing OME-TIFFs back to OMERO from CP. This is definitely an achievable
> goal, if not by us, then at least by someone with a checkout of the
> CellProfiler source and some Python programming experience.
> 
> At any rate, I've begun writing a CP module (which may end up being multiple
> modules) for writing to OMERO from CP. We'd basically like to be able to
> tag, rate, and attach data to OMERO images that were imported using the
> OMEROLoadImages module that was contributed to us by Bram Gerritsen et al.
> at NKI. When this work is done I should have a better idea of whether I'll
> have the time to implement the feature you've suggested.
> 
> Thanks for joining the conversation,
> Adam
> 
> On Wed, Mar 9, 2011 at 12:13 PM, Felix Meyenhofer <meyenhof at mpi-cbg.de>wrote:
> 
>> Hi Josh, hi Adam
>> 
>> 
>> it has been some time that I wanted to reply to your mail. Things have
>> changed in the mean time, in that I had the occasion to talk to Josh and get
>> a short overview over the possible features of an OME-integration in
>> CellProfiler. However let me start by telling the story before the most
>> recent events.
>> 
>> Here at the MPI-CBG we're using CellProfiler for a while now, and since the
>> release of CP2 the usage is increasing. I'm also part of the OME mailing
>> list to observer the progress. Until now the following issues kept us from
>> seriously considering centralizing the image storage and put all the image
>> data in a uniform format:
>> 1) We have multiple microscopes and for fluid workflows we try to use the
>> image processing solutions that come with it. Importing the image data into
>> OME, would either mean data duplication or we risk, that the software from
>> the microscope can't access the data according it's own conventions.
>> 2) We deal with TB of data a month. Importing the images into OME, would
>> mean an additional step in our pipelines that consumes considerable amount
>> of time (especially in setting it up).
>> 3) Our data is scattered among several project spaces, that reside on
>> different server units. Implementing a OME image server for our purposes
>> would probably mean that we need to buy a new machine having sufficient
>> storage capacity.
>> 4) We are also afraid, that one big and complex system for all images might
>> makes us as a facility more vulnerable in terms of stability. De-centralized
>> structures have the advantage that one can avoid standing still during
>> downtime by switching to other projects.
>> 
>> I'm no specialist in database systems, but as far as I understand such an
>> enterprise would occupy at least one Geek full time for quite while. The
>> migration of existing data will be a big task too. Administrating the system
>> is also quite a lot of work. In other words the investment in work and
>> hardware is quite intimidating.
>> 
>> So far the scary aspects of the matter...
>> 
>> 
>> What is keeping me curious, is the fact that, we so fare do not have an
>> elegant and universal solution to go back to image inspection after the
>> screen is analyzed. Depending on the microscope we used and the image
>> processing pipeline setup with Acapella or Harmony (PerkinElmer),
>> BioApplications (Cellomics), CellProfiler, or MotionTracking - Kalaimoscope
>> the process of pulling up the images from a particular project can be more
>> or less pain full. To enable the analyst to switch swiftly between
>> quantitative data, images and its objects would be indeed a great tool to
>> get the most out of the data.
>> Another aspect is the continuously available metadata, that could simplify
>> or even take away some work in parametrizing the image processing pipelines.
>> 
>> Now concerning the OME integration in CellProfiler, this could be very
>> interesting indeed. I'm not sure what you are aiming for, but a scenario
>> that appeals very much to me and now seems to be reachable is:
>> CellProfiler would do the image processing, convert the images into the
>> ome-tif, where it would also add the segmentation results and the object
>> quantification. The ome-tiffs could than be easily imported into an OME-DB.
>> For further analysis we mostly use KNIME. We then could implement some nodes
>> to import the data from the OME-server and to the data analysis. The nice
>> thing about having the raw images, roi's and the object quantification
>> together in one DB would allow to do object classification in KNIME by only
>> adding a training-node that handles the image data and allows the user to
>> interact with it (classify). Also it would be a lot easier to pull up the
>> images once the interesting/significant events where determined to validate
>> them by image and segmentation inspection. Finally one could use OMERO
>> archiving mechanism for images that do not show any significant results, but
>> keep the "hits".
>> 
>> For me it would be a good starting point to have a complete solution for
>> HCS campaigns using CellProfiler and KNIME (KNIME, since it is more generic
>> than the CellAnalyst). In other words, this way we could provide similar
>> (better) data handling when processing screens with CP2 as do other
>> platforms like Cellomics.
>> 
>> To wrap this up: unfortunatly I can't contribute with my own experiences of
>> integrating OME into a HCS data processing pipeline but I'm hoping that I'm
>> about to collect some in the near future where I plan to catch up with on
>> your current work (Thanks for the link Josh).
>> In any case I would be very interested on what concept you are pursuing
>> Adam and if what I described is contained in it.
>> 
>> Best regards
>> Felix
>> 
>> 
>> 
>> On Feb 9, 2011, at 10:54 PM, Adam Fraser wrote:
>> 
>> Thanks to Josh for sending this out, I guess I'll chime in briefly.
>> 
>> First of all, this is a great opportunity for people who:
>> 
>>   - are using OMERO to organize and access your HCS data, and you would
>>   like an easier way to get your data into CellProfiler 2.0 for analysis.
>>   - are using CellProfiler for HCS image analysis and you'd benefit from
>>   a centralized data repository with easy-to-use tools for browsing and
>>   annotating your data.
>> 
>> I've been a software engineer working in Anne Carpenter's group<http://www.broadinstitute.org/~anne/people.html> at
>> the Broad Institute in Cambridge, MA for the past 3 years. I'm currently
>> working to bridge the gap between OMERO and our current version of
>> CellProfiler <http://cellprofiler.org/>, but unfortunately I have yet to
>> find anyone who is using both tools or at least would like to. If this is
>> something that interests you (anyone!), I'd love to hear your thoughts and
>> ideas.
>> 
>> I'm currently getting started by porting the existing OMERO CP modules<http://trac.openmicroscopy.org.uk/omero/wiki/OmeroCellProfiler> from
>> CP1.0 to CP2.0. I've gotten the impression that this is something useful
>> that people have been meaning to do, but once again, I'd love to hear from
>> people with real use cases.
>> 
>> Thanks,
>> Adam Fraser
>> 
>> On Fri, Feb 4, 2011 at 4:23 AM, Josh Moore <josh at glencoesoftware.com>wrote:
>> 
>>> Hi list,
>>> 
>>> Adam Fraser from Broad Institue (CC'd) has started investigating the
>>> interface between CellProfiler / CellProfiler Analyst 2.0 and OMERO. We are
>>> certainly willing (and hopefully able!) to help him on the technical side of
>>> the integration, but were wondering if anyone could make suggestions on how
>>> to most quickly and effectively bring CP & CPA into a real OMERO user's
>>> workflow. "How is your data laid out?", "How has it been tagged &
>>> classified?", "How do you want to visualize and interact with your
>>> results?", "How should results be stored?" are among the questions which it
>>> might be good to have answers to. If you have time and are interested,
>>> either contact Adam directly or email the list.
>>> 
>>> As always, thanks for your support.
>>> ~Josh
>>