[ome-devel] data grouping/searching, creating dataset

[Harry Hochheiser]

Bernd:

Interesting stuff: some comments below.

On Nov 23, 2005, at 2:21 PM, Bernd Jagla wrote:
> Then we want to display a particular feature for all the wells/ 
> plates as a heat map to identify by eye any biases or problems in  
> the data set. For this we envision right now using Partek’s  
> Screener Solution which could interface with OME through ODBC. A  
> simple (complex) select statement should bring all the necessary  
> information into the Screener Solution.

As a data visualization guy, I'm interested in what you're doing here  
- can you provide more detail?
When you say heat map, I assume that the feature for the wells/plates  
would be something that has a numeric value? Do you want to do  
dendrograms, or is outlier detection your main task?

> Then we have to select hits and analyze the screen:
>
> Display a particular feature for all wells and plates. (This is for  
> quality control).
>
> Select a relevant feature and a threshold.
>
> Write back the results (somehow) to OME. Those results will  
> eventually group the results in different categories (hit, non-hit,  
> not-known, positive control, negative control, etc). This actually  
> still has to be solved.
Custom STs, or CategoryGroup/Category annotations can help.

> Then we want to see/ cross-link the results with other screens: of  
> these hits, which other assays where they active in? Which  
> compounds where active in a given set of screens? Give me a  
> complete profile of all the compounds and in which assays they  
> where active, given a specified threshold etc.

when you say hit, do you mean image? ie, which other datasets/ 
experiments is a given image in? Also, can you clarify threshold?

> Of course at some point we want to see the compounds with the  
> results and everything but this will wait a while and needs  
> probably further discussions on how to integrate all this.
right.
> Then we have to push some of the data to PubChem at the NIH. How  
> could we do this? A simple script with select statements could do  
> the work, no? Maybe we can write a chain module that automatically  
> propagates the data to PubChem (they use XML – their own format of  
> course).
>

One possibility would be to use OME's exporting facilities to  
generate OME XML files and then XSLT to translate into PubChem's schema.

>
>
> At the moment I believe that the most important part is to improve  
> (or for me to learn how to do) the grouping features of OME. I  
> don’t think it is good that you have to know in advance that you  
> want to add something to a data set. From my experience I would  
> rather play with the data and mark e.g. bad wells along the way,  
> select images that look interesting in one way or the other and  
> say: Wow they look cool lets make a new dataset and collect them  
> here, or lets add them to the list of interesting data. As far as I  
> know this is only possible once you know the name of the data set,  
> and then you have to search for them again.
there are lots of ways to think about this sort of functionality -  
we've discussed pieces of it. The closest thing that is implemented  
is the category group/category annotation tool, but that does require  
starting from a selected set of images.
>
>
> Also for the sake of better understanding can you please point me  
> to all the relevant information concerning data sets: how to  
> programmatically do it, how to do it with ODBC, and the web interface?
The best places to start from are in the "Developer's documentation"  
section of the web site links.  We can give more feedback for more  
specific questions.
>
>
> I would be more than happy to explain in more detail or discuss our  
> strategies as we are just in the process of getting started…
>
>
>
> Thanks a lot for all your kind help.
>
>
You're very welcome...
-harry

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20051128/7829b346/attachment-0001.htm