[ome-devel] Shoola-back-end consistency issues

[Chris Allan]

On Wed, Jul 06, 2005 at 04:06:48PM -0400, Ilya Goldberg wrote:
> 
> On Jul 6, 2005, at 3:35 PM, Chris Allan wrote:
> 
> >On Wed, Jul 06, 2005 at 09:08:01PM +0200, Josh Moore wrote:
> >>Harry Hochheiser wrote:
> >>>Josh, et al
> >
> >Hiya all.
> >
> >What I really don't want to degrade this into is a situation where
> >*every* update to *anything* gets a new MEX. We'd have hundreds of
> >thousands of MEXs very quickly that way. Can you imagine the act of
> >linking 10,000 images from a high-throughput screen to one dataset and
> >generating 10,000+ "copy/modification" MEXs? By definition you've
> >modified the images too right? Does each deserve a MEX? Is it just the
> >entire operation that gets a MEX?
> 
> Actually, the only thing that gets modified here is the ImageDatasetMap 
> object, and you can have one MEX for any number of them.  Since the 
> ImageDatasetMap is a non-ST object now, it has no MEX.  If we make it 
> an ST, you can ask the server "Give me ImageDatasetMap objects created 
> since time X".
> 
> > When do they expire? Can I expire your
> >notification MEXs? Can you expire mine? If no one expires them how long
> >do they stay around?
> 
> What are notification MEXes?  If we do polling, then its a simple (and 
> fast) query to determine if certain things you're interested in have 
> changed since last you looked.  The client knows what its displaying 
> and what its interested in.  That's why nobody uses notification 
> systems.  Its not the server's business to know what the client is 
> displaying or what needs updating.  We should provide sufficient tools 
> for a client to quickly determine if it needs to update its state.  
> This is mostly true for STs because they are all time-stamped, but its 
> not true for all objects.

Nobody? Say what? I think you've spent far too much time looking at web
technology Ilya. All the "new breed" desktop searching systems use
notification to populate their databases for instance. Beagle uses the
Linux kernel's inotify API and Spotlight uses MacOS X's kernel
notification API. Notification systems have been available for
filesystems for ages and every modern operating uses them.

All GUI toolkits use notification in the form of events to communicate.
Without this you'd have some very unresponsive user interfaces.

Publisher-subscriber event based programming has been around since the
mid 90's and lots of people use it.

I think instead of providing sufficient tools for a client to quickly
determine if it needs to update its state we should avoid shoving these
responsibilities onto our client developers and provide them with
notification when things they're interested change. Notification systems
are all about making client development easier.

> 
> >
> >Artificially keeping state using the DB and Analysis Engine is a recipe
> >for disaster in my opinion. Not to mention a serious brain buster to
> >decide where modification notification boundaries lie.
> 
> Using the DB to keep state is perfectly fine (that's what they're for). 
>  I think that notification systems are brain-busters.  Too centralized. 
>  Just farm that stuff out - the best place to know what needs updating 
> is in the client, not in the server.

Yes it is, and the job of notifying is the job of the server, not the
client.

> 
> -Ilya

Ciao.

-Chris