[ome-devel] Explorations of alternative architectures for remote clients

[Harry Hochheiser]

Following up on discussions that we had at our March meeting in 
Baltimore, I've recently been playing around with  alternative designs 
for getting OME data from a remote database to remote clients.  
Currently, this is done with OME-JAVA and the OME data server code that 
is part of the Shoola distribution.  In an effort to see what might be 
gained from considering alternative tools, I've built a demonstration 
project that explores some possibilities that might be considered for 
future remote clients.

Before I go into the details, some credit where it is due: Josh Moore 
has been working along these lines for much longer than I have, and he 
provided some invaluable assistance. Many of the ideas below are 
directly attributable to him. Couldn't have done it without you, Josh. 
Thanks.

At Josh's suggestion, I have used a combination of the Jakarta Tomcat 
server and the spring framework to build a remote framework. On the 
back-end, this uses Spring's JDBC classes to turn SQL queries into 
instantiated data objects. One of Spring's 3 options for serializing 
and transmitting Serializable Java objects is then used for sending 
these data objects to a remote client, which can then work with them.

There are several advantages to this approach:

- It eliminates the need us to do our own management of serialization 
and deserialization. All of the OME-Java code and the corresponding 
XML-RPC Perl code on the server can be eliminated, in favor of 
open-source components (tomcat and spring) that can be used right out 
of the box.

- it's straightforward: the same classes and interfaces can be used 
both on the client and the server.

- It's simple: we get these advantages without having to buy into all 
of the complexity and overhead of an application server.

- It's flexible: working with tomcat and java provides us with a path 
for doing web development in JSPs and servlets, and generation of 
interfaces for SOAP is straightforward.

- Using Spring's "dependency injection"  (essentially extensive support 
for run-time configurability), I can switch from running the code over 
the network to a direct JDBC connection with one line of XML. This 
significantly speeds testing and development (Thanks to Josh for 
pointing me in this direction).

- Perhaps most importantly, it's _fast_. for relatively small queries, 
and with all code running on my laptop, i'm seeing an order of 
magnitude speed improvement over the OME-JAVA +XML-RPC + Perl back-end 
that we currently use.

there are also a few downsides:

- It still requires specification of queries and construction of DTO 
trees. Currently, much of this must be done by hand.  This requires 
some work, but (in my opinion) not much more than is required when a 
new DTO request is added to Shoola.

- The query/DTO construction code is exclusively on the back-end. In 
Shoola, a criteria query can be constructed on the client side and then 
sent to the server for processing. My approach doesn't currently handle 
this, but this is a classic flexibility vs. performance trade-off. The 
generality of the Perl Facades for processing arbitrary criteria for 
DTOs made it _slow_.

- nagging questions like Object uniqueness and client-side caching are 
left unresolved.

I don't want to say much more about this right now, but it's my 
impression that some of these concerns might be addressed by replacing 
the direct JDBC calls on the back-end with an object-relational mapping 
tool. It's my impression that Josh has been working on using Hibernate 
to provide this mapping,  with some success. i'll leave it to Josh to 
say more (or not) about this.

Of course, moving to an architecture like this would be a big step. M 
ore evaluation and planning would be needed before we could move ome 
clients to a platform like this, and there may be good reasons for not 
doing so.
we'd need to understand some more of the issues (particularly the role 
of Hibernate), and potential costs in more detail. Plus, we'd  need to 
find some way to migrate existing Shoola apps to a new data 
architecture. However, if we can eliminate literally 1000s of lines of 
plumbing code while improving data transfer performance by a factor of 
10, we'll be able to achieve qualitatively different levels of output 
in the OME user agents. In the spirit of pushing this process along a 
bit, I'd like to invite discussion, commentar, criticism, and hopefully 
enthusiasm for taking the next step.

I've used this architecture to build a basic demonstration project that 
grabs projects and datasets out of the database and displays them in a 
prototype navigation gui. It's pretty simple on the back-end, but it 
demonstrates the possibility.  It's not at all documented,  but i'd be 
glad to send the code out to interested follks - just let me know.


-harry