[ome-devel] Remote Framework: A review

[Douglas Creager]

> Sorry there's no FM to read.  The Remote framework has gone through 
> some changes and has really only stabilized with the current release.  
> But ask, and yee shall be guided.

I'm actually working on this as we speak.  Should have a rough draft 
ready in a day or two.

A couple of points about XMLRPC (excerpted from the upcoming FM):

<snip>

The Remote Framework consists of three major parts:

• A logical model in which remote function calls can be  made. This 
model specifies the primitive and  compound data types that remote 
framework methods can  operate on.

• An extensible set of methods which provide the access to the  
underlying metadata on the data server.

• A network transfer protocol. This specifies how values in  the 
logical type model are turned into binary strings sent  across the 
network wire. It also specifies how function  requests and results are 
transferred across the network,  and how connections to the data server 
are made.

The logical model and remote methods are defined in such a way  that 
they are completely independent of the network transfer  protocol. We 
currently use XML-RPC as the transfer protocol,  but this is not an 
explicit requirement of the remote  framework. The existing server and 
client components have  both been written in such a way that a 
different transfer  protocol could be used as a drop-in replacement. 
Any transfer  protocol is acceptable, as long as it can represent all 
values  defined by the logical type model.

</snip>


It's important to point out that when I first started designing the 
remote framework, I was not aiming for a full-functionality web 
services solution.  We're just using it to transfer data.  When I was 
working through the prototype versions of the framework, we used SOAP 
as a transfer protocol.  However, we quickly ruled out SOAP, since it's 
basically XML-RPC with a lot of crap grafted on which we didn't need.  
(Plus, the particular Perl and Java SOAP implementations which we were 
using didn't like to talk to each other, which we took as a bad sign.)  
So, we switched over to XML-RPC.

Why XML-RPC?  Easy -- existing libraries meant that I didn't have to 
code up any transfer protocol libraries in Perl and in Java.  However, 
XML-RPC obviously has its drawbacks -- most notably all of the extra 
overhead for the XML tags around the actual payload.  This can quickly 
add up if you're receiving a list of 1,000-2,000 images plus metadata 
in a given call.

I had coded up a basic, bare-bones transfer protocol with about 90% 
less overhead for describing the data, and a JLex/JavaCup parser on the 
client side to speed up parsing.  It's not anywhere near ready for 
prime-time, but it certainly proved that the drop-in-replacement 
functionality works on both the Perl server side and Java client side.  
So, ideally, any C remote framework client would also allow the 
transfer protocol to be switched easily, allowing this lovely 
Shakespearean "XML-RPC or not XML-RPC?" question to be deferred for 
now.

Anyway, I'm going to get back to this FM, so that it can be ready for 
y'all to peruse fairly soon.


--doug

------------------------------------------
Douglas Creager <dcreager at alum.mit.edu>
Visiting Scholar, MIT Dept. of Biology
77 Mass Ave, Rm 68-371, Cambridge MA 02139
M: +1 617/501-8340
W: +1 617/452-2955