[ome-devel] Understanding the 'Big Picture' behind the OME API

[josh.moore at gmx.de]

Hi guys,

Nick, hopefully you're back safe and sound in the states. Various
responses inline:

Bernhard Holländer writes:
 > Hi Nick!
 > 
 > If you want to understand the server-client communication you should
 > take a look at the Ice manual. I just read chapters 2 and 3 and
 > looked into details when I needed them.

This is something that I've debated, but Bernhard is probably
right. Ice is simply too large to try to document all of it. I've
modified OmeroClients to point readers in that direction.

 > Ice provides a middleware that allows to write server client
 > applications with object oriented communication. Actually, when you
 > work with the OMERO service objects in your client you have a proxy
 > with a remote object of that service on the server that has access to
 > the filesystem, database etc...

 > In order to work with the objects that are defined on the server, your
 > client needs to know *how* they are defined. You need header files and
 > a library in order to work with them. That's where slice comes into
 > the game. You define your API, services, objects that hold data etc.
 > in the slice language and a code generator creates the header and
 > implementation files for you which are needed to instantiate them.
 > That's not the full truth, as objects methods are not implemented,
 > because slice does not allow to define method logic, it just defines
 > the interface.
 > 
 > So, the Omero server is an application that defines some services and
 > objects which should be exposed to client applications. The developers
 > took the slice language to write the definitions of these to generate
 > the library and header files to create client applications.
 > 
 > Thus, the best place to look for the methods defined in the API are
 > the slice files. But I also often have to look into the header file to
 > get the method signature right.

Bernhard, you'll have to let me know what you think about the
generated docs. Would they save you that hassle?

 > The nice thing about Ice and the slice code generator is that it
 > produces code for different target languages. All the basic types and
 > containers are nicely mapped to the standard containers of that
 > language. So when you define a method in slice that returns a list,
 > you get a python list, a c++ std::vector, a Java list etc. in the code
 > that is generated.
 > 
 > Ice itself is provides the necessary parts that are needed for the
 > communication in the different target languages and most of the time
 > you don't care, it happens behind the scenes, in particular if you
 > just work with the API on the client side.
 > 
 > I hope that helps a little bit to understand the big picture.
 > 
 > Best wishes! Bernhard
 > 
 > On Thu, Sep 10, 2009 at 5:32 PM, Nick Perry <nperry at stanford.edu>
 > wrote:

 > > Okay, so having read this page
 > > (http://trac.openmicroscopy.org.uk/omero/wiki/OmeroClients?onlycpp)
 > > about 1000 times, I'm ready to test if I finally understand
 > > what's going on here.  I'm just going to start listing
 > > points/concepts, so if anyone can just confirm/correct as they
 > > see fit, that would be extremely helpful. It could also probably
 > > be useful in helping clean up that page since it is rather
 > > confusing (to someone who didn't develop this stuff, anyway).

:) Ok. I'll try to tone it down a little.

 > > - ICE provides generic types (primitives, sequences,
 > > dictionaries, etc) that serve as the 'middle-man' for converting
 > > between those types between languages (ie, going from a list in
 > > python to a vector in C++, or something like that via the Ice
 > > 'sequence').

Right. Another way of saying it is maybe that Ice provides a
language-independent type system, or alternatively an interface
definition language. Not really unlike web services.

 > > The reason for using ICE is so that people can write client stuff
 > > in any of the 4 languages (Python, Java, C++, Matlab) and
 > > everything gets converted the same way.

And do it with a binary protocol as /opposed/ to web services.

 > > All the information about the types provided and what they
 > > convert to in each language are found in the slice definitions
 > > that can be found here:

 > > https://trac.openmicroscopy.org.uk/omero/browser/trunk/components/blitz/resources/omero

Almost. We also code-generate a whole slew of *.ice files based on the
OME model. Those go into blitz/generated.

 > > - From the types that ICE provides (primitives, sequences, etc),
 > > OME made a bunch of objects, which are the objects that get
 > > passed to and from (along with the ICE types) the server from the
 > > client. However, once these objects are on the client side (my
 > > side) I can cast them back to their native language. For example,
 > > if I have a sequence of objects (omero::api::IObjectList), I can
 > > cast it back to a vector of those objects in C++ (a vector of
 > > IObjects).

Pretty much. IObjectList is actually just a typedef for
sequence<omero::model::IObject> so you don't actually have to case the
list. But each of the contained elements does have to be cast.

 > > Question: Where is the master list of objects? Right now, I guess I can just
 > > check out Image.h to find out about the Image class, but there must be a
 > > list online somewhere?

Ultimately, the list comes from http://www.ome-xml.org, the OME
model. However, the central representation of it in OMERO (at the
moment) is:

http://trac.openmicroscopy.org.uk/omero/browser/trunk/components/model/resources/mappings

 > > Question: Is Image a subclass of IObject? Similar to how Map is a subclass
 > > of Object in Java (hopefully I'm using my terminology correctly...)

Yes. Originally OMERO began using Ice, IObject was a Java
interface. All objects like Image, implemented that interface. Ice
doesn't support Java interfaces, so IObject became an abstract class.

 > > - Every OME object (for example, Image) has a constructor with an 'I' added
 > > (ImageI(), for example) which returns a pointer (ImagePtr) to that object,
 > > all provided by the API library.

Right. If a class defined in Ice has a method:

class Ice {
  String getName();
  //...
}

then the generated class will be abstract. The components/blitz build
system generates all the concrete classes which implement those
methods. In Ice, by convention, such implementations are named with an
"I" at the end.

 > > - In addition to objects, OME also made a bunch of methods. If i
 > > were to click through each service on
 > > http://trac.openmicroscopy.org.uk/omero/wiki/OmeroApi I would see
 > > every method available to me through the API, and know which
 > > category of use it belongs to.

A better way would be to click through:

http://hudson.openmicroscopy.org.uk/job/OMERO/javadoc/slice2html/

(which should be getting a new URL soon)

 > > Question: If this is not the case, are all the methods defined here
 > > (http://hudson.openmicroscopy.org.uk/job/OMERO/javadoc/) under ome.api?

Again, before OMERO began using Ice, all the interfaces were defined
in ome.api and documented with Javadoc. Slowly, we are moving the
documentation to the *.ice files.

 > > - I can use normal C++ types in my code, and ICE handles
 > > converting it to the 'generalized' form when it's compiled that
 > > the (Blitz?) server can understand.

"Blitz" is the code-name for one of the main server components (the
one handling Hibernate and sessions) so it's fine to say the Blitz
server.

Ice handles everything needed to send the model objects over the wire,
right.

 > > The server otherwise doesn't understand just plain C++ types
 > > (especially the ones the other 3 languages don't share).

Correct. They each have a language-dependent mapping, which is
detailed in the Ice manual. The code generators ("slice2java",
"slice2py", and "slice2cpp") take care of all the heavy lifting.

 > > However, sometimes in my code, like when a method returns an
 > > object used by ICE (for example, a omero::api::IObjectList), i
 > > need to write that as what is being returned in my code. But, I
 > > can immediately cast it to vector if I wanted to. So in my code I
 > > can use both the ICE types and c++ types that ICE supports, and
 > > everything will be ok.

As I mentioned above, in the case of the sequences, they really are
just vectors via typedefs, which probably makes a bit obscure.

 > > Question: Will everything be ok..?

Of course!

 > > - rtypes just give primities in a 'wrapper' that allow them to be
 > > NULL. kind of like how Integer is a wrapper for an int in Java to
 > > provide special benefits.

Exactly. The also provide a secondary benefit: they allow us to pass
differing types through the same method (inheritance-based polymorphism).

 > > In summary:
 > >
 > > - ICE makes my basic types (in my incorrect vocabulary, my
 > > arrays, maps, ints, floats, etc) (documentation:
 > > https://trac.openmicroscopy.org.uk/omero/browser/trunk/components/blitz/resources/omero)

 > > - OME made objects that utilize the above ICE basic types (documentation: i
 > > have no idea)

 > > - OME made methods for the above OME objects and ICE basic types used to
 > > communicate to the server. (documentation: maybe at
 > > http://hudson.openmicroscopy.org.uk/job/OMERO/javadoc/)

Correct. Though as I mentioned, we'll be moving from the javadoc to
the slice2html documentation over the next while.

 > > Thanks for the help
 > > Nick

Gladly (just sorry for the long delay).
Cheers,
~J.