[ome-devel] Modularising the bio-formats build using git submodules

[Roger Leigh]

I've been thinking a little bit about how we might be able to better
modularise things so that individual components could be built
separately, independently released and used by others without having to
build all of bio-formats, but also still allowing the existing setup to
be used.  I've recently been looking at Qt, and their repo at
http://qt.gitorious.org/ (git://gitorious.org/qt/qt5.git) contains 34
submodules referenced by this tiny top-level repo; run the
"init-repository" script to set it all up.  Qt is huge, with each
component being an entire project in its own right.  You can clone and
build each separately, or as a whole by cloning the toplevel container repo.

I've made a test repository using this one component per submodule
scheme, which is here: https://github.com/rleigh-dundee/bioformats5
(https://github.com/rleigh-dundee/bioformats5.git); you can see all the
dependent submodules here:
https://github.com/rleigh-dundee?tab=repositories (they have a
"bioformats5-" prefix).

After cloning, you just need to do a "git submodule init; git submodule
update" to set it all up.  There are 23 components in total.  We could
also split out the junk like the binary jars into a submodule as well.

This was done in a very simplistic way, one submodule per component with
a pom.xml.  It may not be the ideal arrangement, it's just a proof of
concept.  Each submodule was generated from current develop with
"git filter-branch --tag-name-filter cat --prune-empty
--subdirectory-filter components/test-suite HEAD".  The top-level is
just the current develop with the components deleted and replaced by the
corresponding submodule.  But the top-level could be replaced with a new
repo which would drop all the history duplicated in the submodules.  Or
we could have multiple toplevels with different selections of
submodules, which would allow old, unused components to be dropped.

This scheme could also allow openmicroscopy.git to include the set of
specific bioformats components it needs, rather than the bioformats
submodule used currently.  And it could itself be split up into
submodules per component in the same manner, though this would require
some preparatory refactoring.


Regards,
Roger

--
Dr Roger Leigh -- Open Microscopy Environment
Wellcome Trust Centre for Gene Regulation and Expression,
College of Life Sciences, University of Dundee, Dow Street,
Dundee DD1 5EH Scotland UK   Tel: (01382) 386364

The University of Dundee is a registered Scottish Charity, No: SC015096