[ome-devel] Bio-Formats build system and repository structure [was: Re: Error building Bio-Formats develop]

[Mark Hiner]

>   - a pull request into whichever repository houses the specification
>     (currently bioformats.git)
>   - creation of "release" artifacts from whichever repository houses the
>     specification
>   - a pull request into ome-xml.git (to update the autogenerated code)
>   - creation of release JARs from ome-xml.git
>   - a pull request into scifio.git (to update SCIFIO readers)
>   - creation of release JARs from scifio.git
>   - a pull request into bioformats.git (to update Bio-Formats readers)
>

Nitpicky: there won't be a dependency on OME-XML in SCIFIO, so for this
example that step can be omitted.

Also, the creation of artifacts would all be automated, right...?

Conceivably, version number bumping could be automated as well... I believe
Johannes made a script to do that for pom-scijava.
Having a parent equivalent
pom-scijava<https://github.com/scijava/scijava-common/tree/master/pom-scijava>for
all these repositories would simplify things as well, as you would
only
need to update one direct dependency, and the versions for the rest of the
components would be derived. So it's really not that much bookkeeping.

Modifying the top layer of the stack does require several updates, but -
and someone please correct me if I'm wrong here - conceivably you only need
to make one PR like this per project per release cycle.. and it would just
be part of the release process, when you're doing a commit to bump version
numbers anyway. Also I suspect there are additional git/maven options to
simplify things here.

Anyway, I personally am already working with a 3-deep dependency chain with
SCIFIO to IJ2:
SCIFIO <-> imglib -> ImageJ2

But locally I just set my dependencies to the SNAPSHOT version (which is
easy to do if you have that parent pom, because you just need to override
<http://stackoverflow.com/questions/13876165/how-to-override-maven-property-in-command-line>its
properties<https://github.com/scijava/scijava-common/blob/master/pom-scijava/pom.xml>)
and develop normally. For ant builds, we could have Maven generate the
dependency jars automatically. I'm going to have to put some more work into
artifact generation anyway to get a loci_tools.jar with proper sez-poz
annotations.. so it's an area I'll be working on.

Our policy so far has been to release everything at once
>

 I don't see why you couldn't release all at once still.. the action of
releasing multiple components may not be atomic, but there's no reason we
can't say "I'm releasing Bio-Formats, SCIFIO and OME-XML on July 11th"...
right?

It also avoids unnecessary versioning.. like "releasing" a new fork even
though nothing has changed. I guess I don't know here.. looking at the git
log.. do version numbers change for every component on every release, even
if nothing changed within that component? My assumption was that they do..
but maybe I'm wrong.

Note that we've previously (and relatively recently) put quite a bit of
> effort into getting the OME-XML code and specification into bioformats.git.
>

This is a fundamental SCIFIO thing... the proposed structure of SCIFIO has
always been:

SCIFIO.jar (open format readers/writers)
    |
OME-XML.jar (OME-XML classes, OME-XML metadata class, translators to/from
core SCIFIO formats & OME-XML)
    |
Bio-Formats.jar (proprietary formats, translators to/from proprietary
formats & OME-XML)

I admit I have not been keeping up as well as I'd like with Bio-Formats
development, because I've been splitting my time between SCIFIO and IJ2
this past month. But, given this hierarchy, I think the OME-XML project
clearly has value on its own, and it makes sense to be able to separate it
from Bio-Formats under the SCIFIO design (even though it's an integral part
of the openmicroscopy/bio-formats.git right now).

Anyway, I agree this would be a non-trivial change, but I think it creates
a cleaner and more modular software stack, and could add more meaning to
version numbers (and, if versions *are *always updated regardless of work,
this would eliminate unnecessary commits).

I will almost certainly be doing this split in scifio.git regardless, so
maybe that will be helpful to test the waters and see what the workflow
actually is like..?

- Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20130611/6cdc14ea/attachment-0001.html>