<div dir="ltr"><div><div><div>Hi all,<br><br></div> Thanks for sending this out, Curtis.<br><br></div> If anyone's interested, <a href="https://github.com/scifio/scifio/issues/3">I filed this as a github issue</a> under the SCIFIO repo... so you can post comments there as well.<br>
<br></div>- Mark<br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Jun 10, 2013 at 10:50 AM, Curtis Rueden <span dir="ltr"><<a href="mailto:ctrueden@wisc.edu" target="_blank">ctrueden@wisc.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Roger & everyone,<div><br></div><div>Sorry for the delay in reply. After spending the last couple of weeks on ImageJ build issues related to native code components (specifically, the ImageJ launcher in C), I have some new perspective on the new code generation of the Bio-Formats build system.</div>
<div><br></div><div>First of all, I want to say thanks to Roger for solving the build for both Ant and Maven. I know maintaining the dual build systems can be substantial extra work. But I think the Maven system has many advantages, so I am happy it is being maintained.</div>
<div><br></div><div>That said, I think Bio-Formats would greatly benefit from substantial modularization of components. We are realizing this with SCIFIO, and I think it applies to the OME-XML component as well.</div>
<div><br></div><div>Below, I will lay out what I think is a better structure for the build system, which would result in more advantages and less pain than with the current structure.</div>
<div><br></div><div><div>> One thing which might be an issue is that while xsd-fu generates the</div><div>> ome-xml model code, which could potentially be downloaded, it also</div><div>> generates all the MetadataStore, MetadateRetrieve and all the other</div>
<div>> Metadata-related classes in scifio, including OMEXMLMetadataImpl.</div><div>> Given that these are paired with the generated model code, generating</div><div>> one and downloading the other may result in breakage on model changes,</div>
<div>> or changes in xsd-fu or the templates which change the generated code.</div><div><br></div><div>MetadataStore, MetadataRetrieve, etc., would move to the ome-xml component, keeping all generated code together.</div>
<div><br></div><div>One Git repository for each of:</div><div><br></div><div>- SCIFIO (<a href="https://github.com/scifio/scifio" target="_blank">https://github.com/scifio/scifio</a>)</div><div>- OME-XML (<a href="https://github.com/openmicroscopy/ome-xml" target="_blank">https://github.com/openmicroscopy/ome-xml</a>)</div>
</div><div>- Bio-Formats (<a href="https://github.com/openmicroscopy/bioformats" target="_blank">https://github.com/openmicroscopy/bioformats</a>)</div><div>- Fork: Apache POI (<a href="https://github.com/openmicroscopy/ome-poi" target="_blank">https://github.com/openmicroscopy/ome-poi</a>)</div>
<div> -- change package prefix to avoid third party code collisions</div><div>- Fork: MDB Tools Java (<a href="https://github.com/openmicroscopy/ome-mdb-tools" target="_blank">https://github.com/openmicroscopy/ome-mdb-tools</a>)</div>
<div> -- change package prefix to avoid third party code collisions</div><div>- Fork: JAI Image I/O (<a href="https://github.com/scifio/scifio-jai-image-io" target="_blank">https://github.com/scifio/scifio-jai-image-io</a>)<br>
</div>
<div> -- change package prefix to avoid third party code collisions</div><div>- Stub: LWF (<a href="https://github.com/scifio/lwf-stubs" target="_blank">https://github.com/scifio/lwf-stubs</a>)</div><div><br></div><div>
In other words, OME-XML gets its own Git repository, which includes all the code generated code. Each fork and stub also has its own repository in the relevant namespace.</div><div><br></div><div>Dependencies between repositories would be done by release version coupling. For Maven projects (i.e., SCIFIO), simply making releases and using release dependencies would be sufficient to facilitate repeatable builds. For Ant-based projects (i.e., stuff in openmicroscopy namespace), release JARs would continue to be committed to the repository as they are now, or they could be resolved remotely via Ivy or similar.</div>
<div><br></div><div>This would making building Bio-Formats much simpler and faster. As Roger pointed out, we do not really need to code generate the OME-XML stuff on every build, but rather only when the schema changes. Of course, the OME-XML component contains other code which would be subject to change between schema releases, but that's fine.</div>
<div><br></div><div>This more modular structure would also facilitate these components being developed on separate release cycles. The forks and stubs rarely change and do not need to be released with every OME release. And the OME-XML project could be released along side schema changes (i.e., twice a year) rather than with every OME release.</div>
<div><br></div><div>Comments welcome.</div><div><br></div><div>Regards,</div><div>Curtis</div>
<div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, May 2, 2013 at 12:25 PM, Roger Leigh <span dir="ltr"><<a href="mailto:r.leigh@dundee.ac.uk" target="_blank">r.leigh@dundee.ac.uk</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>On 02/05/2013 16:52, Curtis Rueden wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
> If so, the build is completely identical--the sources which get<br>
> generated on the fly from the templates by xsd-fu are identical bar a<br>
> few lines comments in the top boilerplate.<br>
<br>
OK, good to know.<br>
<br>
One more question/concern: presumably, the Bio-Formats build no longer<br>
functions on Windows, due to the Python + Genshi dependency. With the<br>
Ant build, this might be non-trivial to solve. But solving the issue<br>
with Maven is very straightforward: include the "ome-xml" module in the<br>
reactor only within a profile. Then, when that profile is not enabled,<br>
Maven will resolve the ome-xml dependency from the remote repository<br>
rather than regenerating and rebuilding the code. This would eliminate<br>
the need to install Genshi, and make it easier to build on Windows<br>
again. What do you think?<br>
</blockquote>
<br></div>
I'm afraid I'm no authority on Maven, so I'm not sure. Maybe Melissa or<br>
Josh have a better take on this than me. I assume that this will work<br>
correctly on Windows if python is installed?<br>
<br>
One thing which might be an issue is that while xsd-fu generates the<br>
ome-xml model code, which could potentially be downloaded, it also<br>
generates all the MetadataStore, MetadateRetrieve and all the other<br>
Metadata-related classes in scifio, including OMEXMLMetadataImpl. Given<br>
that these are paired with the generated model code, generating one and<br>
downloading the other may result in breakage on model changes,<br>
or changes in xsd-fu or the templates which change the generated code.<br>
<br>
While it's not all enabled yet, I'd like to have the model selectable as<br>
an ant properly (it's xsdfu.schemaver), so that it's possible to change<br>
to a different model when building. There's currently some hardcoded<br>
"2012-06" versions which need to be switched to change to use the<br>
property value.<br>
<br>
<br>
Regards,<br>
Roger<br>
<br>
--<br>
Dr Roger Leigh -- Open Microscopy Environment<br>
Wellcome Trust Centre for Gene Regulation and Expression,<br>
College of Life Sciences, University of Dundee, Dow Street,<br>
Dundee DD1 5EH Scotland UK Tel: (01382) 386364<div><div><br>
<br>
The University of Dundee is a registered Scottish Charity, No: SC015096<br>
<br>
______________________________<u></u>_________________<br>
ome-devel mailing list<br>
<a href="mailto:ome-devel@lists.openmicroscopy.org.uk" target="_blank">ome-devel@lists.<u></u>openmicroscopy.org.uk</a><br>
<a href="http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel" target="_blank">http://lists.openmicroscopy.<u></u>org.uk/mailman/listinfo/ome-<u></u>devel</a><br>
</div></div></blockquote></div><br></div></div>
<br>_______________________________________________<br>
ome-devel mailing list<br>
<a href="mailto:ome-devel@lists.openmicroscopy.org.uk">ome-devel@lists.openmicroscopy.org.uk</a><br>
<a href="http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel" target="_blank">http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel</a><br>
<br></blockquote></div><br></div>