[ome-devel] OME Files: new source code repositories and recent development work

[Roger Leigh]

Dear all,

With reference to the reduction in support for OME Files within OME for 
code review, maintenance and continuous integration 
(http://lists.openmicroscopy.org.uk/pipermail/ome-devel/2018-August/004294.html), 
I would like to announce the creation of a new set of code repositories 
and replacement continuous integration infrastructure.  I hope that 
these repositories can serve as a place for users of OME Files to obtain 
a supported and maintained set of libraries and tools for contemporary 
platforms, as well as being a place for collaborative maintenance, 
perhaps until such time OME is in a position to pick them back up.

Repositories
------------

https://gitlab.com/codelibre/ome-files is a super-project using git 
sub-modules.  After cloning, run
   "git submodule update --init --remote"
to check out all of the components.  This will allow the entire 
collection of libraries and tools to be built and used with ease, 
including development within an IDE, should you desire.

The individual repositories used within the super-project:

- https://gitlab.com/codelibre/ome-common-cpp
- https://gitlab.com/codelibre/ome-model
- https://gitlab.com/codelibre/ome-files-cpp
- https://gitlab.com/codelibre/ome-files-view [was ome-qtwidgets]
- https://gitlab.com/codelibre/ome-files-py
- https://gitlab.com/codelibre/ome-files-performance

may also be cloned and built individually.

The CMake super-build 
(https://gitlab.com/codelibre/ome-cmake-superbuild) is deprecated, to be 
replaced by https://github.com/Microsoft/vcpkg/ and the above 
super-project.  It is waiting on the merge of 
https://github.com/Microsoft/vcpkg/pull/4575 before it can be fully 
retired.  This should happen within the next few days.  VCpkg is a 
homebrew-style package manager for Windows which allows all dependencies 
to be built simply and straightforwardly.  It does everything the 
super-build did to fetch and build third-party dependencies, but the 
maintenance burden is greatly reduced by sharing it with others.

Continuous integration
----------------------

Each repository uses GitLab CI to test the code on a wide variety of 
platforms.  Examples include:

- https://gitlab.com/codelibre/ome-model/pipelines/37267293
- https://gitlab.com/codelibre/ome-files-cpp/pipelines/36771859

This has finer-grained coverage across a wider range of platforms when 
compared with the previous Jenkins CI 
(https://ci.openmicroscopy.org/view/Files/).  The exception is the lack 
of specific support for MacOS, which is to be added at a later date 
[pending GitLab support for MacOS X CI or addition of a dedicated 
runner].  These jobs also build and deploy the documentation.

Issues
------

Each code repository has GitHub issues enabled.  Bugs, feature requests, 
support requests etc. may be opened and contributed to by anyone.

Code contribution
-----------------

Merge requests may be opened by anyone against any of the code 
repositories.  The process is completely open, and all contributions are 
welcome.

I am also available to work on any aspect of OME Files on a contract 
basis.  Please get in touch privately if you would like to discuss any 
feature development or maintenance work you require.

Changes since the last OME Files release
----------------------------------------

The key changes are:

- Full support for reading OME-TIFF with sub-resolution pyramids
- Full support for writing OME-TIFF with sub-resolution pyramids
- Full support for Python 3 in ome-model (and all other components)
- Support for Visual Studio 2017 in all components
- Initial support for C++17 in all components (currently tested with 
LLVM6 and VS2017 with GitLab; GCC 8.2/Ubuntu 18.10 tested locally)
- Comprehensive support for GitLab CI in all components, to replace the 
existing ad hoc mixture of Jenkins, Travis and AppVeyor
- Initial C++17 filesystem and variant support in OME Compat
- Units, XML and XSLT components split out of OME Common into separate 
libraries, to allow future replacement of Xerces and Xalan dependencies; 
they are likely to be split into separate repositories in the future
- OME Model is built with Gradle and is available under the 
net.codelibre coordinate on Maven Central 
(https://search.maven.org/artifact/net.codelibre/ome-xml/5.7.1/jar); it 
remains completely compatible with the org.openmicroscopy artifacts
- Some Boost dependencies are being dropped, and some made optional, to 
ease deployment
- Model and Files components have been kept up-to-date with all relevant 
upstream changes

Releases
--------

I will make releases of all the components, containing all of the work 
reviewed, tested and merged to date, during the next few weeks. 
Following on from this, releases will be made as and when required for 
new features, bug-fixes and platform support.

Releases will be announced on these lists as and when they are made.

Funding
-------

I would like to thank Damir Sudar and Michel Nederlof of Quantitative 
Imaging Systems, LLC for funding the completion of the OME-TIFF 
sub-resolution pyramid feature.



Kind regards,
Roger