[ome-users] Giving Omero 5.0.0-beta1 a good kicking. Some questions.

[Josh Moore]

On Jul 30, 2013, at 9:51 AM, Jason Swedlow wrote:

> Hi Jake
> 
> These are great questions. Forwarding to our ome-users list so everyone can benefit from the discussion.
> Cheers,
> Jason

Jake et al,

> Jake Carroll <jake.carroll at uq.edu.au> wrote:
> 
> Hey OME guys.
> 
> Hope you're well.

Think most if not all of us are. Thanks.


> Dan's dealing with some other formats that aren't happily importing, also.


Which ones? Perhaps someone else has seen the same issues already.


> A couple of things have come up - as we intend to get this thing into
> production one of these days soon. I have questions:
> 
> 1. How far out do you see a "Stable" build of Omero5, with FS inclusive
> being pushed out to the public?

There will be 2 different paths of stabilization: the API and the production-readiness. The beta in the version number will remain until the API is stabilized, at which points developers should be able to assume that code which worked with 5.0.n will also work with 5.0.n+1. In other words, version numbers define API compatibility and the beta means there will likely still be changes.

In terms of production-readiness, we certainly intended users to be able to start adopting beta1. Though there's no upgrade from 4.4 versions yet, we will provide upgrades between each of the 5.0 versions, whether "beta" is present in the version number or not. One of the next releases, hopefully shortly after the summer, will permit upgrading from 4.4 as well, at which time we will very likely encourage general adoption, with the caveat that each 5 release will be (hopefully slightly) less stable than the latest 4 release, which is perhaps obvious.


> 2. I've read all the "Scaling Omero" threads - and it doesn't really
> resolve my concerns. I need to present Omero to a whole institute of
> people who are pathological nightmares in terms of imaging. I am picturing
> 5 people all sitting there from acquisition hosts uploading 20GB+ IMS or
> ZVI images, all at once. Now, on a 1GbE pipe from each computer, that
> probably represents a few hundred megabits per second of IO,
> realistically. Can the OME/bioformats importer, Ice, Xmx + Java actually
> cope with that kind of concurrent IO in your experience?

I'll leave someone else to comment on how close they've come to this level of throughput, but you're certainly right that at some point there will be a level which a single Java server cannot handle. There are ways to try to parallelize the upload and perhaps even lower some of the Java overhead, which it would be good to work through with you. More importantly, though, may be that in the worst case, we should be able to provide a fallback. During the Paris Users' meeting, there were several requests for "in-place imports" where transferring the data at all is simply not possible. This is certainly something we intend to provide, but there will be a list of caveats which will have to be accepted before setting up such a workflow.


> 3. I want to make it a single sign on environment. Have you seen OME work
> successfully when tied to an organisation-wide LDAP or ActiveDirectory
> before?

Use of LDAP is quite wide-spread, but not with SSO. Do you have a provider in mind?


> 4. Backing up the OME environment. How do you do it? I like and am happy
> with the data structures that OmeroFS presents now, which is consistent
> and maintainable, but what of the "front end". If I blew up a front end
> host that "hosted" the OME database tomorrow, how might I rebuild it such
> that the data underneath can be "pointed at" to regenerate where we were?

Here there should be no real change in what's required beyond 4.4, i.e. you should (in fact MUST) backup both your data directory (or possibly directories with FS) AND your PostgreSQL database. The one difference is that if for whatever reason you then lose your database dump, the files will be far more usable under FS than before FS.


> Plenty more questions to follow..

Excellent!


> --j

Cheers,
~Josh