[ome-devel] Write (append) access to images after initial creation

[Josh Moore]

On Nov 6, 2014, at 5:12 PM, Christian Carsten Sachs wrote:

> Hello,

Hi Christian,

> ...
> I tried the obvious way, first creating an image normally; then loading
> the associated pixels object, changing the sizeT parameter, saving it;
> then trying to push pixels to the 'newly created' plane, but it fails
> with an ome.conditions.ApiUsageException [message = "Write-method not
> permitted."].
> 
> Checking the OMERO source shows me that the RomioPixelBuffer created has
> an internal writable-flag, which is set to false on re-creation from
> on-disk data. The appropriate location would be line 587 in
> PixelsService.java [
> https://github.com/openmicroscopy/openmicroscopy/blob/3667342755fc8a42585ee6004ee55706772b5ea1/components/romio/src/ome/io/nio/PixelsService.java#L587
> ].
> 
> Changing the last false to true solves my problem, and I can add
> additional planes - with as far as I can tell, no ill effects. [ I am
> aware that changing other dimensions (C/Z) after creation would mangle
> the file due to the on disk format ...]
> 
> Therefore I'd like to ask:
> 
> What would be the best way to integrate this feature? I guess that the
> described use case is not totally off, and others might be interested in
> a supported way of doing this as well.
> 
> I just started digging into the OMERO source, so I'm a bit wary in just
> submitting PRs with my limited knowledge:
> 
> Would it be safe for - as of the time being - use OMERO with the
> aforementioned false changed to true?

Depends on what you mean by safe. :) If you know your own access patterns and you can guarantee that you won't write from multiple threads, then it's "safe". You'll still have to deal with corrupted data on reads, but since this is during acquisition, that's probably ok.

If you want something that's properly safe, then a different subclass of AbstractBuffer, or perhaps a subclass of RomioPixelBuffer, would be needed which handles locking the file during write, and possibly uses a copy-on-write semantic. The BfPyramidPixelBuffer does something like this.

I assume that such a feature would be generally useful and could either become the default or be available based on some configuration property.


> Are there any implications why it was set to false? I assume to protect
> integrity of deposited data. Is user access control properly checked at
> this point, or would I circumvent this by changing it to false?

Access control won't be affected. It's strictly a matter of concurrent write protection.


> If there is any 'easy' way to do this, and a developer would be able to
> do it quickly, I'd love to see this implemented.

Providing a flag to go into "unsafe" mode only requires 2-3 lines of code, if that would suffice. Otherwise, assuming you accept the caveats above, harding the  flag to "true" is probably your easiest bet.


> If it is more work, and you're generally interested in this feature,
> I'll prepare a patch. I'm thinking about adding either another parameter
> to setPixelsId or another function like setPixelsIdWritable to request a
> writable PixelBuffer, if changing it on a global level [like discussed
> above] would be undesirable ...

You're likely right that there needs to be API support. If you come up with a patch, do submit it. We'll likely need to have a wider discussion about the API then, but it'll be generally to everyone's benefit.

Thanks for raising the issue!

> I'd be happy to hear your feedback.
> 
> Thanks, regards,
> Christian Sachs

Cheers,
~Josh