[ome-devel] CLI render - and a philosophic question re: rendering settings

[Josh Moore]

Hi Damir,

On Fri, Aug 11, 2017 at 6:15 PM, Damir Sudar <dsudar at lbl.gov> wrote:
> Hi Josh,
>
> On 8/11/2017 8:32, Josh Moore wrote:
...
>>> Now the philosophical question: as in IDR, we have a very large image
>>> dataset that we are quantitatively analyzing and also want to show
>>> visually.
>>> This is part of a large project to asses the effects of a set of
>>> perturbing
>>> agents (perturbagens) on a panel cell lines, where in our case a large
>>> panel
>>> of different microenvironment components and drugs are used as the
>>> perturbagens. During import into OMERO each image gets its rendering
>>> settings from the min/max values in the image itself so all images have
>>> different settings and the unsuspecting viewer may see all kinds of
>>> interesting intensity effects that are mostly fake while real effects may
>>> be
>>> masked. So I'm looking for a general way to set the rendering settings
>>> across the entire experiment to something that allows comparing images
>>> but
>>> doesn't generate loads of black and over-exposed-looking images. I was
>>> thinking that you guys encountered this same issue with many of the IDR
>>> datasets so was hoping for some wise thoughts.
>>
>> We definitely do but per study, and typically we can only define that
>> after visual inspection since they are external datasets. You're
>> looking to set something *before import*?
>
>
> No, after import makes more sense since one doesn't really know what the
> expected min and max values and desired range setting are going to be, just
> as you said. We have been playing with ideas such as:
> - we could look at the min and max values per channel across a "set" of
> wells/plates (tbd what a "set" means) and set the rendering settings to be
> at 2% - 98% (or something) of those values. I'd need to write a script to
> extract such min/max values for a "set" of plates and then we could test
> what those percent values should be for a pleasing result.

Something like https://github.com/ome/scripts/pull/103 (also unmerged)


> - question to answer then is: what is a "set" and we're back to different
> rendering settings when you do want to compare images across "sets".
> - one option is to let an external query/visualization tool set the
> rendering settings. The OMERO webgateway allows you to query the min/max
> (via a JSON query) and set the rendering settings via the UR. So that would
> rely on an external visualization software do the "right" thing.
> Just to give you an idea of the data we're dealing with, check out our
> AWS-hosted public server at:
> https://omero.lincsclarion.org/webclient/userdata/?experimenter=52
> The front-end portal with query/visualization tools is still being built.

Looking forward to it!


>>> Anyway, this is why I'm so interested in the render.py utility and thus
>>> my hope it will make it to production-level soon.
>>
>> I don't have a timeline for you on the production status of render.py,
>> but what we *would* like to do for 5.4 is make it separately
>> installable, so that you can upgrade just render.py as we releases
>> fixes without needing to update all of OMERO.
>
>
> That sounds like a great approach.

We'll keep you posted.


>> P.S. from your Jul 12th email while I was on vacation:
>>
>>> Meanwhile, I had kludged together a regular OMERO script
>>> (HCS_Render_Settings.py) that does a number of the things that the render.py
>>> plugin does but much much less elegantly and very slowly. I'll use that for
>>> now. That script is on: https://github.com/dsudar/OMEROscripts
>>
>> There's no inherent reason a script should be slower that the CLI. Do
>> you have specifics?
>
>
> I think it's mostly that my script is not at all efficiently implemented
> where it does repeated calls to objects to first get metadata (such as value
> ranges) and then again to set values and probably causes multiple thumbnail
> creation calls.

Understood. ~J



> Cheers,
> - Damir
>
>>> On 7/10/2017 6:45, Josh Moore wrote:
>>>>
>>>>
>>>>> Next I tried a few of the basic commands of the render plugin:
>>>>> 1 ---
>>>>
>>>> ...
>>>>>
>>>>>     File
>>>>> "/home/omero_user/OMERO.server/lib/python/omero/plugins/render.py",
>>>>> line 117, in init_from_channel
>>>>>       self.min = channel.getWindowMin()
>>>>
>>>> ...
>>>>>
>>>>> omero.UnloadedEntityException: Object unloaded:object #0
>>>>
>>>> Vanilla 5.2 is not loading the necessary metadata for the plugin to
>>>> work. Unfortunately, solving this isn't as simple as copying a new
>>>> file into your installation. In fact, I haven't yet found a smoking
>>>> gun between the versions.
>>>>
>>>>
>>>>> 2---
>>>>> AttributeError: 'ImageI' object has no attribute '_closeRE'
>>>>> ---
>>>>
>>>> Here again, changes from 5.3 are not available in your version, though
>>>> at least these I know where they come from.
>>>>
>>>>
>>>>> Rather than trying more of this, I thought to check with you first
>>>>> whether
>>>>> I'm missing something fundamental.
>>>>
>>>> 5.3 :) (or the upcoming 5.4)
>>>>
>>>> Fundamentally, the plugin is not production quality and has only been
>>>> used in isolated situations. We certainly intend to get it into the
>>>> testing pipeline for 5.4 but that's not likely to uncover what's going
>>>> on with 5.2. If you or anyone on your side wants to continue digging,
>>>> we're happy to evaluate your changes, but there's a question of how
>>>> far you want to go in modifying your local installation.
>>>>
>>>> Cheers,
>>>> ~Josh.
>>>>
>>>>
>>>>
>>>>> Thanks,
>>>>> - Damir
>>>>>
>>>>> On 6/2/2017 2:35, Simon Li wrote:
>>>>>
>>>>> The render plugin (not tested on 5.3):
>>>>>
>>>>>
>>>>> https://github.com/openmicroscopy/openmicroscopy/blob/metadata53/components/tools/OmeroPy/src/omero/plugins/render.py
>>>>> Try dropping this into OMERO.server/lib/python/omero/plugins/
>>>>>
>>>>> Example YAML render settings file:
>>>>>
>>>>>
>>>>> https://github.com/IDR/idr-metadata/blob/master/idr0015-UNKNOWN-taraoceans/screenA/idr0015-screenA-renderdef.yml
>>>>>
>>>>> omero render edit --help
>>>>> omero render edit --copy Plate:1 renderdef.yml
>>>>> omero render copy Image:SrcID Plate:TargetID
>
>
> --
> Damir Sudar - Affiliate Scientist
> Lawrence Berkeley Natl Laboratory / MBIB
> One Cyclotron Road, MS 977, Berkeley, CA 94720, USA
> T: 510/486-5346 - F: 510/486-5586 - E: DSudar at lbl.gov
> http://biosciences.lbl.gov/profiles/damir-sudar-2/
>
> Visiting Scientist, Oregon Health & Science University
>