[ome-devel] Request for feature development for FIGURE - Responseto Will's reply

Tue May 22 22:25:48 BST 2018

Hi Will to answer some of your questions:

  1.  The stand-alone labels would be very helpful. As it stands if we want to e.g. put a graph in with microscopy data we've found we have to do it in Illustrator or similar. When this is re-sized in figure we've had problems with non-scalable vector graphics making the text become pixellated. I think stand-alone labels would help with this a lot.
  2.  Any sort of folder system would be very helpful. I realise most users would generate very few figures so the search option works, but we are generating hundreds, a lot of which have similar names. If we could have them at the project etc. level as in Omero it would be a big help, especially if we could move figures between different folders (which would be nice) and if we could have a shared folder that all group members could save to that would be perfect. I realise the second option might be difficult with permissions etc.

________________________________
From: Ilan Davis <ilandavis at me.com>
Sent: 22 May 2018 22:06:55
To: OME External Developer List; William Moore (Staff); Jason Swedlow
Cc: Richard Parton; mkaythomp_gmail_com; Darragh Ennis; David Susano Pinto
Subject: Re: [ome-devel] Request for feature development for FIGURE - Responseto Will's reply

Many thanks Will for focusing this discussion to be more specific. I think there are others in my team who can better answer a few points, I will put their names in red in the hope that they will see this and contribute.  Apologies if I get wrong end of the stick on any of these points.

1) Make some small improvements to Figure so it is sufficient for making figures for papers (other than drawing diagrams and plots). e.g. scalable text and better text labels. Does not have to be extensive and complex. Just one labelling approach that is good.

We have an existing feature request for “Stand-alone labels” (labels that can be placed anywhere on the page) https://trello.com/c/wtu7IVuN/10-stand-alone-labels.
Does this cover your needs or are there other improvements to the existing panel labels that you’d like to see?
You can set font-sizes on labels already. Does “scalable text” include some other functionality or more flexible sizes?

Josh and Darragh are better placed to answer this. I think the stand-alone labels is one issue and text that can be put anywhere inside a panel is another. I think I may have been wrong about the scalability issue - I think it is scalable currently.

2) Create a hierarchy of folders housing figures so that figures can be organised in a sensible way over time, at scale.

The folders feature request is listed at https://trello.com/c/LwAVYUjm/165-save-figures-in-folders with some discussion as to options and timescale
(which is currently limited by the OMERO model for Folders - so this needs to change first in OMERO).

If I understand you correctly, the file hierarchy would have to be limited in the same way the OMERO file hierarchy is limited (not like a file system). That would be fine, as Figures correspond to groups of data on OMERO (sessions on the microscopes or data collections or whatever).

3) Use some kind of tagging system, similar to Auto-tag (or repurpose the Auto-tag code) - extracting tags from the folder and figure names.

A tagging system would consist of 2 main features.
- ‘Add Tags’ dialog/panel - similar to that in webclient, and/or extend Auto-Tag (if users have lots of tokens in their figure names).
- Browse Tag hierarchy from within the figure app.
This could be a fair bit of work to do this properly (e.g. webclient Tag dialog is quite complex, so is webtagging). If we had “folders for figures”, would you still want Tagging for figures? Do you need both?
NB: currently webclient (or Tag-searcher) will not display Tagged File-Annotations (figures).
Card created at https://trello.com/c/X2lvut2c/192-add-tags-to-figures-browse-by-tag

This is getting a bit technical (I do not know how these things are built). But is it possible for the autotage programme to deal with each figure like an OMERO image and associate tags with it in the same way?

For our current way of operating we would want to export the list of figure names with list of associated tags in csv format. Name of tag on the top of the column and 1 or 0 indicating wether the tag applies to the figure image. Josh, MK?

4) Allow batch export of figures from pull down menus as well as CLI. At the moment we are pulling the figures from json files using our own bespoke python scripts.

Batch export of Figures was previously being discussed at https://github.com/ome/omero-figure/issues/151 (Josh Titlow).
However, one of the main issues has been identified by David Pinto https://github.com/ome/omero-figure/issues/289 which is the fact that the export creates a File Annotation on OMERO.

We really need to split the Figure Export functionality out of the Figure_To_Pdf.py script so that we can generate figures directly from Python without requiring the scripting service. Some of that work was started at https://github.com/ome/omero-figure/pull/240 to allow the web app to generate figures without the Scripting service. I abandoned this when it became clear that this wouldn’t work from the web, but it would still be useful in your case (and others) to allow batch exports etc.

I am not sure whether we are talking at cross purposes. At the moment you can export a figure with a pull down menu - but only one at a time. All I am asking for is to be able to pull down a menu, see a long list of figures and select lots of them to export in batch. We have hundreds and hundreds of figures, so exporting one at a time is really impractical. Surelly this is not a big programming task.  The creation of of file annotation - I do not understand - as the export works fine at the moment as it is.

5) Allow more format of figure export especially .png

PNG export should be quite straightforward to implement. Created card at https://trello.com/c/N0PQjMKJ/189-export-as-png

Thanks - PNG is very useful as it works better in ppt and keynote than pdf (I see lots of weird stuff with resolution in conferences that is due to embedding pdf in keynote - mac / PC conversion etc. etc.). It also happens to be the easy format of Zegami.

6) Ideally export metadata with the figures including the tags and automate publishing of figure collections through Zegami with metadata (via .png and csv files).

Is there particular metadata other than Tags you’d like to export? Exporting an extra csv file would require export of a zip instead of PDF/TIFF, which would complicate
the workflow a bit and we’d probably want to make it optional in that case.
It may be that the best option here is to create a custom Python script that exports all the metadata you want in a format of your choice.
This gives you the most flexibility and can be updated according to your needs without the UI and workflow changes for other users.

In addition to exporting the tags in csv format files, it would be good to export the microscope metadata - or any metadata associated with the parent images in OMERO that are used to make figures. The hard part is that I do not know how to export all the different metadata sensibly - i.e. from multiple different OMERO images to associate with one figure. Perhaps just be very expansive. Again a csv file with the first column being the same of the figure and the subsequent columns listing all tags and metadata. But admittedly, that does not sound particularly sensible- probably needs a much cleverer approach for the microscope metadata.  MK. Josh?

Thanks again for your thoughtful comments and time it took to understand all my points. Not that easy to explain them in writing.
Hopefully Josh and MK can explain some of the details of our workflow with examples much more easily.

Regards
Ilan

____________________________________________________________________________
Prof. Ilan Davis   ilan.davis at bioch.ox.ac.uk<mailto:ilan.davis at bioch.ox.ac.uk>   http://www.ilandavis.com
https://twitter.com/ilandavis
Department of Biochemistry, The University of Oxford, South Parks Road, OXFORD OX1 3QU, UK
Direct Office No: (44) (0)1865 613265  Office Fax: (44) (0)1865 613340
Lab manager: Dr Darragh Ennis darragh.ennis at bioch.ox.ac.uk<mailto:darragh.ennis at bioch.ox.ac.uk> (44) (0)1865 613271 / 613272
PA:  jolanta.parkinson at bioch.ox.ac.uk<mailto:jolanta.parkinson at bioch.ox.ac.uk>  (44) (0)1865 613218
____________________________________________________________________________

On 22 May 2018, at 15:41, William Moore (Staff) <W.Moore at dundee.ac.uk<mailto:W.Moore at dundee.ac.uk>> wrote:

Hi Illan,

 Thanks for the all the feedback.
It’s great to hear that OMERO.figure is being so useful for you and to get a clear idea of what improvements are needed.

I’ll respond to the various requests below in-line...

On 16 May 2018, at 16:30, Ilan Davis <ilan.davis at bioch.ox.ac.uk<mailto:ilan.davis at bioch.ox.ac.uk>> wrote:

Dear OME developer community

We love OMERO-figure. It has become a central way we use OMERO and we believe it is an under appreciated tool that everyone should use for all their day to day imaging. We would like to see the OMERO team embracing this and putting a small amount of effort in getting it even better. Please please please put these request high up on a priority list…

Motivation
Figure was developed to make it possible to make figures for papers directly from OMERO and bypassing illustrator and the like.
However, for us it has enabled an unintended additional use that we believe is more significant. It allows every time a user has an imaging session on a microscope to summarise the typical result of that session. It also allows a systematic record of a medium scale screen, again summarising how the user thinks the data should be viewed. Viewing raw data from a screen is rather difficult in comparison. We have been using Zegami, as a visualisation tool that can organise and slice and dice the figures at scale using rich n-dimensional meta-data of our choice including bioinformatics.

The following are specific requests framed in general terms and have mostly been posted previously over an extended period by various members of my group and Micron in Oxford.  David Pinto, Josh Titlow, MK Thompson will be at the next OMERO meeting in Dundee and can discuss these points in more detail.

Development requests

1) Make some small improvements to Figure so it is sufficient for making figures for papers (other than drawing diagrams and plots). e.g. scalable text and better text labels. Does not have to be extensive and complex. Just one labelling approach that is good.

We have an existing feature request for “Stand-alone labels” (labels that can be placed anywhere on the page) https://trello.com/c/wtu7IVuN/10-stand-alone-labels.
Does this cover your needs or are there other improvements to the existing panel labels that you’d like to see?
You can set font-sizes on labels already. Does “scalable text” include some other functionality or more flexible sizes?

2) Create a hierarchy of folders housing figures so that figures can be organised in a sensible way over time, at scale.

The folders feature request is listed at https://trello.com/c/LwAVYUjm/165-save-figures-in-folders with some discussion as to options and timescale
(which is currently limited by the OMERO model for Folders - so this needs to change first in OMERO).

3) Use some kind of tagging system, similar to Auto-tag (or repurpose the Auto-tag code) - extracting tags from the folder and figure names.

A tagging system would consist of 2 main features.
- ‘Add Tags’ dialog/panel - similar to that in webclient, and/or extend Auto-Tag (if users have lots of tokens in their figure names).
- Browse Tag hierarchy from within the figure app.
This could be a fair bit of work to do this properly (e.g. webclient Tag dialog is quite complex, so is webtagging). If we had “folders for figures”, would you still want Tagging for figures? Do you need both?
NB: currently webclient (or Tag-searcher) will not display Tagged File-Annotations (figures).
Card created at https://trello.com/c/X2lvut2c/192-add-tags-to-figures-browse-by-tag

4) Allow batch export of figures from pull down menus as well as CLI. At the moment we are pulling the figures from json files using our own bespoke python scripts.

Batch export of Figures was previously being discussed at https://github.com/ome/omero-figure/issues/151 (Josh Titlow).
However, one of the main issues has been identified by David Pinto https://github.com/ome/omero-figure/issues/289 which is the fact that the export creates a File Annotation on OMERO.

We really need to split the Figure Export functionality out of the Figure_To_Pdf.py script so that we can generate figures directly from Python without requiring the scripting service. Some of that work was started at https://github.com/ome/omero-figure/pull/240 to allow the web app to generate figures without the Scripting service. I abandoned this when it became clear that this wouldn’t work from the web, but it would still be useful in your case (and others) to allow batch exports etc.

5) Allow more format of figure export especially .png

PNG export should be quite straightforward to implement. Created card at https://trello.com/c/N0PQjMKJ/189-export-as-png

6) Ideally export metadata with the figures including the tags and automate publishing of figure collections through Zegami with metadata (via .png and csv files).

Is there particular metadata other than Tags you’d like to export? Exporting an extra csv file would require export of a zip instead of PDF/TIFF, which would complicate
the workflow a bit and we’d probably want to make it optional in that case.
It may be that the best option here is to create a custom Python script that exports all the metadata you want in a format of your choice.
This gives you the most flexibility and can be updated according to your needs without the UI and workflow changes for other users.

 Thanks again for your input.

 I look forward to more discussions at the OME meeting next week,

 Regards,

  Will.

PS. Another release of OMERO.figure just went out today: http://www.openmicroscopy.org/2018/05/22/figure-4-0-0.html

I welcome comments, clarifications, questions or counterpoints.

with best regards
Ilan
For transparency:
I am on the SAB of Zegami and on the SAB of OMERO-IDR

p.s.If you would like to know more about Zegami, please look up.  https://zegami.com/

____________________________________________________________________________
Prof. Ilan Davis   ilan.davis at bioch.ox.ac.uk<mailto:ilan.davis at bioch.ox.ac.uk>   http://www.ilandavis.com<http://www.ilandavis.com/>
https://twitter.com/ilandavis
Department of Biochemistry, The University of Oxford, South Parks Road, OXFORD OX1 3QU, UK
Direct Office No: (44) (0)1865 613265  Office Fax: (44) (0)1865 613340
Lab manager: Dr Darragh Ennis darragh.ennis at bioch.ox.ac.uk<mailto:darragh.ennis at bioch.ox.ac.uk> (44) (0)1865 613271 / 613272
PA:  jolanta.parkinson at bioch.ox.ac.uk<mailto:jolanta.parkinson at bioch.ox.ac.uk>  (44) (0)1865 613218
____________________________________________________________________________

_______________________________________________
ome-devel mailing list
ome-devel at lists.openmicroscopy.org.uk<mailto:ome-devel at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel

The University of Dundee is a registered Scottish Charity, No: SC015096 _______________________________________________
ome-devel mailing list
ome-devel at lists.openmicroscopy.org.uk<mailto:ome-devel at lists.openmicroscopy.org.uk>
http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20180522/38f182c0/attachment.html>