[ome-devel] batch image export

Fri Sep 5 20:04:35 BST 2014

Tested Insight 64bit on Windows 7.

Same with the OMERO web, when I select a project or dataset, the saving
option is disabled unless I select only images. Would it be possible to
allow project and dataset download in Insight program?

Another problem is when I click download, the activity window opens behind
the main Insight window, and there's no indication whether download has
started or not on the main Insight window, this is rather confusing. Could
you make the activity window pops up in front of the main Insight Window?
Or at least provide some visual indication on the main Insight Window to
tell user files are downloading.

Thanks,
Yanling

On Thu, Sep 4, 2014 at 6:38 AM, William Moore <will at lifesci.dundee.ac.uk>
wrote:

> Hi Yanling,
>
> Using Insight to download is probably better for large files than web.
> There's no size limit that I'm aware of.
> Is it 'not runnable on Mac' because of Java 8? If so, the recent 5.0.4
> release should fix it.
>
>
> I created a little script for deleting all 'Batch Image Export' attachments
> https://gist.github.com/will-moore/a23dc5ffa6a5394bae89
>
>
> # we can also get Files by name - but this won't have the linked file
> annotations
> files = conn.getObjects("OriginalFile", attributes={'name':
> 'Batch_Image_Export.zip'})
> for fa in files:
>     print fa.id, fa.getName()
>
>
> # For most flexibility, use the Query Service
> params = omero.sys.ParametersI()
> params.addLong('size', 4000)    # 4 kb
> params.addString('name', 'Batch_Image_Export.zip')
>
> query = "select an from Annotation as an join fetch an.file as f" \
>     " where f.name=:name and f.size>:size"
>
> # NB - since we are doing a direct query here, we get omero.model Objects
> returned...
> fileAnns = conn.getQueryService().findAllByQuery(query, params,
> conn.SERVICE_OPTS)
> for fa in fileAnns:
>     # ... and need to use .val to unwrap attributes
>     # can use the fileAnnotation Id to delete it as in the gist example
> above
>     print fa.getId().val, fa.getFile().getName().val
>
>
> # Final example of filtering by time
>
> params = omero.sys.ParametersI()
> d1 = datetime.strptime("2014-09-04 00:00:00", "%Y-%m-%d %H:%M:%S")
> created = time.mktime(d1.timetuple())*1000
> params.map["start"] = omero.rtypes.rtime(created);
>
> query = "select an from Annotation as an join fetch an.file as f" \
>     " join fetch an.details.creationEvent as ce" \
>     " where ce.time>:start"
>
> # do query as above
>
>  Hope that helps,
>
>    Will.
>
>
>
> On 3 Sep 2014, at 21:44, Yanling Liu <vrnova at gmail.com> wrote:
>
> Hi Will,
>
> Thanks for your reply.
>
> Is there a way to query OME for a list of attachment for name, date, and
> size? Could I remove unneeded attachment as the root user from command
> line? I see this may need to be run periodically to remove old attachment
> files as a system maintenance procedure.
>
> Would it be better to download image using the insight tool? Any size
> limitation in the insight tool? I can not test it as the OME client is
> currently not runnable on Mac system.
>
> Thanks,
> Yanling
>
>
>
> On Wed, Sep 3, 2014 at 4:25 PM, William Moore <will at lifesci.dundee.ac.uk>
> wrote:
>
>> Hi Yanling,
>>
>>  Yes, the Batch Image Export zip is permanently attached to the dataset
>> until someone deletes it.
>> This functionality was developed as a script (same script can be run in
>> both clients) but
>> unfortunately we don't have a nice way for the export script to pass the
>> zip back to the client
>> once it's finished. So attaching it is the only option.
>>
>> The main limitation we have with downloading a whole Project or Dataset
>> at a time is that
>> this could easily exceed a practical size of data to download.
>> Particularly in the web, where all the data for a single download needs
>> to be collated into
>> a single directory, zipped up and downloaded in one shot.
>> The size of this zip, and the time it takes to generate the zip need to
>> be compatible with http requests.
>> E.g. the web server (apache etc) will usually time-out after 1 or 2
>> minutes, and will struggle
>> to download multi-gigabyte files.
>>
>> I guess we could add the option to download a Dataset at a time, and
>> provide good indication
>> to the user of the size of the download before-hand, so they could
>> reconsider if it looks too big?
>>
>>   Regards,
>>
>>    Will.
>>
>>
>>
>> On 3 Sep 2014, at 21:17, Yanling Liu <vrnova at gmail.com> wrote:
>>
>> My mistake, what I mean "the mouse right click" is not right click, it
>> should be selecting images and then click the down arrow at the right side
>> to download images.
>>
>> It would be great to be able to select one project or data set and click
>> the down arrow at the right side to download.
>>
>> Thanks,
>> Yanling
>>
>>
>>
>> On Wed, Sep 3, 2014 at 4:08 PM, Yanling Liu <vrnova at gmail.com> wrote:
>>
>>> Dear OME developers,
>>>
>>> After running batch exporting script, the zipped images is linked as
>>> attachment to the dataset being exported. My question is how long will this
>>> attachment live in the system? If user forget to delete the zip file after
>>> downloading, will it be kept in the system forever? If so then potentially
>>> people can generate lots of zip packages and waiting storage space.
>>>
>>> The mouse right click and download feature is a much better solution.
>>> However, for now only images can take advantages of this solution, will it
>>> be extended to dataset and project so user can mouse right click on one
>>> dataset to download as original files?
>>>
>>> Thanks,
>>> Yanling
>>>
>>
>> _______________________________________________
>> ome-devel mailing list
>> ome-devel at lists.openmicroscopy.org.uk
>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20140905/e3ee0708/attachment-0001.html>