[ome-users] Importing large image series and Cell^R data

[Colin Rickman]

On 21 May 2013, at 16:02, Roger Leigh <r.leigh at dundee.ac.uk> wrote:

> On 21/05/2013 10:37, Colin Rickman wrote:
> 
>> On 15 May 2013, at 11:13, Roger Leigh <r.leigh at DUNDEE.AC.UK
>> <mailto:r.leigh at DUNDEE.AC.UK>> wrote:
> 
>>> Can you reproduce the failure with a smaller series?
>> 
>> I've tested this and I can with what seems to be any number of
>> individual files. In all cases the importer states "imported 0 of 1" and
>> sits scanning indefinitely. However, I did try entering the directory in
>> the import window and selecting all of the individual tiff files (a
>> 10,000 file subset) and after a long wait to start the importer stated
>> "imported 0 of 10,000" and slowly started working through the files - it
>> is currently just over half way through.
> 
> This sounds like progress, albeit slow.  If you have any sample plain
> TIFF files like this you'd like us to look at WRT import speed, then you
> can also upload these as a zip as for the other data, below.

I'l upload a small subset shortly. The upload of 10,000 files failed after just over 5000 files. All remaining files failed to upload. I hit submit failures (unticking file and selecting log) so you can see what happened - it looks like it may have generated a single log for every file rather than one log file for all files. 


>> I've uploaded a test data set to the link above. I also tested uploading
>> the same dataset to our OMERO server. The dataset consists of *.apl,
>> *.mtb and *.tnb files which contain all of the metadata etc. In a
>> separate subdirectory are the multidimensional tiff files. Importing the
>> whole dataset sort of worked. In the example I sent you the resulting
>> upload contained 16 image stacks (the correct number). However,  one of
>> the .tif files was now given the *.apl file name and the metadata was
>> not imported fully. All of the remaining .tif files had their cryptic
>> name used by the database rather than their human understandable name
>> given in the metadata. There were a couple of errors reported trying to
>> access this dataset on the OMERO server and these have been reported
>> under my email address.
>> 
>> Hope that helps. Let me know what else you need to know.
> 
> Thanks for uploading the data, I've tested it with the current dev_4_4
> development code, and I think I have reproduced what you've described.
> I've opened ticket #10942
> (http://trac.openmicroscopy.org.uk/ome/ticket/10942) for this issue.

Thanks. Hopefully this can be rectified in the next release. If you require any other sample datasets for testing just let me know.

Colin


Dr Colin Rickman
Institute of Biological Chemistry, Biophysics and Bioengineering
School of Engineering and Physical Sciences
Heriot-Watt University
Edinburgh
EH14 4AS

Tel: +44 131 4514193 (Office)

http://www.ib3.eps.hw.ac.uk
http://www.esric.org






----- 
Sunday Times Scottish University of the Year 2011-2013
Top in the UK for student experience
Fourth university in the UK and top in Scotland (National Student Survey 2012)

We invite research leaders and ambitious early career researchers to 
join us in leading and driving research in key inter-disciplinary themes. 
Please see www.hw.ac.uk/researchleaders for further information and how
to apply.

Heriot-Watt University is a Scottish charity
registered under charity number SC000278.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20130522/7b76e8f8/attachment.html>