[ome-devel] batch import into OME 5.0.2

[Josh Moore]

Hi Yanling,

my apologies for not getting to this sooner. (There are people to hit me with a stick when I forget to reply to a mailing list email, but not when it goes only to me)

On Jun 12, 2014, at 6:17 PM, Yanling Liu wrote:

> Josh,
> 
> bin/omero fs repos
> ...
> the repo did *not* recently move locations.

Yup, looks very healthy.


> I was trying to import a dataset including 161 folders and 8514 images. I
> have created a python script to iterate these 161 folders and import each
> folder as one dataset in OMERO.
> 
> As I indicated in previous message, I would get "file not existing error"
> during the import process. Also after the import process, I am having
> problem to do anything to OMERO including new imports and deletions. If I
> start new import I will get "Too many open files" error message. And OMERO
> will not be able to accept more TCP connections.

Thanks for these. I'll take a look first thing tomorrow. If configuring
ulimit to get rid of the "Too many open files" fixes the problem, please
let me know.

Cheers,
~Josh.



> I have attached complete import log message, along with my python script.
> 
> My python script is simply a wrapper to iteratively call omero import
> command.
> 
> Thanks,
> Yanling
> 
> 
> On Tue, Jun 10, 2014 at 11:51 AM, Josh Moore <josh at glencoesoftware.com>
> wrote:
> 
>> Hi Yanling,
>> 
>> On Jun 10, 2014, at 5:44 PM, Yanling Liu wrote:
>> 
>>> Server configuration: Ubuntu 12.04 VM with 3TB NFS storage attached. OME5
>>> is installed and the ManagedRepository is located on the 3TB NFS storage.
>>> 
>>> When using command line import tool to inject one user folder plus its
>>> subfolders to one dataset in OME, I have got following error on certain
>>> TIFF images (file path removed from log info)  :
>> 
>> Would you mind sending the full log (via private email if you'd prefer)?
>> Also having the command-line as you executed it would be useful.
>> 
>> 
>>> I have verified the file 51-11.tif exists as
>>> ManagedRepository/acharyaj_52/2014-06/06/16-43-40.589/51-11.tif. Tried
>> run
>>> tiffinfo tool again the image file and found no problem. Don't know why
>> OME
>>> report an existing file as non-exist...
>> 
>> Could you send the output of:
>> 
>>  bin/omero fs repos
>> 
>> and:
>> 
>>  bin/omero config get | grep omero.data.dir
>> 
>> Did the repo recently move locations?
>> 
>> 
>>> Thanks,
>>> Yanling
>> 
>> Cheers,
>> ~Josh
> <output.txt><importSingleUserImages_s.py>