[ome-devel] ometiff download toubles
Chris Allan
callan at lifesci.dundee.ac.uk
Tue Mar 1 09:30:07 GMT 2011
Going back to the beginning..
def getOMETIFF(request, iid):
if conn is None or not conn.isConnected():
return HttpResponseRedirect(reverse('websimr_login'))
iid = int(iid)
image = conn.getImage(iid)
name = image.getName()
name = name.replace('\\', '/')
name = os.path.basename(name)
name = os.path.splitext(name)[0] + '.ome.tif'
exporter = conn.createExporter()
exporter.addImage(iid) <------------ this line
causes problems **********************
exporter.generateTiff()
tempdir = settings.FILE_UPLOAD_TEMP_DIR
v1 = iid
v2 = conn._sessionUuid
tempname = os.path.join(tempdir,('%i-%s.download' % (v1, v2)))
temp = tempname.replace('\\','/')
handle = open(temp, 'wb')
offset = 0
while 1==1:
rv = exporter.read(offset, 512*512)
if not rv:
break
offset += len(rv)
handle.write(rv)
handle.close()
ometiff = FileWrapper(file(temp))
response = HttpResponse(ometiff)
response['Content-Length'] = str(offset)
response['Content-Type'] = 'application/force-download'
response['Content-Disposition']='attachment;filename=' + name
return response
The reason this code does not work a second time is because of connection re-use and the fact that "createExporter()" returns you the same service each time you call it on the same connection object. You should put a try/finally around your use of "exporter" with an "exporter.close()" in the finally block. "exportOmeTiff()" is designed around some internal caching strategies for the OME-TIFFs that you create so that's not going to do what you want.
Regarding timeouts you can certainly adjust the HTTP timeouts but that's not going to help you with the fact that the browser is likely to say that the server has stopped responding after 60 seconds. There are several strategies for this that kind of go beyond the scope of OMERO specifically and apply to the general problems of performing long running tasks in a web environment. There is a reason why we have not yet implemented OME-TIFF export, especially for multiple images at a time, in OMERO.web, it's not a 5 minute job. :)
-Chris
On 25 Feb 2011, at 16:51, Wood, Christopher wrote:
> Hi Will,
>
> I looked at that. The code for exportOMETIFF in the version I am using (4.2.2) only returns the ome tiff as one large chunk, but I want to save it to a temp file in chunks. The code Josh linked to does that, so I assumed it was from a yet to be released version.
>
> The code in exportOMETIFF looks very much like what I originally tried to do (maybe it was my inspiration?) by addImage() and generateTIFF(), so I tried something different.
>
> Do you have any ideas as to why the method I first described would cause the server to stop responding?
>
> Another question:
> If the image large (100 MB), the web server or django will timeout while the tiff is generating. Does anyone know what setting(s) to adjust to account for this?
>
> Thanks
> Chris
>
> On 2/24/11 5:39 PM, "Will Moore" <will at lifesci.dundee.ac.uk> wrote:
>
> Sorry, should have replied to the lists too.....
>
> On 24 Feb 2011, at 23:38, Will Moore wrote:
>
>> Hi Chris,
>>
>> Are you developing within the OmeroWeb framework, as described
>> http://trac.openmicroscopy.org.uk/ome/wiki/OmeroWeb
>>
>> Based on the simplified example on that page, you would do
>> something like
>>
>>
>> from omeroweb.webgateway.views import getBlitzConnection
>>
>> def dataset(request, imageId):
>> conn = getBlitzConnection (request) # get a connection to
>> OMERO using values in the http request
>> img = conn.getImage(imageId) # get image by ID
>> data = img.exportOmeTiff()
>>
>> Docs for that method are here:
>> http://hudson.openmicroscopy.org.uk/job/OMERO/javadoc/epydoc/
>> omero.gateway._ImageWrapper-class.html#exportOmeTiff
>>
>> I don't have a complete example to hand just now (not on my work
>> laptop) but can find more details for you tomorrow.
>>
>>
>> Will.
>>
>>
>>
>> On 24 Feb 2011, at 18:59, Wood, Christopher wrote:
>>
>>> Josh,
>>>
>>> I used
>>> client = omero.client(host)
>>> session = client.createSession(user, pw)
>>> gateway = session.createGateway()
>>>
>>> image = gateway.getImage(id)
>>> exporter = session.createExporter()
>>>
>>> and it is working like I expect.
>>>
>>> I am getting the username and password from the request object,
>>> e.g.,
>>>
>>> username = request.session.get('username', r.get('username', None))
>>>
>>> Is it necessary to do this, or is the user, password, host, and
>>> port already set by the request?
>>>
>>> Thanks
>>> Chris
>>>
>>> On 2/24/11 9:33 AM, "Josh Moore" <josh at glencoesoftware.com> wrote:
>>>
>>> Hi Chris,
>>>
>>> the message "Cannot configure during output" comes from
>>> ExporterI.java:
>>>
>>> http://git.openmicroscopy.org/?p=ome.git;a=blob;f=components/
>>> blitz/src/ome/services/blitz/impl/ExporterI.java;#l230
>>>
>>> and implies that addImage() was called after generateTiff(), but
>>> that certainly doesn't look possible from your code snippet. This
>>> may be caused by the service wrapper from omero.gateway, but I'll
>>> let others speak to that. You might want to try either using the
>>> raw service (acquired from the session as opposed to the
>>> BlitzGateway), or use the facilities of the gateway, like the
>>> exportOmeTiff method:
>>>
>>> http://git.openmicroscopy.org/?p=ome.git;a=blob;f=components/
>>> tools/OmeroPy/src/omero/gateway/__init__.py#l5641
>>>
>>> Cheers,
>>> ~Josh.
>>>
>>>
>>> On Feb 24, 2011, at 4:14 PM, Wood, Christopher wrote:
>>>
>>>> Hi,
>>>>
>>>> I am trying to write a python script to be able to download an
>>>> OME-TIFF from omeroweb. The code below will work, but it only
>>>> works once. On the second attempt I get the error below the code.
>>>> On the third attempt, it stops working at omero needs to be
>>>> restarted.
>>>>
>>>> I do this same sort of thing for original raw files and I don't
>>>> have a problem.
>>>>
>>>> OMERO Beta4.2.2-r8718-Beta4.2-b56
>>>> python 2.4
>>>>
>>>> Here is the process:
>>>> 1) restart omero and omeroweb
>>>> 2) goto http://chi/websimr/getOMETIFF/3914/
>>>> 3) file downloads as expected
>>>> 4) try the same or different file again, I get the error below
>>>> 5) try again and omero need to be restarted. (Insight stop
>>>> responding, webclient stop responding, ....)
>>>>
>>>> Let me know if you need anything from the log files.
>>>>
>>>> Thanks,
>>>> Chris
>>>> ====================================================
>>>> def getOMETIFF(request, iid):
>>>>
>>>> if conn is None or not conn.isConnected():
>>>> return HttpResponseRedirect(reverse('websimr_login'))
>>>>
>>>> iid = int(iid)
>>>> image = conn.getImage(iid)
>>>> name = image.getName()
>>>> name = name.replace('\\', '/')
>>>> name = os.path.basename(name)
>>>> name = os.path.splitext(name)[0] + '.ome.tif'
>>>>
>>>> exporter = conn.createExporter()
>>>> exporter.addImage(iid) <------------ this line
>>>> causes problems **********************
>>>> exporter.generateTiff()
>>>>
>>>>
>>>> tempdir = settings.FILE_UPLOAD_TEMP_DIR
>>>> v1 = iid
>>>> v2 = conn._sessionUuid
>>>> tempname = os.path.join(tempdir,('%i-%s.download' % (v1, v2)))
>>>> temp = tempname.replace('\\','/')
>>>>
>>>> handle = open(temp, 'wb')
>>>> offset = 0
>>>>
>>>> while 1==1:
>>>> rv = exporter.read(offset, 512*512)
>>>> if not rv:
>>>> break
>>>> offset += len(rv)
>>>> handle.write(rv)
>>>>
>>>>
>>>> handle.close()
>>>>
>>>> ometiff = FileWrapper(file(temp))
>>>> response = HttpResponse(ometiff)
>>>>
>>>> response['Content-Length'] = str(offset)
>>>> response['Content-Type'] = 'application/force-download'
>>>>
>>>> response['Content-Disposition']='attachment;filename=' + name
>>>> return response
>>>>
>>>> ====================================================
>>>> this is the debuggingout put from the web page
>>>> ---
>>>> Environment:
>>>>
>>>> Request Method: GET
>>>> Request URL: http://chi/websimr/getOMETIFF/3914/
>>>> Django Version: 1.1.1
>>>> Python Version: 2.4.3
>>>> Installed Applications:
>>>> ['django.contrib.admin',
>>>> 'django.contrib.markup',
>>>> 'django.contrib.auth',
>>>> 'django.contrib.contenttypes',
>>>> 'django.contrib.sessions',
>>>> 'django.contrib.sites',
>>>> 'omeroweb.feedback',
>>>> 'omeroweb.webadmin',
>>>> 'omeroweb.webclient',
>>>> 'omeroweb.webgateway',
>>>> 'omeroweb.webtest',
>>>> 'omeroweb.websimr',
>>>> 'omeroweb.limsweb',
>>>> 'omeroweb.webmobile']
>>>> Installed Middleware:
>>>> ('django.middleware.common.CommonMiddleware',
>>>> 'django.contrib.sessions.middleware.SessionMiddleware',
>>>> 'django.middleware.doc.XViewMiddleware')
>>>>
>>>>
>>>> Traceback:
>>>> File "/OMERO/omero_dist/lib/python/django/core/handlers/base.py"
>>>> in get_response
>>>> 92. response = callback(request, *callback_args,
>>>> **callback_kwargs)
>>>> File "/OMERO/omero_dist/lib/python/omeroweb/websimr/views.py" in
>>>> getOMETIFF
>>>> 114. exporter.addImage(iid)
>>>> File "/OMERO/omero_dist/lib/python/omero/gateway/__init__.py" in
>>>> wrapped
>>>> 2306. return inner(*args, **kwargs)
>>>> File "/OMERO/omero_dist/lib/python/omero/gateway/__init__.py" in
>>>> inner
>>>> 2267. return f(*args, **kwargs)
>>>> File "/OMERO/omero_dist/lib/python/omero_api_Exporter_ice.py" in
>>>> addImage
>>>> 81. return _M_omero.api.Exporter._op_addImage.invoke
>>>> (self, ((id, ), _ctx))
>>>>
>>>> Exception Type: ApiUsageException at /websimr/getOMETIFF/3914/
>>>> Exception Value: exception ::omero::ApiUsageException
>>>> {
>>>> serverStackTrace =
>>>> serverExceptionClass =
>>>> message = Cannot configure during output
>>>> }
>>>>
>>>
>>>
>>> _______________________________________________
>>> ome-devel mailing list
>>> ome-devel at lists.openmicroscopy.org.uk
>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>>
>
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
More information about the ome-devel
mailing list