[ome-devel] thumbnail generation + original files

Wed Dec 6 18:45:21 GMT 2006

Hi
Original Files are definitely still stored on the image server, and  
thumbnails are still automatically generated (not by the image  
server, but by the ImportEngine).

I suspect that something is going wrong when you import your files,  
but the error isn't propagated (or isn't obvious) for whatever reason.
It would be useful to try and import one file (or one file group)  
with debugging output on (setenv OME_DEBUG 1) and take a careful look  
at what's happening.
It looks like your importer is making the right calls, but I  
obviously can't run it because it depends on command-line utilities  
that I don't have.

This very well may be related.  The hash reference you're sending to  
getRegexGroups() is now keyed by file ID, not file name (the values  
of the hash are still file objects).  This is to parallel the hash  
structure you get as a parameter to your getGroups method.

This is almost surely unrelated, but your convertPlane method over- 
ride looks like its doing the same thing the image server would do  
with its convertPlane call.  Your original file is already uploaded  
to the image server by the time your code is reached, so you may as  
well use its method.  The whole local file class is deprecated and  
may be going away soon.

This is probably unrelated.  Its not clear from a cursory look that  
adding pixels to pre-existing images after the fact is going what you  
expectt.  Whatever it is though, it should work the same as before.

Ddebug output from importing one image's worth of files would be  
needed for more diagnosis.

Regarding the general strategy of your imports, you might be  
interested in taking a look at LOCI's BioFormats.  To integrate OME  
with this software package (which is in pure Java), we're doing the  
parsing and conversion on the image server using a BioFormats command- 
line utility.  BioFormats also likes reading files on the local file- 
system, but in this case its local file system is OMEIS's local file  
system.  The nice thing about BioFormats is that it converts all the  
metadata it extracts from the image files it can read to OME XML, so  
the importer on the "back-end" (i.e. the UCSFreader.pm equivalent) is  
trivial, not to mention universal.  The pixels are extracted locally  
to the image server using its own file system, so its not  
transferring a big payload over the network, or doing any unnecessary  
copying.  The other nice thing is that you can play with your formats  
in pure Java (which is a good thing, I hear), and you've already got  
support for most image formats.  There's also no OME voodoo in  
BioFormats per-se other than OME-XML - there are only a couple of  
OMEIS calls in the command-line utility wrapper to resolve IDs to  
local OMEIS paths.

-Ilya

On Dec 5, 2006, at 10:34 PM, kris wrote:

> Hello all,
>
>   I've recently tried upgrading from OME 2.4 to OME 2.6+ (cvs version
> dated 11/30).   I've noticed two annoyances and was wondering if
> anybody could shed some light on the situation.
>
> I first removed OME completely from the development machine.
>     (rm -rf /OME ; dropdb ome)
> and did a fresh install
>   sudo perl install.pl
>
> All went well as far I can tell.
>
> I then imported some images several hundred images, using the command
> line importer.
>
> 1.  Thumbnails are not generated automatically by OMEIS
>
>     under /OME/OMEIS/Pixels/
>     I see numbered (1-578) and .info files.
>       This appears to have changed since the last version where
>       it stored .bz2 and .thumb and .info files.
>
>     Permission on the /OME/OMEIS
>      $ ll -d /OME/OMEIS/Pixels/
>    drwx--S--- 2 www-data ome 28288 2006-12-05 17:20 /OME/OMEIS/ 
> Pixels//
>
>    Nothing else to speak of in the logs (apache) except the missing  
> file
>
>    Here is the generated thumbnail link:
>
>     http://arno.ece.ucsb.edu/cgi-bin/omeis? 
> Method=GetThumb&PixelsID=165;
>
>   and the response:
>
> Error calling GetThumb with PixelsID=165: Could not get information  
> for thumbnail at Pixels/165.thumb
> System Error: No such file or directory
>
>
> 2.  Original files are missing.
>
>      We use a temporary directory on symbolic links to our real data
>      during import.  In previous versions, the original file was also
>      stored on the OMEIS, is this not the case anymore?
>
>
> I should not that we are using our own image reader based on
> OME::ImportEngine::AbstractFormat.  I believe we have ported
> all the changes needed since the previous version (mostly cosmetic
> changes of renaming subroutine names i.e. __storeFileInfo ->
> storeFileInfo) and a few other dealing with the groups (filename ->  
> file
> id), but do not believe this is related.  The images themselves
> are there, just not the thumbnails and original files).
> Code is here just in case:
>
> http://biodev.ece.ucsb.edu/projects/bisque/browser/perl/BISQUE/ 
> ImportEngine/UCSBreader.pm
>
>
> Thanks for any help,
> kris
>
>
>
>
> -- 
> Kristian Kvilekval
> kris at cs.ucsb.edu  http://www.cs.ucsb.edu/~kris w:805-893-2526 h: 
> 504-9756
>
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>