[ome-devel] OME install woes: upload file through omeis

[Ilya Goldberg]

Hi all
More fun programming around other people's bugs.
Turns out this error:
500 EOF when chunk header expected
reported when uploading OME XML files during installation on some  
"latest-greatest" platforms had nothing to do with libxml.  Libxml  
may be the cause of other problems (please report any!), but this one  
is caused by Perl's LWP module (libwww-perl).  Googling around  
revealed that this affected a whole slew of programs other than OME.
Strong work.

A number of mitigating factors led me to drop the dependency on LWP:
* Its the second time we've run across a bug in the LWP included in  
stock Linux distros.
* We already have a dependency on libcurl (from C), which does the  
same thing.
* libwww is slower than libcurl.
* libwww-perl's API suffers from excessive application of "There's  
more than one way to do it" (corollary: "There's more than one way to  
screw it up").
* I wanted to have a hand with Inline::C

So, the solution is waiting for y'all on CVS.  I rolled up our own  
binding for libcurl using Inline::C (its the coolest thing since  
sliced bread, BTW), dropped all the dependencies on LWP and libwww,  
and put Inline::C (and its Parse::RecDescent dependency) in the perl  
module check and in the module repository (http://openmicroscopy.org/ 
packages/perl/).

Why didn't I use the existing perl binding for libcurl?  Really  
didn't like its API - its basically the same as C with all of its  
callback horribleness.  Cribbed a chunk of code from Apache's test  
script, which rolls its own perl binding for libcurl in 1/2 a page of  
code using Inline::C (!).  Ours is fancier at about a page and a 1/2,  
but Apache's test script was the inspiration.  The new cURL binding  
is in src/perl2/OME/Util/cURL.pm.

Well, try it out and see how you like it.
I had Curtis try out an intermediate form of this solution and it got  
rid of his "500 EOF when chunk header expected" errors.
Note that Postgres 8.1 is still not working!  Please downgrade to a  
8.0.x version if 'psql -V' reports 8.1 or greater.

-Ilya

P.S.:
Due to a "regression" in Fedora Core 2, this does not work.   
Actually, any newly compiled perl bindings for C libraries don't work  
in FC2, and require manually editing the Makefile produced by the  
'perl Makefile.PL' command to re-add the shared libraries taken away  
by the "regression".  Read more about it here:
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=136009
This has been fixed in FC3 updates and FC4.

It would be good to know which other distros suffered from this.
The error is generated right after installing the core Perl modules.   
It reports that 'curl_easy_init' is an unresolved symbol.
Checking library dependencies on the code bundle:
ldd /var/tmp/OME/Inline/lib/auto/OME/Util/cURL/cURL.so
reports that there is no dependency on libcurl.

Tried getting around this one for a while, but life's too short.
So the short answer is don't use FC2, which is "legacy" now anyway.   
FC4 is the recommended platform (which is now reflected on the web- 
site, or will be tomorrow).

-I

On Jun 1, 2006, at 1:27 PM, Ilya Goldberg wrote:

>
> On Jun 1, 2006, at 12:03 PM, august wrote:
>
>>> There were some recent commits to CVS for the image server.
>>> The original problem was an attempted work-around for a race
>>> condition in Berkeley DB, which apparently held for the versions  
>>> that
>>> were tested, and failed miserably for others - especially when first
>>> populating a new OMEIS repository.
>>>
>
> [snip]
>
>>> Anyway, long story short:  It would be helpful to try this again to
>>> see if this was related to the recent OMEIS/Berkeley DB fix.
>>
>>
>> Ilya,
>>
>> thanks for the report.   Unfortunately, I have a working installtion
>> right now with lots of data and I don't want to muck with it.   As
>> soon
>> as I make a new install attempt (which should be soon) I will
>> report any
>> oddities.   [Or, is there a way to check the install withough
>> having to
>> dump the DB?]
>
> The installer doesn't require dropping the DB - its designed to be an
> updater as well.
> You can update just by saying 'sudo perl install.pl -y', and it will
> keep all your previous settings without asking for any input.
>
> It will go through the motions of re-loading all the XML, so its not
> a bad test (though it would be on a populated repository rather than
> a new one).
> Its still a good idea to do a pg_dumpall before updating though.
>
> -Ilya
>
>
>
>>
>> -august.
>>
>
> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>