[ome-devel] Bio-Formats and IDL

Wed Apr 11 20:02:53 BST 2012

Hi Guy,

On Wed, Apr 11, 2012 at 4:59 AM, Guy Blanchard <gb288 at cam.ac.uk> wrote:

> I hope you don't mind my replying directly - please pass on if someone
> else is better suited to answer.
>

No problem; I have CCed the appropriate mailing list.

> With thanks mainly to Karsten for showing how to get bio-formats reading
> going through the IDL JavaBridge, I have written an object in IDL for
> reading image volumes which is working fine on our Macs of various age, OS
> and IDL version, and with the IDL Virtual Machine. It is happy with almost
> all file formats I've thrown at it, and so I'm very happy with it. I have a
> few questions below that I hope someone can answer.
>
> Once I have the IDL object running robustly for all our desired
> functionality, I'd be very happy to share this, and support your project.
>

That would be greatly appreciated!

FYI, effective nice way to share is via GitHub: fork the Bio-Formats
repository (https://github.com/openmicroscopy/bioformats.git), make changes
to your fork, then file a pull request for us to merge the changes back
upstream.

> A few questions:
>
> 1 Looking at the metadata in many file formats, there is information about
> the size of the x-, y- and z- steps, mostly/always in millimetres. I did
> not find anything in your documentation about ImageReader (or other
> objects) retrieving this information - either I've missed it, or is this
> kind of information too varied between formats for you to be able to
> extract it reliably?
>

Bio-Formats does retrieve the information, and standardizes it to OME data
structures (i.e., OME-XML). In OME terminology, they are called
PhysicalSizeX, PhysicalSizeY and PhysicalSizeZ, and are stored in microns.

Here is an example of how to access it from Java:
https://github.com/openmicroscopy/bioformats/blob/develop/components/bio-formats/utils/GetPhysicalMetadata.java

> 2 FilePatterns
> I can successfully generate appropriate FilePatterns (through
> FileStitcher) to read lists of tiff files in one directory.
> However, our native file format that was set up about 10 years ago and
> which we use extensively (which is not very sensible in retrospect, but it
> made sense at the time) for 4D movies (tiffs) is:
>
> .../basepath/basename.ch_[c]/basename.[z]/basename[t].[z]
>
> where the [z] number is both in the name of a directory, and in the
> filenames in that directory. I have not been able to find a way to set up
> an appropriate FilePattern that works for this format. Is this too
> complicated, or is there a way to do this either with regexp or by teaching
> FilePattern that the [z] is used twice in the name?
>

Hmm, unfortunately, I am guessing we will need to file a ticket for that.

> 3 log4j
> I'm afraid I'm not very experienced with Java, and I'm having trouble
> getting log4j working.
>
> % log4j:WARN No appenders could be found for logger
> (loci.common.NIOByteBufferProvider).
> % log4j:WARN Please initialize the log4j system properly.
>
> There seems to be some log4j already bound up in your loci_tools.jar, and
> I've tried various combinations of adding the various jars from the
> log4j/slf4j site that you point us to, but in the end can't work out which
> jars to include. Currently I am just using loci_tools.jar and idljavab.jar
> (rather than bio-formats.jar and others, since one download is simpler than
> many).
>

Yes, all the needed log4j stuff is already included in loci_tools.jar. So
in theory those messages should not be appearing. You see them only when
calling Bio-Formats from IDL?

> I'm hoping that once this is running, getting control back from the IDL
> JavaBridge will be possible - at the moment if I interrupt while control is
> with bio-formats through the Java Bridge, my session hangs.
>

Sorry to hear that. Unfortunately, I am not experienced enough with IDL to
be able to recommend a solution.

Regards,
Curtis

On Wed, Apr 11, 2012 at 4:59 AM, Guy Blanchard <gb288 at cam.ac.uk> wrote:

> Hi Curtis,
> I hope you don't mind my replying directly - please pass on if someone
> else is better suited to answer.
>
> With thanks mainly to Karsten for showing how to get bio-formats reading
> going through the IDL JavaBridge, I have written an object in IDL for
> reading image volumes which is working fine on our Macs of various age, OS
> and IDL version, and with the IDL Virtual Machine. It is happy with almost
> all file formats I've thrown at it, and so I'm very happy with it. I have a
> few questions below that I hope someone can answer.
>
> Once I have the IDL object running robustly for all our desired
> functionality, I'd be very happy to share this, and support your project.
>
> A few questions:
>
> 1 Looking at the metadata in many file formats, there is information about
> the size of the x-, y- and z- steps, mostly/always in millimetres. I did
> not find anything in your documentation about ImageReader (or other
> objects) retrieving this information - either I've missed it, or is this
> kind of information too varied between formats for you to be able to
> extract it reliably?
>
> 2 FilePatterns
> I can successfully generate appropriate FilePatterns (through
> FileStitcher) to read lists of tiff files in one directory.
> However, our native file format that was set up about 10 years ago and
> which we use extensively (which is not very sensible in retrospect, but it
> made sense at the time) for 4D movies (tiffs) is:
>
> .../basepath/basename.ch_[c]/basename.[z]/basename[t].[z]
>
> where the [z] number is both in the name of a directory, and in the
> filenames in that directory. I have not been able to find a way to set up
> an appropriate FilePattern that works for this format. Is this too
> complicated, or is there a way to do this either with regexp or by teaching
> FilePattern that the [z] is used twice in the name?
>
> 3 log4j
> I'm afraid I'm not very experienced with Java, and I'm having trouble
> getting log4j working.
>
> % log4j:WARN No appenders could be found for logger
> (loci.common.NIOByteBufferProvider).
> % log4j:WARN Please initialize the log4j system properly.
>
> There seems to be some log4j already bound up in your loci_tools.jar, and
> I've tried various combinations of adding the various jars from the
> log4j/slf4j site that you point us to, but in the end can't work out which
> jars to include. Currently I am just using loci_tools.jar and idljavab.jar
> (rather than bio-formats.jar and others, since one download is simpler than
> many).
> I'm hoping that once this is running, getting control back from the IDL
> JavaBridge will be possible - at the moment if I interrupt while control is
> with bio-formats through the Java Bridge, my session hangs.
>
> Many thanks for your help,
> Guy
>
>
> =======================================
>  Dr Guy Blanchard
> Dept Physiology, Development and Neuroscience
> Downing Site
> Cambridge CB2 3DY
>
> email: gb288 at cam.ac.uk
> mobile: +44 7930 988145
> =======================================
>
>
> On 9 Mar 2012, at 17:10, Curtis Rueden wrote:
>
> Hi Guy,
>
> Hi, we have cell tracking and analysis software that we have been
>> developing for over 10 years now in IDL that we are planning to make
>> generally available soon. After many years of fighting with lots of
>> different image/movie formats, we are planning now to replace all our
>> existing image file reading and management to your Bio-Formats C++ library
>> that we can call both from within IDL and from within our C(++) DLLs that
>> we call from IDL.
>>
>
> This is exactly the sort of use case we want to enable with Bio-Formats.
> We should not all have to "reinvent the wheel" to support the explosion of
> proprietary file formats, but rather share a common library for doing so.
>
> Before embarking on this, I wondered if you think that in principle this
>> should all work, and also whether you know of anyone that has already
>> invested in calling your Bio-Formats library (in some form) from IDL that
>> we could talk to?
>>
>
> Karsten Rodenacker developed a script to call Bio-Formats from IDL using
> the IDL-Java Bridge. With this approach, you do not need to use the
> Bio-Formats C++ bindings to call from IDL, but rather can invoke
> Bio-Formats more or less directly.
>
> Please see the Bio-Formats IDL page for details:
>     http://loci.wisc.edu/bio-formats/idl
>
> Note that the Bio-Formats IDL script is several years old now and I am
> sure Karsten would welcome any improvements. I have CCed Karsten on this
> reply in case he has any further comments.
>
> Regarding your other requirement—calling Bio-Formats from C/C++ DLLs—the
> Bio-Formats C++ bindings (BF-CPP) could be very useful for that. Be warned,
> though, that we have found cross-platform BF-CPP support to be riddled with
> obstacles, particularly with respect to dependency management and
> portability of shared libraries. That said, BF-CPP is in use every day at
> LOCI and works very well. And we have current funding to improve access to
> Bio-Formats from C++, so this type of usage will become easier over time.
>
> More generally, could you comment on whether you intend to continue to
>> keep your Bio-Formats up to date to be able to deal with new movie formats
>> as they are 'invented' by the various Microscopy companies? We would like
>> to invest in Bio-Formats - but only if you plan to keep up to date with
>> continuing format developments.
>>
>
> We are committed to developing and maintaining Bio-Formats now and for the
> foreseeable future. The development is active, well funded, and vital to
> many projects worldwide (see http://loci.wisc.edu/bio-formats/applications).
> The project is a key component of the Open Microscopy Environment project (
> http://openmicroscopy.org/), as well as the burgeoning SciJava initiative
> (http://scijava.github.com/). It is here to stay.
>
> Regards,
> Curtis
>
>
> On Fri, Mar 9, 2012 at 4:33 AM, <gb288 at cam.ac.uk> wrote:
>
>> Dr Guy Blanchard sent a message using the contact form at
>> http://loci.wisc.edu/contact.
>>
>> Hi, we have cell tracking and analysis software that we have been
>> developing for over 10 years now in IDL that we are planning to make
>> generally available soon. After many years of fighting with lots of
>> different image/movie formats, we are planning now to replace all our
>> existing image file reading and management to your Bio-Formats C++ library
>> that we can call both from within IDL and from within our C(++) DLLs that
>> we call from IDL.
>> Before embarking on this, I wondered if you think that in principle this
>> should all work, and also whether you know of anyone that has already
>> invested in calling your Bio-Formats library (in some form) from IDL that
>> we could talk to?
>>
>> More generally, could you comment on whether you intend to continue to
>> keep your Bio-Formats up to date to be able to deal with new movie formats
>> as they are 'invented' by the various Microscopy companies? We would like
>> to invest in Bio-Formats - but only if you plan to keep up to date with
>> continuing format developments.
>>
>> With thanks for your help,
>> Guy
>>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-devel/attachments/20120411/de2c2748/attachment-0001.html>