[ome-devel] Bio-Formats and IDL

Melissa Linkert melissa at glencoesoftware.com
Thu Apr 12 20:12:13 BST 2012 

Hi Guy,

> > 2 FilePatterns
> > I can successfully generate appropriate FilePatterns (through
> > FileStitcher) to read lists of tiff files in one directory.
> > However, our native file format that was set up about 10 years ago and
> > which we use extensively (which is not very sensible in retrospect, but it
> > made sense at the time) for 4D movies (tiffs) is:
> >
> > .../basepath/basename.ch_[c]/basename.[z]/basename[t].[z]
> >
> > where the [z] number is both in the name of a directory, and in the
> > filenames in that directory. I have not been able to find a way to set up
> > an appropriate FilePattern that works for this format. Is this too
> > complicated, or is there a way to do this either with regexp or by teaching
> > FilePattern that the [z] is used twice in the name?
> >
> 
> Hmm, unfortunately, I am guessing we will need to file a ticket for that.

I believe this ticket already covers what you are looking for:

https://trac.openmicroscopy.org.uk/ome/ticket/6479

If you would like to be automatically notified of updates, please let me
know and I will CC you.

> > 3 log4j
> > I'm afraid I'm not very experienced with Java, and I'm having trouble
> > getting log4j working.
> >
> > % log4j:WARN No appenders could be found for logger
> > (loci.common.NIOByteBufferProvider).
> > % log4j:WARN Please initialize the log4j system properly.
> >
> > There seems to be some log4j already bound up in your loci_tools.jar, and
> > I've tried various combinations of adding the various jars from the
> > log4j/slf4j site that you point us to, but in the end can't work out which
> > jars to include. Currently I am just using loci_tools.jar and idljavab.jar
> > (rather than bio-formats.jar and others, since one download is simpler than
> > many).
> >
>
> Yes, all the needed log4j stuff is already included in loci_tools.jar. So
> in theory those messages should not be appearing. You see them only when
> calling Bio-Formats from IDL?

You do not need any additional .jar files, but you do need to either add a
log4j.properties file in your CLASSPATH or call the "enableLogging"
method in loci.common.DebugTools.

An example log4j.properties file is here:

https://github.com/openmicroscopy/bioformats/blob/develop/tools/log4j.properties

And an example of how to call the enableLogging method is here:

https://github.com/openmicroscopy/bioformats/blob/develop/components/scifio/src/loci/formats/tools/ImageInfo.java#L920

Regards,
-Melissa

On Wed, Apr 11, 2012 at 02:02:53PM -0500, Curtis Rueden wrote:
> Hi Guy,
> 
> 
> On Wed, Apr 11, 2012 at 4:59 AM, Guy Blanchard <gb288 at cam.ac.uk> wrote:
> 
> > I hope you don't mind my replying directly - please pass on if someone
> > else is better suited to answer.
> >
> 
> No problem; I have CCed the appropriate mailing list.
> 
> 
> 
> > With thanks mainly to Karsten for showing how to get bio-formats reading
> > going through the IDL JavaBridge, I have written an object in IDL for
> > reading image volumes which is working fine on our Macs of various age, OS
> > and IDL version, and with the IDL Virtual Machine. It is happy with almost
> > all file formats I've thrown at it, and so I'm very happy with it. I have a
> > few questions below that I hope someone can answer.
> >
> > Once I have the IDL object running robustly for all our desired
> > functionality, I'd be very happy to share this, and support your project.
> >
> 
> That would be greatly appreciated!
> 
> FYI, effective nice way to share is via GitHub: fork the Bio-Formats
> repository (https://github.com/openmicroscopy/bioformats.git), make changes
> to your fork, then file a pull request for us to merge the changes back
> upstream.
> 
> 
> 
> > A few questions:
> >
> > 1 Looking at the metadata in many file formats, there is information about
> > the size of the x-, y- and z- steps, mostly/always in millimetres. I did
> > not find anything in your documentation about ImageReader (or other
> > objects) retrieving this information - either I've missed it, or is this
> > kind of information too varied between formats for you to be able to
> > extract it reliably?
> >
> 
> Bio-Formats does retrieve the information, and standardizes it to OME data
> structures (i.e., OME-XML). In OME terminology, they are called
> PhysicalSizeX, PhysicalSizeY and PhysicalSizeZ, and are stored in microns.
> 
> Here is an example of how to access it from Java:
> https://github.com/openmicroscopy/bioformats/blob/develop/components/bio-formats/utils/GetPhysicalMetadata.java
> 
> 
> 
> > 2 FilePatterns
> > I can successfully generate appropriate FilePatterns (through
> > FileStitcher) to read lists of tiff files in one directory.
> > However, our native file format that was set up about 10 years ago and
> > which we use extensively (which is not very sensible in retrospect, but it
> > made sense at the time) for 4D movies (tiffs) is:
> >
> > .../basepath/basename.ch_[c]/basename.[z]/basename[t].[z]
> >
> > where the [z] number is both in the name of a directory, and in the
> > filenames in that directory. I have not been able to find a way to set up
> > an appropriate FilePattern that works for this format. Is this too
> > complicated, or is there a way to do this either with regexp or by teaching
> > FilePattern that the [z] is used twice in the name?
> >
> 
> Hmm, unfortunately, I am guessing we will need to file a ticket for that.
> 
> 
> 
> > 3 log4j
> > I'm afraid I'm not very experienced with Java, and I'm having trouble
> > getting log4j working.
> >
> > % log4j:WARN No appenders could be found for logger
> > (loci.common.NIOByteBufferProvider).
> > % log4j:WARN Please initialize the log4j system properly.
> >
> > There seems to be some log4j already bound up in your loci_tools.jar, and
> > I've tried various combinations of adding the various jars from the
> > log4j/slf4j site that you point us to, but in the end can't work out which
> > jars to include. Currently I am just using loci_tools.jar and idljavab.jar
> > (rather than bio-formats.jar and others, since one download is simpler than
> > many).
> >
> 
> Yes, all the needed log4j stuff is already included in loci_tools.jar. So
> in theory those messages should not be appearing. You see them only when
> calling Bio-Formats from IDL?
> 
> 
> 
> > I'm hoping that once this is running, getting control back from the IDL
> > JavaBridge will be possible - at the moment if I interrupt while control is
> > with bio-formats through the Java Bridge, my session hangs.
> >
> 
> Sorry to hear that. Unfortunately, I am not experienced enough with IDL to
> be able to recommend a solution.
> 
> 
> Regards,
> Curtis
> 
> 
> On Wed, Apr 11, 2012 at 4:59 AM, Guy Blanchard <gb288 at cam.ac.uk> wrote:
> 
> > Hi Curtis,
> > I hope you don't mind my replying directly - please pass on if someone
> > else is better suited to answer.
> >
> > With thanks mainly to Karsten for showing how to get bio-formats reading
> > going through the IDL JavaBridge, I have written an object in IDL for
> > reading image volumes which is working fine on our Macs of various age, OS
> > and IDL version, and with the IDL Virtual Machine. It is happy with almost
> > all file formats I've thrown at it, and so I'm very happy with it. I have a
> > few questions below that I hope someone can answer.
> >
> > Once I have the IDL object running robustly for all our desired
> > functionality, I'd be very happy to share this, and support your project.
> >
> > A few questions:
> >
> > 1 Looking at the metadata in many file formats, there is information about
> > the size of the x-, y- and z- steps, mostly/always in millimetres. I did
> > not find anything in your documentation about ImageReader (or other
> > objects) retrieving this information - either I've missed it, or is this
> > kind of information too varied between formats for you to be able to
> > extract it reliably?
> >
> > 2 FilePatterns
> > I can successfully generate appropriate FilePatterns (through
> > FileStitcher) to read lists of tiff files in one directory.
> > However, our native file format that was set up about 10 years ago and
> > which we use extensively (which is not very sensible in retrospect, but it
> > made sense at the time) for 4D movies (tiffs) is:
> >
> > .../basepath/basename.ch_[c]/basename.[z]/basename[t].[z]
> >
> > where the [z] number is both in the name of a directory, and in the
> > filenames in that directory. I have not been able to find a way to set up
> > an appropriate FilePattern that works for this format. Is this too
> > complicated, or is there a way to do this either with regexp or by teaching
> > FilePattern that the [z] is used twice in the name?
> >
> > 3 log4j
> > I'm afraid I'm not very experienced with Java, and I'm having trouble
> > getting log4j working.
> >
> > % log4j:WARN No appenders could be found for logger
> > (loci.common.NIOByteBufferProvider).
> > % log4j:WARN Please initialize the log4j system properly.
> >
> > There seems to be some log4j already bound up in your loci_tools.jar, and
> > I've tried various combinations of adding the various jars from the
> > log4j/slf4j site that you point us to, but in the end can't work out which
> > jars to include. Currently I am just using loci_tools.jar and idljavab.jar
> > (rather than bio-formats.jar and others, since one download is simpler than
> > many).
> > I'm hoping that once this is running, getting control back from the IDL
> > JavaBridge will be possible - at the moment if I interrupt while control is
> > with bio-formats through the Java Bridge, my session hangs.
> >
> > Many thanks for your help,
> > Guy
> >
> >
> > =======================================
> >  Dr Guy Blanchard
> > Dept Physiology, Development and Neuroscience
> > Downing Site
> > Cambridge CB2 3DY
> >
> > email: gb288 at cam.ac.uk
> > mobile: +44 7930 988145
> > =======================================
> >
> >
> > On 9 Mar 2012, at 17:10, Curtis Rueden wrote:
> >
> > Hi Guy,
> >
> > Hi, we have cell tracking and analysis software that we have been
> >> developing for over 10 years now in IDL that we are planning to make
> >> generally available soon. After many years of fighting with lots of
> >> different image/movie formats, we are planning now to replace all our
> >> existing image file reading and management to your Bio-Formats C++ library
> >> that we can call both from within IDL and from within our C(++) DLLs that
> >> we call from IDL.
> >>
> >
> > This is exactly the sort of use case we want to enable with Bio-Formats.
> > We should not all have to "reinvent the wheel" to support the explosion of
> > proprietary file formats, but rather share a common library for doing so.
> >
> > Before embarking on this, I wondered if you think that in principle this
> >> should all work, and also whether you know of anyone that has already
> >> invested in calling your Bio-Formats library (in some form) from IDL that
> >> we could talk to?
> >>
> >
> > Karsten Rodenacker developed a script to call Bio-Formats from IDL using
> > the IDL-Java Bridge. With this approach, you do not need to use the
> > Bio-Formats C++ bindings to call from IDL, but rather can invoke
> > Bio-Formats more or less directly.
> >
> > Please see the Bio-Formats IDL page for details:
> >     http://loci.wisc.edu/bio-formats/idl
> >
> > Note that the Bio-Formats IDL script is several years old now and I am
> > sure Karsten would welcome any improvements. I have CCed Karsten on this
> > reply in case he has any further comments.
> >
> > Regarding your other requirement—calling Bio-Formats from C/C++ DLLs—the
> > Bio-Formats C++ bindings (BF-CPP) could be very useful for that. Be warned,
> > though, that we have found cross-platform BF-CPP support to be riddled with
> > obstacles, particularly with respect to dependency management and
> > portability of shared libraries. That said, BF-CPP is in use every day at
> > LOCI and works very well. And we have current funding to improve access to
> > Bio-Formats from C++, so this type of usage will become easier over time.
> >
> > More generally, could you comment on whether you intend to continue to
> >> keep your Bio-Formats up to date to be able to deal with new movie formats
> >> as they are 'invented' by the various Microscopy companies? We would like
> >> to invest in Bio-Formats - but only if you plan to keep up to date with
> >> continuing format developments.
> >>
> >
> > We are committed to developing and maintaining Bio-Formats now and for the
> > foreseeable future. The development is active, well funded, and vital to
> > many projects worldwide (see http://loci.wisc.edu/bio-formats/applications).
> > The project is a key component of the Open Microscopy Environment project (
> > http://openmicroscopy.org/), as well as the burgeoning SciJava initiative
> > (http://scijava.github.com/). It is here to stay.
> >
> > Regards,
> > Curtis
> >
> >
> > On Fri, Mar 9, 2012 at 4:33 AM, <gb288 at cam.ac.uk> wrote:
> >
> >> Dr Guy Blanchard sent a message using the contact form at
> >> http://loci.wisc.edu/contact.
> >>
> >> Hi, we have cell tracking and analysis software that we have been
> >> developing for over 10 years now in IDL that we are planning to make
> >> generally available soon. After many years of fighting with lots of
> >> different image/movie formats, we are planning now to replace all our
> >> existing image file reading and management to your Bio-Formats C++ library
> >> that we can call both from within IDL and from within our C(++) DLLs that
> >> we call from IDL.
> >> Before embarking on this, I wondered if you think that in principle this
> >> should all work, and also whether you know of anyone that has already
> >> invested in calling your Bio-Formats library (in some form) from IDL that
> >> we could talk to?
> >>
> >> More generally, could you comment on whether you intend to continue to
> >> keep your Bio-Formats up to date to be able to deal with new movie formats
> >> as they are 'invented' by the various Microscopy companies? We would like
> >> to invest in Bio-Formats - but only if you plan to keep up to date with
> >> continuing format developments.
> >>
> >> With thanks for your help,
> >> Guy
> >>
> >
> >
> >
> >
> >
> >

> _______________________________________________
> ome-devel mailing list
> ome-devel at lists.openmicroscopy.org.uk
> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel

More information about the ome-devel mailing list