[ome-devel] ScanR

[Rubén Muñoz]

Hi Melissa,
while now the bfconvert always generates an output file. Ive been  
testing to directly import the folder into OMERO.importer. For that:

- Replaced the bio-formats.jar into the importer folder
- Inserted 'Scanr' and 'Companion/Scanr' new records in the Omero  
Posgres DB at table 'formats'.

The result is:

- Was promoted for Screening name and Proceeded with 'Add to queue'.
- Import process ended without Errors.
- Screen in Omero appears empty at the end of the smooth process (no  
wells in the screen).
- At recent images some of the images 'could not be displayed because  
are invalid images'

I believe that youre very near to the Scanr importer, and wish to  
provide you with more detailed test. But as a hint I tried to convert  
our current Data Set to OME.tif with bfconvert and then to OME.tif  
again with OME.tif. Didnt work for me.

Is the order of the images in the output the correct? The sizes of the  
Data Set and output file do correspond. Also the Dimensions are well  
taken (Ive checked). Only the Timepoint aré calculated based on the  
other dimensions and the number of files...

My sincere gratitude,
Ruben

El 30/10/2009, a las 17:17, Melissa Linkert  
<melissa at glencoesoftware.com> escribió:

> Hi Ruben,
>
>> I believe you use -bigtiff flag but Im getting trouble with the big
>> datasets. Can you see if I miss something:
>>
>> ./bfconvert -bigtiff
>> /Users/gonzales/Images/161009test2/151009_001/data/--W00001-- 
>> P00001--Z00000--T00000--Cherry.tif test.ome.tif
>
> This is the correct command; however, there was a bug that caused the
> '-bigtiff' flag to be ignored.  This command should work as expected
> if you update to the latest trunk build of Bio-Formats.
>
>> While you improve the ScanR,  Josh suggested that we provide you  
>> Leica
>> "ome.tif" that are not really compliant. If theres place in the FTP  
>> I do it
>> right away with name Leica.tar.bz2
>
> Feel free to upload, unless Leica.tar.bz2 is larger than 6 GB.
>
> Regards,
> -Melissa
>
> On Thu, Oct 29, 2009 at 4:47 PM, Rubén Muñoz <ruben.munoz at embl.de> w 
> rote:
>> On Oct 29, 2009, at 5:12 PM, Melissa Linkert wrote:
>>
>>> Hi Ruben,
>>
>> Hi Melissa,
>>
>>>
>>> I'm moving this discussion to the ome-devel list, as it may be of
>>> interest to others.
>>>
>>>> Please consider my ScanrReader.java as possible help.
>>>
>>> Thank you very much for the patch.  I've committed a modified  
>>> version
>>> of it to the LOCI SVN repository; you can view the changes here:
>>
>> Thats so kind of you, glad to hear that.
>>
>>>
>>> https://skyking.microscopy.wisc.edu/trac/java/changeset/5652
>>
>> I believe you use -bigtiff flag but Im getting trouble with the big
>> datasets. Can you see if I miss something:
>>
>> ./bfconvert -bigtiff
>> /Users/gonzales/Images/161009test2/151009_001/data/--W00001-- 
>> P00001--Z00000--T00000--Cherry.tif
>> test.ome.tif
>> /Users/gonzales/Images/161009test2/151009_001/data/--W00001-- 
>> P00001--Z00000--T00000--Cherry.tif
>> [Olympus ScanR] -> test.ome.tif [OME-TIFF]
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> ... 
>> .........................................................Exception in
>> thread "main" loci.formats.FormatException: File is too large; call
>> setBigTiff(true)
>>        at loci.formats.out.TiffWriter.saveBytes(TiffWriter.java:188)
>>        at loci.formats.out.TiffWriter.saveBytes(TiffWriter.java:223)
>>        at loci.formats.out.OMETiffWriter.saveBytes 
>> (OMETiffWriter.java:193)
>>        at loci.formats.ImageWriter.saveBytes(ImageWriter.java:185)
>>        at
>> loci.formats.tools.ImageConverter.testConvert(ImageConverter.java: 
>> 228)
>>        at loci.formats.tools.ImageConverter.main 
>> (ImageConverter.java:253)
>>
>>>
>>> The only major difference between the committed changes and your  
>>> patch
>>> is that the committed changes do not have hard-coded well, position,
>>> Z, T, and channel counts.  The number of wells is currently being
>>> calculated from the well names in the "well selection table + cDNA"
>>> table.  The number of channels (core[0].sizeC) is equivalent to the
>>> number of channels defined in experiment_description.xml that have  
>>> the
>>> "idle" flag set to 0.
>>
>> So the information was there, just awaiting for someone to find the  
>> way to
>> read it.
>>
>>>
>>> The latest revision of ScanrReader does correctly detect the
>>> dimensions for all of the datasets that I have.  If you continue to
>>> experience problems, though, please let me know.
>>>
>>
>> While you improve the ScanR,  Josh suggested that we provide you  
>> Leica
>> "ome.tif" that are not really compliant. If theres place in the FTP  
>> I do it
>> right away with name Leica.tar.bz2
>>
>>> Regards,
>>> -Melissa
>>
>> Regards,
>>
>> Ruben
>>
>>>
>>> On Fri, Oct 23, 2009 at 5:53 PM, Rubén Muñoz <ruben.munoz at embl.d 
>>> e> wrote:
>>>>
>>>> Hi Melissa, thanks for your reply.
>>>>
>>>> I'd be happy to fix this, but first would like to clarify that an
>>>> assumption is correct.
>>>>
>>>> core[0].sizeC (the number of channels) is taken from a block like  
>>>> this:
>>>>
>>>> <Name>multiple_channel_typedef</Name>
>>>> <Dimsize>3</Dimsize>
>>>>
>>>> Yes, but not all the channels are finally used.
>>>> While the experiment_descriptor.xml reads:
>>>> <Name>multiple_channel_typedef</Name>
>>>> <Dimsize>12</Dimsize>
>>>> The directory only has two channel Cherry and eGFP.
>>>>
>>>> My understanding is that Dimsize is used in multiple places, and  
>>>> its
>>>> usage is determined by the value in in the "Name" element.  Is this
>>>> correct?  If so, do you know what the correct Name values are for
>>>> channels and positions?
>>>>
>>>> That's how it should be...
>>>> ... but the new set only gets converted for me when I fix all the  
>>>> next
>>>> values:
>>>> wellColumns = 2;
>>>> wellRows = 3;
>>>> core[0].sizeC = 2;
>>>> core[0].sizeT = 6;
>>>> core[0].sizeZ = 3;
>>>> Im sorry to do that now but a colleague is going to offer me  
>>>> details
>>>> about
>>>>  experiment_descriptor.xml.
>>>> Additionally I introduced some code to handle different positions  
>>>> in a
>>>> well,
>>>> P00001, P00002 and so on.
>>>> Please consider my ScanrReader.java as possible help. It is  
>>>> working well
>>>> for
>>>> the set and generates an OME.TIF with 1.7G that imports to OMERO  
>>>> and
>>>> displays Z, T, and C data in the correct way
>>>> .
>>>> Regards,
>>>> Ruben
>>>>
>>>>
>>>>
>>> <ScanrReader.java>
>>
>>