[ome-devel] SPW ome-tiffs & how to proceed?

[William Moore]

Hi Ian,

 The only downside to "one big SPW file" is that the size of it could get beyond what is practical to download in web.

We have to build a zip file on the fly that contains all the files in the file-set. This has to happen faster than the http request timeout.
Then the zip has to be a reasonable size to download, which will depend on bandwidth etc.
I don't know the upper limit on what will actually work, but I imagine it's possible to have a plate exceed several GB?

Also, I'm wondering if all software that reads one.tiff will handle SPW data?
Maybe if your users are using BioFormats to read the data, it may handle SPW just as easily as single ome.tiffs?
But is that also true of other packages that read one.tiff?

Anyway, it doesn't affect my web code - just wanted to flag a couple of considerations.

Should have a PR open soon….


  Will.



On 18 Aug 2014, at 15:16, "Munro, Ian" <i.munro at imperial.ac.uk> wrote:

> Hi again Will
> 
> After further discussions  we’re leaning towards putting the whole plate set into one big file. (option 2) in my original post).
> If you think that’s likely to cause any problems with implementing downloading then please let me know?
> 
> Ian
> 
> 
> On 18 Aug 2014, at 13:18, Munro, Ian <i.munro at imperial.ac.uk> wrote:
> 
>> Thanks again Will 
>> 
>> That information helps a lot. Am I then correct  that if I were to get the binaryOnly option working , as suggested by Curtis, that 
>> the whole plate would download as it’s a file set?
>> 
>> Ian
>> 
>> 
>> 
>> On 18 Aug 2014, at 13:09, William Moore <will at lifesci.dundee.ac.uk> wrote:
>> 
>>> Hi Ian,
>>> 
>>> For "fixing downloads", it turns out that there's not much difference whether you have "1 big file for the whole plate"
>>> or if you've created the plate in OMERO via script/API.
>>> 
>>> In both cases, for all the selected images, we just lookup the fileset(s) and download all the linked files.
>>> 
>>> I guess the main question is what you want to be in the download:
>>> 
>>> If you have "1 big file for the whole plate" then you'll only be able to download the whole plate at a time,
>>> since there will be a single file-set for all the images.
>>> 
>>> If you import the images into OMERO separately, there will be a fileset for each image and you will
>>> just download the files you need for the selected images (with no SPW info).
>>> 
>>> Will.
>>> 
>>> 
>>> 
>>> On 18 Aug 2014, at 12:57, "Munro, Ian" <i.munro at imperial.ac.uk> wrote:
>>> 
>>>> Thanks for the reply  Will
>>>> 
>>>> The final decision is down to the PIs I’m trying to gather information about our options.
>>>> Would it be easier to fix the download for this option than say having 1 big file for the whole plate?
>>>> 
>>>> 
>>>> Ian
>>>> 
>>>> 
>>>> On 18 Aug 2014, at 12:16, William Moore <will at lifesci.dundee.ac.uk> wrote:
>>>> 
>>>>> Hi Ian,
>>>>> 
>>>>> On 15 Aug 2014, at 15:24, "Munro, Ian" <i.munro at imperial.ac.uk> wrote:
>>>>>> 
>>>>>> 6)  Store data as ome-tiffs with no SPW xml but add this info in OMERO after import (either via dataset-to-plate script or the API)
>>>>>>   Estimated bloat. Unclear -ask OME team
>>>>>>   Pros. Might allow one file per FOV & work with future plate tools.
>>>>>>   Cons: Might not work - ask OME team
>>>>>>             Big software effort and end of project is approaching rapidly.
>>>>>> 
>>>>> 
>>>>> If you are happy to try this approach, I should be able to fix web so that you can download
>>>>> the original files for selected wells, just as if you'd selected multiple images in a Dataset.
>>>>> 
>>>>> Of course, the images you download will have no SPW data associated with them,
>>>>> but if that's OK for you, then let's give this a try…
>>>>> 
>>>>> 
>>>>> Will.
>>>>> 
>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> ome-devel mailing list
>>>>>> ome-devel at lists.openmicroscopy.org.uk
>>>>>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>>>>> 
>>>> 
>>> 
>> 
>> _______________________________________________
>> ome-devel mailing list
>> ome-devel at lists.openmicroscopy.org.uk
>> http://lists.openmicroscopy.org.uk/mailman/listinfo/ome-devel
>