[ome-users] How big can a dataset be?

[Michael Porter]

Hi Anatole

I don't think it's a silly question, it's pretty close to a workflow that I thought about previously.

Firstly - that is a lot of images! Whether that 'breaks' things depends on how you plan to interact with the data. Browsing datasets in insight, I think, would not work. If you are querying the server from something like matlab it might work if you are very specific about the data you request. From my own experience in matlab, with datasets >15000 images, I can easily request planes if I supply an imageId. I can also successfully request all the image objects from that dataset. However, if I ask for everything I have on the server I get an error. I don't know how many images I have in total, but I'm guessing it's less than 500000. There doesn't appear to be a limit on the number of images you can put in a dataset though.

If by 'all the logic is handled outside' you mean you will keep track of imageIds etc as you import them, then I would say have a go! As for using tags to be able to search that much data I'm afraid that's outside of my experience.

Hope this helps,

Michael.


The University of Dundee is a registered Scottish Charity, No: SC015096
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openmicroscopy.org.uk/pipermail/ome-users/attachments/20140704/e18778c9/attachment.html>