Can I populate dataset by adding folder instead of files?

Min_Shi

When I populate mnist dataset, I have to create each folder one by one, like creating folder 0, 1, 2, ...9, then upload files to each of these folders. I would like to know if there is easy way to populate dataset from the parent folder, which the sub folders have been well organized on my local disk.

rtaylor

Are you trying to place a collection of files into a single DSS folder, or are you trying to nest DSS folders?

You can place any number of files (within reason) into a single DSS folder, and then create a single dataset from the folder. You should ensure the schema for each file is identical thought. As far as nesting DSS folders to mimic a files system structure, I am not sure if that is possible and have never tried.

A third method, although I have had mixed experiences with it, is to create folders in DSS that mirror my local folders, and then upload files folder by folder, then combine the folders in subsequent flow steps (Prepare/Stack/Python)

simonamaggio

Hi Min_Shi,

You can actually zip the folder on your local machine, upload the whole zip file and use the button 'Uncompress to current folder' in the right window of the dataset view (see attached picture).

For the specific tutorial you are looking at, you can also use a 'Download ' visual recipe to download the MNIST archive: https://github.com/myleott/mnist_png/raw/master/mnist_png.tar.gz. And then uncompress the file as mentioned before.