Create a new dataset by stacking files in a folder
Options
e_pap
Registered Posts: 11 ✭
Hi team,
I am connecting with sharepoint folder to access a couple of files and I was wondering if there is a way to create a vertically-stacked dataset from all the files in the folder.
Thank you
Best Answer
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,209 DataikerOptionsAssuming the files in the folder have different schemas, you must use a code recipe.
If they all have the same schema, you can just use Files in Folder dataset. If you only have a few files you can use multiple and use a Visual Stack recipe
If you have a lot of files to read all files in a Folder and stack them into a data frame you can use something like below, but you need to decide on how you stack / concat these and how you consolidate the schema.
Answers
-
Thank you @AlexT
!