Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
We have multiple csv files being read by amazon s3 and we figured out that there is schema inconsistency in it. How can we handle it using dataiku?
Because as per my knowledge it will only read certain files with matching schema to the first in the line.
Operating system used: Ubuntu
Hi @Rushil09 ,
In this case, it seems it would be better to use Folder to upload the different files with different schemas.
Then use files in the folder dataset ( +Dataset - Internal - Files in folder) to create various datasets from the files and then stack them ( Stack recipe) as needed.
With "files in folder" dataset you can also specify which file to read the schema from.
Hope that helps!