CSV files with Date Column and Others with no Date Column - How to combine them?
Hi,
I have several CSV files in a 2024 folder that feeds into DataIku. These files have a date column already. The new files that I will be adding to this folder will not have a date column.
Example:
Existing files -
Column A | Column B | Column C
02-10-2024 | Name | Product
New files
Column A | Column B | Column C
Name | Product | ID#
Can I have DataIku add a date value and move the Name value, Product value and ID# over to the right?
I know I could create a new dataset and add the date column and merge the datasets after, but I was wondering if there was another way of doing it before making the build more complex.
Thanks,
Operating system used: Windows
Answers
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,925 Neuron
I recently responded to a very similar question to yours. In summary either use a Python recipe to read all the files from the folder and stitch them together manually in code or use a pair of Files in Folder datasets to split the files by structure and then stack them together with a Stack recipe. Basically choose between the "Coders" or the "Clickers" approach.