Community Conundrum 25:Feature Visualization is now live! Read More

Read CSVs from a folder

Level 2
Read CSVs from a folder

I have a folder with CSVs in it (by "folder" I mean the thing you get when you're doing +dataset -> Folder from the flow) . They are named "dataset_01", "dataset_02" and so on.



I'm trying to read one of them in a Python recipe. What's the code ?



I tried something like this, but it wants me to add "path_of_csv" to inputs, so it's not what I'm looking for.



 




# -*- coding: utf-8 -*-
import dataiku
import pandas as pd, numpy as np
from dataiku import pandasutils as pdu
import os

# Recipe inputs
folder_path = dataiku.Folder("FuShmlsH").get_path()

path_of_csv = os.path.join(folder_path, "dataset_01.csv")
my_dataset = dataiku.Dataset(path_of_csv).get_dataframe()

# Recipe outputs
test = dataiku.Dataset("test")
test.write_with_schema(my_dataset)


Thanks.

0 Kudos
3 Replies
Dataiker
Dataiker

Hello,



You can only import inputs to your recipe using "dataiku.Dataset("xx").get_dataframe()"



In your case, the input is not a dataset, it's a folder! So you correctly used "dataiku.Folder("xx")" already and you're done.



Now you can just read some files from it!




# -*- coding: utf-8 -*-
import dataiku
import pandas as pd, numpy as np
from dataiku import pandasutils as pdu
import os

# Recipe inputs
folder_path = dataiku.Folder("FuShmlsH").get_path()

path_of_csv = os.path.join(folder_path, "dataset_01.csv")

my_dataset = pd.read_csv(path_of_csv)


 

Level 2
Author
Thanks.

Could you also give me the code to write a CSV to a folder please ?
0 Kudos
Level 2
Author
In case it's of interest to anyone :

your_pandas_dataframe.to_csv(os.path.join(write_path, "name_of_file"), sep=";")
0 Kudos
Labels (4)