Read CSVs from a folder

[bored_panda]

I have a folder with CSVs in it (by "folder" I mean the thing you get when you're doing +dataset -> Folder from the flow) . They are named "dataset_01", "dataset_02" and so on.

I'm trying to read one of them in a Python recipe. What's the code ?

I tried something like this, but it wants me to add "path_of_csv" to inputs, so it's not what I'm looking for.

# -*- coding: utf-8 -*-
import dataiku
import pandas as pd, numpy as np
from dataiku import pandasutils as pdu
import os

# Recipe inputs
folder_path = dataiku.Folder("FuShmlsH").get_path()

path_of_csv = os.path.join(folder_path, "dataset_01.csv") 
my_dataset = dataiku.Dataset(path_of_csv).get_dataframe()

# Recipe outputs
test = dataiku.Dataset("test")
test.write_with_schema(my_dataset)

Thanks.

[cperdigou · Accepted solution]

Hello,

You can only import inputs to your recipe using "dataiku.Dataset("xx").get_dataframe()"

In your case, the input is not a dataset, it's a folder! So you correctly used "dataiku.Folder("xx")" already and you're done.

Now you can just read some files from it!

# -*- coding: utf-8 -*-
import dataiku
import pandas as pd, numpy as np
from dataiku import pandasutils as pdu
import os

# Recipe inputs
folder_path = dataiku.Folder("FuShmlsH").get_path()

path_of_csv = os.path.join(folder_path, "dataset_01.csv") 

my_dataset = pd.read_csv(path_of_csv)

[bored_panda]

Thanks.

Could you also give me the code to write a CSV to a folder please ?

[bored_panda]

In case it's of interest to anyone :

your_pandas_dataframe.to_csv(os.path.join(write_path, "name_of_file"), sep=";")