Reading a file from a Managed Folder
pao800a
Registered Posts: 1 ✭
Hi everyone
I am trying to create dataset from a csv file contained in a managed folder. This is something that I did several times without having any particular issue.
This time however, once I have imported the list of paths, when I try to read the csv file, I get error:
FileNotFoundError: [Errno 2] No such file or directory: '/app/dataiku/data_dir/managed_folders/ANNUAL_SHAREHOLDING_STUDY/Wz3QQFf4/FS0000AQGD.csv'
The code I have written is the following.
folder_path = dataiku.Folder("Wz3QQFf4").get_path() folder_files = dataiku.Folder("Wz3QQFf4").list_paths_in_partition() path_of_csv = os.path.join(folder_path, folder_files[0][1:]) print(path_of_csv) df = pd.read_csv(path_of_csv)
Thanks a lot
Answers
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,226 Dataiker
@pao800a
,
Is this a locally managed folder or remotely managed folder( S3, GCS, Azure Blob)
Are you using local execution or containerized?
In both of the above cases using os.path.join will not work and you should use the read/write API get_download_stream to read the file instead.
or you can use +Dataset - Internal Files in Folder dataset type.
Thanks