Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Hi everyone 🙂
I am trying to create dataset from a csv file contained in a managed folder. This is something that I did several times without having any particular issue.
This time however, once I have imported the list of paths, when I try to read the csv file, I get error:
FileNotFoundError: [Errno 2] No such file or directory: '/app/dataiku/data_dir/managed_folders/ANNUAL_SHAREHOLDING_STUDY/Wz3QQFf4/FS0000AQGD.csv'
The code I have written is the following.
folder_path = dataiku.Folder("Wz3QQFf4").get_path()
folder_files = dataiku.Folder("Wz3QQFf4").list_paths_in_partition()
path_of_csv = os.path.join(folder_path, folder_files[1:])
df = pd.read_csv(path_of_csv)
Thanks a lot 🙂
@pao800a ,Is this a locally managed folder or remotely managed folder( S3, GCS, Azure Blob)Are you using local execution or containerized?In both of the above cases using os.path.join will not work and you should use the read/write API get_download_stream to read the file instead.or you can use +Dataset - Internal Files in Folder dataset type.Thanks
Post a Question