I have a code recipe currently running on DSS, i switched the computation from DSS to Kubernetes and stated having issues.
How do I move from folder from managed folder in DSS to Kubernetes.
Note : i want to be able to call all the dat in the folder not individual data
Below is the python code i have written so far :
# Read recipe inputs
Clinical_data = dataiku.Folder("TEST_WITH_FANG.LeD8SADS")
Clinical_data_info = Clinical_data.get_info()
# pass a partition identifier if the folder is partitioned
#paths = Clinical_data.list_paths_in_partition()
path = Clinical_data.get_path()
for file_path in paths:
with Clinical_data.get_download_stream(file_path) as f:
with open(os.path.join('test_data', file_path[1:]), 'wb') as local_fp:
testing = local_fp.write(f.read())