Writing files to a folder in a project
Hi,
I manually created a folder in my project and want to download a zip file, via REST API, into it. I get this permissions error:
Permission denied: '/data/dataiku/data_dir/managed_folders/MYPROJECT/pMlFp07H/blah_2023-08-30T12-28-51_kdfs.zip
I know how to write to a folder with the folder handle. Is there anyway I can workaround this so I can download directly from a requests call?
Operating system used: Windows 10
Answers
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,043 Neuron
Can you please post your code?
-
resp = requests.get(zip_url, stream=True) if resp.status_code == 200: total = int(resp.headers.get("content-length", 0)) file_path = ( file_path if file_path else self.controller._cache_dir / f"{zipname}.zip" ) with open(file_path, "wb") as file, tqdm( desc=f"Downloading to {file_path}", total=total, unit="B", unit_scale=True, unit_divisor=1024, ) as bar: for data in resp.iter_content(chunk_size=1024): size = file.write(data) bar.update(size)
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,043 Neuron
I am bit confused on what you are trying to do. First you say:
@info-rchitect
wrote:
I manually created a folder in my project and want to download a zip file, via REST API, into it. I get this permissions error:"download a zip file into it" sure means to upload a file to the folder not download. But then you say:
@info-rchitect
wrote:
I know how to write to a folder with the folder handle. Is there anyway I can workaround this so I can download directly from a requests call?"Write to a folder" indicates you are uploading to a folder but then you say "so I can download directly from a requests call". So are you trying put (upload) or get (download) a file from managed folder?
Now to your code. The code you posted is pure Python code trying to write to a folder. I see nowhere you using the relevant REST API call: https://doc.dataiku.com/dss/api/12/rest/#managed-folders-managed-folder-contents-post
You get permission denied with your code because Dataiku will not allow you to execute code that writes directly to OS folders otherwise imagine what a rogue user could do. So why can't you use the Dataiku Python APIs to upload your file?
-
I was unaware of the POST REST APIs, thx.
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,043 Neuron
No worries. It is not a well known API but it’s basically the backend of the dataikuapi package which wraps around the REST API.