Writing files to a folder in a project

info-rchitect

Hi,

I manually created a folder in my project and want to download a zip file, via REST API, into it. I get this permissions error:

Permission denied: '/data/dataiku/data_dir/managed_folders/MYPROJECT/pMlFp07H/blah_2023-08-30T12-28-51_kdfs.zip

I know how to write to a folder with the folder handle. Is there anyway I can workaround this so I can download directly from a requests call?

---

Operating system used: Windows 10

Turribeach

Can you please post your code?

info-rchitect

                resp = requests.get(zip_url, stream=True)
                if resp.status_code == 200:
                    total = int(resp.headers.get("content-length", 0))
                    file_path = (
                        file_path if file_path else self.controller._cache_dir / f"{zipname}.zip"
                    )
                    with open(file_path, "wb") as file, tqdm(
                        desc=f"Downloading to {file_path}",
                        total=total,
                        unit="B",
                        unit_scale=True,
                        unit_divisor=1024,
                    ) as bar:
                        for data in resp.iter_content(chunk_size=1024):
                            size = file.write(data)
                            bar.update(size)

Turribeach

I am bit confused on what you are trying to do. First you say:

---

@info-rchitect
wrote:
I manually created a folder in my project and want to download a zip file, via REST API, into it. I get this permissions error:

---

"download a zip file into it" sure means to upload a file to the folder not download. But then you say:

---

@info-rchitect
wrote:
I know how to write to a folder with the folder handle. Is there anyway I can workaround this so I can download directly from a requests call?

---

"Write to a folder" indicates you are uploading to a folder but then you say "so I can download directly from a requests call". So are you trying put (upload) or get (download) a file from managed folder?

Now to your code. The code you posted is pure Python code trying to write to a folder. I see nowhere you using the relevant REST API call: https://doc.dataiku.com/dss/api/12/rest/#managed-folders-managed-folder-contents-post

You get permission denied with your code because Dataiku will not allow you to execute code that writes directly to OS folders otherwise imagine what a rogue user could do. So why can't you use the Dataiku Python APIs to upload your file?

info-rchitect

I was unaware of the POST REST APIs, thx.

Turribeach

No worries. It is not a well known API but it’s basically the backend of the dataikuapi package which wraps around the REST API.