Reading folder contents from a managed folder

vishet

Hi,

I am trying to write a recipe that reads from a managed folder and then creates a post request to an API with a file residing in the managed folder. The partial code for the recipe is as follows:

-------------------------------------------------------------------------

handle = dataiku.Folder("Ckfy9sf3")
paths = handle.list_paths_in_partition()
filenames= [path.replace("/","") for path in paths]
for path in filenames:
with handle.get_download_stream(path) as f:
headers = {
'accept': 'application/json',
'Authorization': bearer ,
}
files = {'file':(path, f.read())}
response = requests.post('URL of the API', headers=headers, files=files)
print(response.json())

----------------------------------------------------------------------

I am facing a problem where the API does not recognise the file like it should. I tried to perform this operation on my local machine with python(without managed folders of course and it worked just fine .My main question is: are files handled differently in dataiku and if yes, what are the conversions that I have to make to get my code to work?

fchataigner2

Hi,

the result of get_download_stream() can be passed as-is to the requests library. For example this works fine:

import dataiku
import os, requests

handle = dataiku.Folder("a")
paths = handle.list_paths_in_partition()
filenames = [os.path.basename(path) for path in paths]
for path in filenames:
    with handle.get_download_stream(path) as f:
        files = {'file':(path, f)}
        response = requests.post('https://httpbin.org/post', files=files)
        print(response)

What is the error you see?