Submit your innovative use case or inspiring success story to the 2023 Dataiku Frontrunner Awards! LET'S GO

Fetching folder and then excel file from that folder from S3 bucket in python

Nainish09
Level 2
Fetching folder and then excel file from that folder from S3 bucket in python

Hi,

Actually, I was fetching my folder easily from server's filesystem by using this code:

# Read recipe inputs

data_folder = dataiku.Folder("G9VbZqIP").get_path()
data_path= os.path.join(data_folder, "housing_magic.xlsx")

but as my now i want to access the folder and file which is stored in s3 bucket. How can i do it?

(Note: folder and file has been already stored in S3 bucket)


Operating system used: windows11

0 Kudos
1 Reply
SarinaS
Dataiker

Hi @Nainish09,

Indeed you'll need to use remote access since the data is now stored on S3.  Instead of using os.path, you'll want to use the managed folder API get_download_stream() to read your excel file. 

So you would simply do something like this instead:

from dataiku
import pandas as pd
folder = dataiku.Folder("G9VbZqIP")
with folder.get_download_stream("housing_magic.xlsx") as stream:
    df = pd.read_excel(stream.read())


For example:

Screen Shot 2023-04-07 at 4.37.54 PM.png
If you run into any issues, let us know! 

Thanks,
Sarina

0 Kudos