Fetching folder and then excel file from that folder from S3 bucket in python

Options
Nainish09
Nainish09 Registered Posts: 7
edited July 16 in Using Dataiku

Hi,

Actually, I was fetching my folder easily from server's filesystem by using this code:

# Read recipe inputs

data_folder = dataiku.Folder("G9VbZqIP").get_path()
data_path= os.path.join(data_folder, "housing_magic.xlsx")

but as my now i want to access the folder and file which is stored in s3 bucket. How can i do it?

(Note: folder and file has been already stored in S3 bucket)


Operating system used: windows11

Tagged:

Answers

  • Sarina
    Sarina Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer Posts: 315 Dataiker
    edited July 17
    Options

    Hi @Nainish09
    ,

    Indeed you'll need to use remote access since the data is now stored on S3. Instead of using os.path, you'll want to use the managed folder API get_download_stream() to read your excel file.

    So you would simply do something like this instead:

    from dataiku
    import pandas as pd
    folder = dataiku.Folder("G9VbZqIP")
    with folder.get_download_stream("housing_magic.xlsx") as stream:
        df = pd.read_excel(stream.read())
    


    For example:

    Screen Shot 2023-04-07 at 4.37.54 PM.png
    If you run into any issues, let us know!

    Thanks,
    Sarina

Setup Info
    Tags
      Help me…