Access Files on computer through Jupyter Notebook

WayneKhan
WayneKhan Registered Posts: 7 ✭✭✭✭

I apologize if this has been asked before. I did a search and couldn't find anything.

For example when I use a jupyter notebook through Ananconda Navigator I can read a csv or excel file on my computer. pd.read_csv('./Documents/folder/file.csv')

How do I do that when I use a jpuyter notebook through dss?

Answers

  • CoreyS
    CoreyS Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Registered Posts: 1,150 ✭✭✭✭✭✭✭✭✭

    Hi @WayneKhan
    while you wait me a more detailed response, and just incase you haven't accessed it already, I wanted to share with you Using Jupyter Notebooks in DSS from our Knowledge Base. I hope this helps!

  • Mark_Treveil
    Mark_Treveil Dataiker Alumni Posts: 30 ✭✭✭✭✭
    edited July 17

    Use a Managed Folder to store your file. DSS is not a desktop notebook system and designed to run on servers and on clouds.

    folder = dataiku.Folder("myFolder")
    with folder.get_download_stream("/accidents_2010.csv") as stream:
         df = pd.read_csv(stream)

    This works OK for me with a clean CSV file.

  • Vitaliy
    Vitaliy Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer Posts: 102 Dataiker
    edited July 17

    Hi @WayneKhan
    ,

    As Mark_Treveil mentioned, the proper way to handle any file in DSS is to add it to a Managed folder first and then utilize Dataiku API to open and process it in any way required. In some rare cases, you may still want to read files from system folders, the example below will give you an idea of how to do that:

    import dataiku
    from dataiku import pandasutils as pdu
    import pandas as pd
    import os
    
    #in this example file located in dataiku home directory.
    home_directory_path = os.path.expanduser("~")#find dataiku user home directory
    local_filename = os.path.join(home_directory_path, "us-500.csv")
    print(local_filename)
    with open(local_filename, "rb") as f:
        df = pd.read_csv(f)
    df.head()

    Screenshot 2021-05-28 at 18.08.22.png

    Hope this helps,

    Vitaliy

  • WayneKhan
    WayneKhan Registered Posts: 7 ✭✭✭✭

    Thanks everyone!

Setup Info
    Tags
      Help me…