"No DSS URL or API key found from any location" when trying to get path of a dataset

Options
Rupinder
Rupinder Partner, Neuron, Registered, Neuron 2022, Neuron 2023 Posts: 2 Neuron

Hello,

I am trying to access a csv file uploaded in a folder in dataiku managedfolder stored in DSS filesystem. I want to access this csv dataset in my API designer Python Function (Exposing python function as an API). Its throwing an error when I run following code:

handle = dataiku.Folder("FOLDER_ID")

path = handle.get_path()

error is "No DSS URL or API key found from any location" seems like its not able to locate my folder (I tried folder name instead of folder ID as well).

Before this error, I was having different error "Default project key is not specified (no DKU_CURRENT_PROJECT_KEY in env)" that was resolved using:

os.environ["DKU_CURRENT_PROJECT_KEY"] = "PROJECT_KEY"

Has anyone dealt with such an issue.


Operating system used: Linux

Best Answer

  • Alexandru
    Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,209 Dataiker
    Answer ✓
    Options

    Hi @Rupinder
    ,

    When accessing file from a managed folder from the API Endpoint Python function you have 2 options:

    1) Provide the full information like the project, DSS URL, and API Key within your Python function and use read/write APIs for managed folder and not local filesystem access. Please see the sample here:

    https://community.dataiku.com/t5/Using-Dataiku/Read-a-file-outside-the-API-folder-from-a-DSS-API/m-p/20160/highlight/true#M7882

    This resolves your current error "No DSS URL or API key found from any location". This happens because by design API Designer functions are meant to run outside of the DSS instance.

    2) If feasible given folder size. DSS automatically bundles all files inside managed folders that are defined as working folders in API Endpoint settings

    Screenshot 2021-12-01 at 15.57.51.png
    This means you can sample Python code to read the files directly from the API Node from managed folder that are set as working folders.
    import os 

    def get_file(file_name):
    file_path_list = []
    for index in range(len(folders)):
    file_path = os.path.join(folders[index], file_name)
    try:
    with open(file_path) as f:
    print('Printing first line of {} file'.format(file_name))
    print(f.readline())
    # Do something with the file
    file_path_list.append(file_path)
    except IOError:
    print("File {} not accessible".format(file_path))

    return file_path_list

    def main(file_name):
    file_path_list = get_file(file_name)

    return file_path_list

    Let me know if you have any questions.

Answers

Setup Info
    Tags
      Help me…