Using Libraries in Python API deployed using API Designer

Options
Usersyed
Usersyed Partner, Registered Posts: 29 Partner

Hi all,


I have few Python codes which I have converted into modules by defining them under the "Python" folder present under the libraries section of my project.

These libraries are using some files from the managed folders of the project. I am accessing these folders in these modules by using
"folder = folders[0]"
This has always worked for me. But now I am getting an error.
The script that is using these modules is to deployed as an API using the API designer.

Is there a way to fix this?

Thanks


Operating system used: Windows

Answers

  • Zach
    Zach Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 153 Dataiker
    Options

    Hi @Usersyed
    ,

    Could you please verify that all of the managed folders you're trying to access are set as working folders for the Python endpoint? You can add working folders under the endpoint settings:

    AE9F90F8-EA7B-4F93-B259-9EFE290E02AE_1_201_a.jpeg

    You'll need to redeploy the service after adding any folders.

    If this doesn't solve your issue, could you please provide the following information?

    • The full error message that you're getting
    • Your Python endpoint code, as well as the code from the module that contains the folder = folders[0] line.

    Thanks,

    Zach

  • Usersyed
    Usersyed Partner, Registered Posts: 29 Partner
    Options

    Hey @ZachM


    Yes, I had already added all my working folders in the API settings. But still it does not work.
    This is the code which I want to deploy as an API.

    import pandas as pdfrom my_custom_code import function_1def main_api_function(some paramters):results = function_1(some_other_parameters)return results

    This is the code which is being used as a module in the main code mentioned above
    I have put this code under the "Libraries" section of my project.
    I am getting an error in the line containing the folder path.

    The error is as follows
    Dev server deployment FAILED
    Failed to initiate function server : "<class 'NameError'>" : name 'folders' is not defined

    svc: my_api_service_name: in ep:my_api_service_name: Service my_api_service_name is not active

    import pandas as pdfolder_path = folders[0]def function_1(some paramters):# some computationsreturn something

    The surprising part is that all of this works fine in Jupyter Notebooks and recipe.
    Only while deploying the code as an API, I am getting such errors.

    Apart from the folders[0], I did try things like the one mentioned below, but none of it is working

    folder_path = dataiku.Folder("projectID.folderID").get_path()
    folder_path = dataiku.Folder("folder_name").get_path()

  • Zach
    Zach Dataiker, Dataiku DSS Core Designer, Dataiku DSS Adv Designer, Registered Posts: 153 Dataiker
    Options

    Hi @Usersyed
    ,

    Thank you for the additional information.

    The reason that you're getting this error is that the folders variable is only available from the endpoint code. It isn't accessible from modules.

    You can make it work by passing the folder_path as an argument to function_1, like this:

    Endpoint:

    import pandas as pdfrom my_custom_code import function_1folder_path = folders[0]def main_api_function(some_paramters):results = function_1(folder_path, some_other_parameters)return results

    my_custom_code module:

    import pandas as pddef function_1(folder_path, some_paramters):# do something with folder_path# some computationsreturn something

    Thanks,

    Zach

Setup Info
    Tags
      Help me…