Error when trying to use Dataiku Python APIs in API endpoint (API designer)

NicholasPini

I have created an API endpoint of type Python code. Inside, I want to run a scenario: the idea is that when an API request is received, a scenario run is triggered. The problem is that if I run this example code:

import dataiku

def api_py_function():
    client = dataiku.api_client()
    project = client.list_project_keys()
    return "ok"

I get this error:

Failed: Failed to run function : <class 'requests.exceptions.InvalidSchema'> : No connection adapters were found for '127.0.0.1:61553/dip/publicapi/projects/'

I searched online and I've found no solution.

Bonus question: is this the right approach for dealing with API that need to trigger a flow? I want to receive some input arguments from an outside request, which are used to run the Dataiku flow, but I may be using Dataiku wrong, but I also found very little information online on how to do this.

Turribeach

This is certainly not a desirable pattern. You don't really need to use a Dataiku API endpoint to make an API call to run a scenario in Dataiku. You should really just call the Dataiku Python API directly from where you want to integrate with and trigger the scenario load remotely. If the issue is that your integration "client" does not support Python and you want to call Dataiku using a REST API then you can just use the Dataiku REST API instead.

Bonus Answer: The Dataiku Python API is just a wrapper over the Dataiku REST API.

Bonus Answer 2: The Dataiku Developer website has lots of Python API code samples to help you getting started using the API.

NicholasPini

https://community.dataiku.com/discussion/comment/45661#Comment_45661

I see, thank you. But how does someone sends input to the scenario using Dataiku's API? As far as I can tell, Dataiku can trigger scenarios on dataset changes etc., but what if I want to use an input received by the user of the API inside my scenario/flow?

Turribeach

There are many ways to do this. Can you give more details about the use case? What’s the input and why does it come from the API?

NicholasPini

https://community.dataiku.com/discussion/comment/45663#Comment_45663

Basically, we have an output managed folder which we want to build. This folder depends on a Python code recipe which depends on an input folder which contains some TIF files. The idea is that from the API it is possible to provide some parameters, like the start date and the end date on which to do the analysis. I know Dataiku typically triggers runs on dataset changes, but wouldn't it be quite complicate to put this parameters (start date and end date, for exaple) in some dataset that the frontend needs to change in order to trigger the run? It would be easier to pass these parameters using the Dataiku API directly, if it is possible.

Turribeach

There are 3 ways that I see best suited for this use case:

1. Setting scenario parameters, see Dataiku API. Advantages: no left over parameters Disadvantages: Flow can only run via scenario call via API as parameters must be set
2. Setting project variables, see Dataiku API. Advantages: Flow can run via UI but last set variables will be used Disadvantages: left over parameters from previous run. Parameters are slightly hidden so clickers might not find them easily
3. Having another input managed folder where I would store a simple text file the start date and the end date. Then use the Dataset change trigger which can be set to watch folders. Advantages: Flow can run via UI but last set variables will be used. Easier to see parameters. Disadvantages: left over parameters from previous run.

The issue of not being able to run the scenario without passing the parameters can be bypassed by having another scenario that defines the two dates as variables and pass them in the API call spo you can use that parent scenario to call the child scenario via the API.