Retrieve the value of a project variable as a metric

antonstam Partner, Registered Posts: 25 Partner
edited July 16 in Using Dataiku

I'm using a SQL recipe that refers to a project variable. I also have several metrics on the output table which are auto-computed after every build. What I'd like to do is to access the value of the project variable every time I build the dataset, so that I can later see how the measured metric values relate to the value of the project variable. I've tried writing my own SQL probe like so:

SELECT ${analysis_timespan_months} AS timespan FROM ${DKU_DATASET_TABLE_NAME}

but this gives me an error:

java.lang.Exception : Query failed: Incorrect syntax near '{'.

Is it possible to retrieve the project variable from a metric?

Best Answer

  • fchataigner2
    fchataigner2 Dataiker Posts: 355 Dataiker
    edited July 17 Answer ✓


    only a handful of variables, related to the dataset being probed, are available to SQL probes. Project or instance variables are not available.

    You can do a python probe though (here to retrieve the value of 'a'):

    import dataiku
    def process(dataset, partition_id):
        p = dataiku.Project(dataiku.default_project_key())
        return {'a' : p.get_variables().get('standard', {}).get('a', None)}

    Note that get_variables() returns a dict with the "standard" and "local" variables of the project




  • antonstam
    antonstam Partner, Registered Posts: 25 Partner
    edited July 17

    Hi Frederic,

    Thanks very much for your reply. I was so focused on SQL code that I forgot about the existence of Python probes for a second.

    When running your code, I'm getting an error for the get_variables() method: Project instance has no attribute 'get_variables'. This seems to be because I need to use the DSSProject class obtained from the DSSClient.get_project method rather than the Project class obtained from the dataiku.Project function. My working probe looks like this:

    import dataiku
    def process(dataset, partition_id):
        client = dataiku.api_client()
        p = client.get_project(dataiku.default_project_key())
        return {'analysis_timespan_months' : p.get_variables().get('standard', {}).get('analysis_timespan_months', None)}

    I'm happy to learn about the default_project_key() function, which was new to me. I can't seem to find any documentation for it, may I suggest that you add this function to your documentation?

Setup Info
      Help me…