Retrieve the value of a project variable as a metric

antonstam

I'm using a SQL recipe that refers to a project variable. I also have several metrics on the output table which are auto-computed after every build. What I'd like to do is to access the value of the project variable every time I build the dataset, so that I can later see how the measured metric values relate to the value of the project variable. I've tried writing my own SQL probe like so:

SELECT ${analysis_timespan_months} AS timespan FROM ${DKU_DATASET_TABLE_NAME}

but this gives me an error:

java.lang.Exception : Query failed: Incorrect syntax near '{'.

Is it possible to retrieve the project variable from a metric?

fchataigner2

Hi,

only a handful of variables, related to the dataset being probed, are available to SQL probes. Project or instance variables are not available.

You can do a python probe though (here to retrieve the value of 'a'):

import dataiku
def process(dataset, partition_id):
    p = dataiku.Project(dataiku.default_project_key())
    return {'a' : p.get_variables().get('standard', {}).get('a', None)}
 

Note that get_variables() returns a dict with the "standard" and "local" variables of the project

Regards,

Frederic

antonstam

Hi Frederic,

Thanks very much for your reply. I was so focused on SQL code that I forgot about the existence of Python probes for a second.

When running your code, I'm getting an error for the get_variables() method: Project instance has no attribute 'get_variables'. This seems to be because I need to use the DSSProject class obtained from the DSSClient.get_project method rather than the Project class obtained from the dataiku.Project function. My working probe looks like this:

import dataiku
def process(dataset, partition_id):
    client = dataiku.api_client()
    p = client.get_project(dataiku.default_project_key())
    return {'analysis_timespan_months' : p.get_variables().get('standard', {}).get('analysis_timespan_months', None)}

I'm happy to learn about the default_project_key() function, which was new to me. I can't seem to find any documentation for it, may I suggest that you add this function to your documentation?