Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
I'm using a SQL recipe that refers to a project variable. I also have several metrics on the output table which are auto-computed after every build. What I'd like to do is to access the value of the project variable every time I build the dataset, so that I can later see how the measured metric values relate to the value of the project variable. I've tried writing my own SQL probe like so:
SELECT ${analysis_timespan_months} AS timespan FROM ${DKU_DATASET_TABLE_NAME}
but this gives me an error:
java.lang.Exception : Query failed: Incorrect syntax near '{'.
Is it possible to retrieve the project variable from a metric?
Hi,
only a handful of variables, related to the dataset being probed, are available to SQL probes. Project or instance variables are not available.
You can do a python probe though (here to retrieve the value of 'a'):
import dataiku def process(dataset, partition_id): p = dataiku.Project(dataiku.default_project_key()) return {'a' : p.get_variables().get('standard', {}).get('a', None)}
Note that get_variables() returns a dict with the "standard" and "local" variables of the project
Regards,
Frederic
Hi,
only a handful of variables, related to the dataset being probed, are available to SQL probes. Project or instance variables are not available.
You can do a python probe though (here to retrieve the value of 'a'):
import dataiku def process(dataset, partition_id): p = dataiku.Project(dataiku.default_project_key()) return {'a' : p.get_variables().get('standard', {}).get('a', None)}
Note that get_variables() returns a dict with the "standard" and "local" variables of the project
Regards,
Frederic
Hi Frederic,
Thanks very much for your reply. I was so focused on SQL code that I forgot about the existence of Python probes for a second.
When running your code, I'm getting an error for the get_variables() method: Project instance has no attribute 'get_variables'. This seems to be because I need to use the DSSProject class obtained from the DSSClient.get_project method rather than the Project class obtained from the dataiku.Project function. My working probe looks like this:
import dataiku
def process(dataset, partition_id):
client = dataiku.api_client()
p = client.get_project(dataiku.default_project_key())
return {'analysis_timespan_months' : p.get_variables().get('standard', {}).get('analysis_timespan_months', None)}
I'm happy to learn about the default_project_key() function, which was new to me. I can't seem to find any documentation for it, may I suggest that you add this function to your documentation?