Understanding if Python code is running in a recipe, scenario, web application, etc
Hi,
We have made our own Dataiku Python package `dku` to standardize common code sequences into functions.
A simple example:
from dku import dku from sf import sf from dataiku.core.sql import SQLExecutor2 vars = dku.custom_variables db = vars['snowflake_db'] schema = vars['snowflake_schema'] source_table = f"{db}.{schema}.mysourcetable" with sf.sql('agg') as s: s.keep_remote = True s.text = f"""select part_name, count(distinct id) as id_cnt from {source_table} group by part_name""" s.submit() post_write_statements = [ f"grant select on table {db}.{schema}.myoutputtable to role ROLE1;", f"grant select on table {db}.{schema}.myoutputtable to role ROLE2;" ] dku.set_post_write_statements('myoutputtable', statements=post_write_statements) # Write recipe outputs myoutputtable= dataiku.Dataset("myoutputtable") SQLExecutor2.exec_recipe_fragment(output_dataset=myoutputtable, query=f"select * from {sf.sqls('agg').result_table_path}", overwrite_output_schema=True)
Is there a way to detect if the code is running in a Python recipe, in a scenario, in a notebook, etc. I am trying to make the module intelligent enough to so it can expose/hide code functions as appropriate.
Eventually, the API should look something like this:
if in_recipe: ... elif in_notebook: ... etc
thx
Operating system used: Windows 10
Answers
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 1,990 Neuron
Very interesting question, tagging myself. I suspect Dataiku knows but this might not be available to us users...
-
@Turribeach
Really, this would be a great API to expose to users. I bet I can hack some methods by using the try-except pattern, but would prefer a dataiku approved solution.import dataiku if dataiku.in_python_recipe: ... elif dataiku.in_notebook: etc
-
Hi,
There is indeed no such thing directly exposed to the user. If you think that it can be a useful feature, feel free to write about it in the Product Ideas section.
Best,
Harizo