Understanding if Python code is running in a recipe, scenario, web application, etc

Options
info-rchitect
info-rchitect Registered Posts: 169 ✭✭✭✭✭✭
edited 3:26PM in Using Dataiku

Hi,

We have made our own Dataiku Python package `dku` to standardize common code sequences into functions.

A simple example:

from dku import dku
from sf import sf
from dataiku.core.sql import SQLExecutor2

vars = dku.custom_variables

db = vars['snowflake_db']

schema = vars['snowflake_schema']

source_table = f"{db}.{schema}.mysourcetable"

with sf.sql('agg') as s:
    s.keep_remote = True
    s.text = f"""select part_name, count(distinct id) as id_cnt
from {source_table}
group by part_name"""
    s.submit()

post_write_statements = [
  f"grant select on table {db}.{schema}.myoutputtable to role ROLE1;",
  f"grant select on table {db}.{schema}.myoutputtable to role ROLE2;"
]

dku.set_post_write_statements('myoutputtable',
                              statements=post_write_statements)

# Write recipe outputs
myoutputtable= dataiku.Dataset("myoutputtable")
SQLExecutor2.exec_recipe_fragment(output_dataset=myoutputtable,
                              query=f"select * from {sf.sqls('agg').result_table_path}",
                              overwrite_output_schema=True)

Is there a way to detect if the code is running in a Python recipe, in a scenario, in a notebook, etc. I am trying to make the module intelligent enough to so it can expose/hide code functions as appropriate.

Eventually, the API should look something like this:

if in_recipe:
   ...
elif in_notebook:
   ...
etc

thx


Operating system used: Windows 10

Answers

Setup Info
    Tags
      Help me…