Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Hi,
We have made our own Dataiku Python package `dku` to standardize common code sequences into functions.
A simple example:
from dku import dku
from sf import sf
from dataiku.core.sql import SQLExecutor2
vars = dku.custom_variables
db = vars['snowflake_db']
schema = vars['snowflake_schema']
source_table = f"{db}.{schema}.mysourcetable"
with sf.sql('agg') as s:
s.keep_remote = True
s.text = f"""select part_name, count(distinct id) as id_cnt
from {source_table}
group by part_name"""
s.submit()
post_write_statements = [
f"grant select on table {db}.{schema}.myoutputtable to role ROLE1;",
f"grant select on table {db}.{schema}.myoutputtable to role ROLE2;"
]
dku.set_post_write_statements('myoutputtable',
statements=post_write_statements)
# Write recipe outputs
myoutputtable= dataiku.Dataset("myoutputtable")
SQLExecutor2.exec_recipe_fragment(output_dataset=myoutputtable,
query=f"select * from {sf.sqls('agg').result_table_path}",
overwrite_output_schema=True)
Is there a way to detect if the code is running in a Python recipe, in a scenario, in a notebook, etc. I am trying to make the module intelligent enough to so it can expose/hide code functions as appropriate.
Eventually, the API should look something like this:
if in_recipe:
...
elif in_notebook:
...
etc
thx
Operating system used: Windows 10
Very interesting question, tagging myself. I suspect Dataiku knows but this might not be available to us users...
@Turribeach Really, this would be a great API to expose to users. I bet I can hack some methods by using the try-except pattern, but would prefer a dataiku approved solution.
import dataiku
if dataiku.in_python_recipe:
...
elif dataiku.in_notebook:
etc
Hi,
There is indeed no such thing directly exposed to the user. If you think that it can be a useful feature, feel free to write about it in the Product Ideas section.
Best,
Harizo