-
SQL Compilation for Date-Partitioned Datasets for recipes that require all Partitions
Hi, this is my first time working with partitions so please let me know if I'm making a mistake. I have partitioned various datasets by a date column (set partition timeframe to DAY) and have set things up so that my scheduled scenarios for initial recipes are only doing transformations on the latest few days worth of data…
-
regex that will remove everything between begin: and end:
Hi I've got dataset with regex patterns in one column and python code that uses it to make replacement. I need regex pattern that will remove everything between "begin:" and "end:" Tried to use "begin:[\s\S]*?end: " but it doesn't work. in between I can have more then 3000 chars including special ones. Thank you Ela…
-
Not seeing partitions list in the explore dataset tab
Does anyone know why partitions that are being extracted still don't appear in the partition list under the Explore tab of the dataset? Operating system used: windows
-
how to get all runs of scenarios an their informations?
For my internship i have a project where i have to create a dataset with all the scenarios and their runs, but when i try and import them the informations such as the last run, the progress percentage, number of runs type of triggers and stuff like that remains empty or are marked with a 0 . I hve the id and the name of…
-
Need help with efficient BigQuery to Snowflake data transfer in Dataiku
Hi there! I'm setting up a pipeline to move data from BigQuery to Snowflake and really need some advice from folks who've done this before. Right now I'm using the standard DSS engine but it's painfully slow for larger datasets - takes forever to process even moderate amounts of data. I'm not sure what route is actually…
-
how can I use the LLM api hosted in dataiku?
we have some models running on dataiku. But currently I can only find the python client. Even I get the rest apis, the response from these apis are so different from the officials api, like sonnet. This is not good if I want to integrate the api to other tools. Like, I would like to initegrate the API to vscode plugin or…
-
how to display a running.. while running a scenario by using webapp
how to display a running.. while running a scenario by using webapp and also how to refresh that webapp page after scenario run?
-
ModuleNotFoundError When Accessing Knowledge Bank via REST API SDK
Hi everyone, I'm running into an issue while trying to access a Dataiku Knowledge Bank from an external Python script using the Dataiku REST API client. When I execute my script, I get the following error:ModuleNotFoundError: No module named 'dataiku.core.vector_stores.lifecycle' Here are my environment details: Dataiku…
-
Trouble creating custom Python code environments
I am trying to create a new Python Code Env in my locally installed instance of Dataiku. The only non-conda based option is Python 3.7. All the other options (Python 3.5, 3.6, 3.8, 3.9, 3.10, 3.11) are listed as "Not available" in the dropdown. I have Python 3.10.11 locally installed, but somehow Dataiku is not recognizing…
-
How to prevent users from accessing filesystem files?
I am installing Dataiku on a baremetal ubuntu 22 OS. However I noticed that users can access the root filesystem quite easily by managed folders or jupyter notebooks for example. Is there a way to completely prevent users from accessing any filesystem related files? If so what is the most efficient way to do so? Operating…
