-
How to use "Execute Python unit test" scenario step
A new scenario step was added in a recent DSS version which is to execute a Python unit test. I'd like to start using this. However, the documentation is pretty brief: "this step executes one or more Python pytest tests from a project’s Libraries folder using a Pytest selector". Anyone have more details on or an example of…
-
Filter by comparing two columns
I have a dataset with two columns, say A and B. I would like to only keep the rows where the content of A == content of B. In Pandas, this is expressed as: ```df = df[df['A'] == df['B']]``` Is this possible using Dataiku recipes? I tried with sample/filter recipe but could not figure out how to accomplish this. As a note,…
-
Does anyone know how to add a RecordID column in dataiku. I want to add serial number to each row.
-
Group by with empty value and with Null value
Hello everyone ! I have a dataset with empty values in one of the columns (col1) and I use a group by recipe on an other column (col2) without empty values with col1_distinct as aggregation. I get a volume of 21, 199 and 1608 for the 3 col2 fields. But I wanted to add a condition on col1 with a prepare recipe with a…
-
Dynamic Column
I have an excel input file. Col A to Col T till row T26, I have data where Col T have latest month data. Col V to Col AO have second set of data till AO50. Now, its dynamic data, every month, a new column is added. For example, for next month, Col u will have Dec month's data. same with second set of data, I will be…
-
how can I make a Django app in Dataiku ?
I'm looking for information regarding coding a Django application in Dataiku. Any information on how to achieve this ? thanks
-
Custom trigger to run itself?
Hi everyone, I want to execute scenario again if its fail, so it can try 3 times, sometimes kubernetes fails or, spark fails could be fixed after run again so, i dont miss time range between fail and fix manually. This is the code created with LLM, it used python for that. I changed project name to variable "project name",…
-
Custom trigger does not executes python code
Hi, I am quite new in Dataiku and I am interested why the following code does not work as expected. Namely, I am trying to define custom trigger that will check if the folder is empty. Both from dataiku.scenario import Trigger t=Trigger() folder = dataiku.Folder("folder_id") files = folder.list_paths_in_partition() if…
-
schema propagation problem
In the dataset explore, I can define 'description' using 'edit column schema'. And I can propagate the schema to the following 'flow' using 'schema propagation'. However, sometimes it is not inherited (if there is a 'prepare recipe' in the middle) I want to know how to inherit it normally.
-
How to set up a random forest regression?