-
min between 2 dates
Hello, I'm trying to find the 1st date between 2 date fields. I was thinking of using a min formula but there may be missing values in these fields and the formula doesn't seem to work in this case. Is there another solution other than an "if then" formula ?
-
Recursive query in Dataiku
Hello all, Is there anyway to use the recursive query in Dataiku under DSS engine? Thanks all
-
Is it possible to write multiple datasources at a time using python?
I have a data processing task that requires python. Specifically, I'm reading data from a proprietary file format, then writing the extracted data out to the database. I want to split this data into multiple datasources, one for training data, one for holdouts, and one for bad data (so I can analyze corrupted data from the…
-
Question about the behaviour of the "Split and Fold" prepare processor and NULL handling
Let's say I have a table containing the following data: IDFruitOther Random Data1Apple, Pear, Cherryaksdhkajshda2NULLkasdhjkasjhkas3Watermelonajshdgjashgdjashg If i run the Split and Fold prepare step on the Fruit column, i will get the following result: IDFruitOther Random…
-
How to use regex in DSS Formula replace()?
I'm trying to use a regex in a replace() using the Formula processor in a prepare recipe. According to the documentation/docstring this should work, but I can't get it to recognize the text to replace parameter as a regex and not as literal text. I'm using a formula as below. How can I get the formula language to interpret…
-
Senario Halting when no new data is available
I've got a data pipeline that runs on scenarios. There may or may not be new data every time the scenario runs. I have been doing some inefficient items each time the scenario runs. 1. I have been guaranteeing that I get at least one old record every time the scenario runs, in order to ensure a stable schema. I have…
-
Trouble with IF logic on formula step
Getting some unexpected results with this formula step where some retired customers are showing the max of L12/2019/2020/2021/2022 when they should be showing L12. What I'm looking to do is this: - If a customer is retired (using Retired Flag = TRUE) or if they have bad debt (using Bad Debt Status =TRUE), I want to always…
-
Extract text from html stored in column
How would one extract the text and strip all the html. parseHTML() gives me just the html back, and htmlText() gives me the html as text (no brackets)
-
Concatanate in prepared recipe
Hello, it seems like the concatenate function from the prepared recipe doesn't work when one of the element is missing. Example : Col1Col2concatenated columnHPPCDHPP CDWAP when col 2 is empty the concatenated column is also empty
-
update an existing column without creating a new dataset in flow recipe.
Hi Team, What I am trying to accomplish is to run a couple of column updates in a PostgreSQL after receiving new data, however, every time I try to use a new recipe, it's trying to create a new dataset(table) in the destination DB. Is there a way to create a flow recipe that only runs updates to an existing dataset without…