-
How to create dynamic regex function
I have a column called filename and the field is formatted as name1_name2_name3_name4_name5_YYYYMMDDHHMMSS.txt.pgp name1_name2_name3_name4_YYYYMMDDHHMMSS.txt.pgp name1_name2_name3_name4_name5_name6_YYYYMMDDHHMMSS.txt.pgp up until now, the number of names can differ, but the filename always ends with YYYYMMDDHHMMSS.txt.pgp…
-
Using a Dataset's Column in Email Body
Hi all, simple question here How I can add a dataset's column in my mail body ${column_name} doesn't work. I thought I could add as a local variable but I don't know about the configs. Coould you please help out? Thanks.
-
Percentage of individual values based on total value
I have a dataset with two columns representing individual sales values and another column representing the month. I want to create an additional column that calculates the percentage of individual sales values relative to the total sales values for each month.
-
BigQuery connection
Hello, Dataiku Team. If I want to deploy a project on Automation node, but my project has a connection with BigQuery, I need to create the same connection for Automation node, right? The thing is that I do not have the same options in the automation node as design node. In design node I have set a section called Path…
-
Requesting snowflake with python
Hello, I am working on web app designer, I want to create a tile which lists the tables of my schemas of my snowflake db.I have to do it in python but I don't know how to do it.Since I am already connecting to snowflake from dataiku (I can add a dataset to the flow) I tell myself that in my script I no longer need to put…
-
Using timezone naive datetime
Hi all! Is there way to actually prevent Dataiku from converting the datetime to a timezone-aware format? Whenever I do this, it automatically saves the date as UTC. Thanks!
-
warning using sentence-transformers/all-MiniLM-L6-v2 for embedding
hi, we are using huggingface model that does not required API, already downloaded hugging face model in resources by using this code model_name_fifteen = 'sentence-transformers/all-MiniLM-L6-v2' MODEL_REVISION_FIFTEEN = '8b3219a92973c328a8e22fadcfa821b5dc75636a' tokenizer = AutoTokenizer.from_pretrained(model_name_fifteen,…
-
SCENARIO : get the name of dataset that failed in a step
Hello, am currently trying to recover the name of the datasets which failed in a build step of a scenario. So far I have only found the way to recover the first step which fails, however I specifically want to know the name of all the dataSets which fail in order to transfer them to a separate dataset. Do you have any…
-
Problem with Spark enigine
Hello, when we read hive tables with engine spark version 2.4.7.7.1.7.2038-1 on CDP version 7.1.7 we have a problem with type date, there's a shift forward of 10 min and 4 s as it is showns in the attached file. Could you help us? We have already tried to add: spark.sql.legacy.parquet.int96RebaseModeInWrite --> CORRECTED…
-
Handling 0 in Denominator Column for Month-over-Month Change
I'm trying to calculate the month-over-month percentage change in Dataiku using the following formula ((sales/ sales_lag) - 1) * 100 Where: * sales is the numerator column * sales_lag is the denominator column containing the lagged (previous month's) values However, the sales_lag column can have null or zero values, which…