-
Problem with Spark enigine
Hello, when we read hive tables with engine spark version 2.4.7.7.1.7.2038-1 on CDP version 7.1.7 we have a problem with type date, there's a shift forward of 10 min and 4 s as it is showns in the attached file. Could you help us? We have already tried to add: spark.sql.legacy.parquet.int96RebaseModeInWrite --> CORRECTED…
-
Handling 0 in Denominator Column for Month-over-Month Change
I'm trying to calculate the month-over-month percentage change in Dataiku using the following formula ((sales/ sales_lag) - 1) * 100 Where: * sales is the numerator column * sales_lag is the denominator column containing the lagged (previous month's) values However, the sales_lag column can have null or zero values, which…
-
How to stop keep formatting using prepare / formula recipe
I have string value called hours, formatted as "00" and a value called minutes formatted as "07". Im trying to use the concat function to create an HH:mm value = 00:07 but for some reason when using concat, it gets rid of trailing 0s and results in 0:7. is there a way around this? if not is there any other way I can create…
-
Do I need to create a dataset for every recipe?
I think having to create a dataset for every intermediary step in Dataiku is not very efficient, especially from a data storage standpoint. I think it's causing a lot of redundant data to be stored in the process of creating a workflow. Is there any way of combining and executing multiple recipes together or not storing…
-
Api end point for datalookup
Iam trying to create a an ape end point for data lookup .when i click deplyement policy as "reference"Iam getting test quire results correctly but when i try to acess api through postman showing error "failed to load JDBC driver.Now when i click deplyement policy as "bundled" it showing the error at test query as "Dev…
-
Dataiku Spark Remote Connection
Hello, I am using Dataiku 12.5.2 and currently running Spark 2.4. Dataiku is installed on a server named A, while Spark is installed on a server named B, configured as a standalone Spark installation without Hadoop. Both server A and server B are capable of TCP communication and allow SSH access. How can I use the Spark on…
-
How to return current date and not time
data set full of jobs and each job has a start date, so I want to create a formula that says filter out my data where my start date hasn't happened yet / less than the current date. The issue I'm having is when I create my now() function it gives me the current date and time, I only need the current date in a parsed date…
-
Renaming the output dataset of SQL Script and its impact to the script
It's common to realize there is a better naming for a dataset, or a name for a dataset to require revision after further adjustments made elsewhere in the workflow. In such an instant, I renamed the output dataset (which is a table in SQL Database) in Dataiku. The initial name was "backend" and I updated it to…
-
How to ad leading 0s
I have a column called minutes and that can range from 0 - 60, what I'm trying to do is add a leading 0 to the single digit values, ex 9 = 09, so that no matter what my value is always 2 digits long, is that possible? and if so, how would I accomplish that? Operating system used: windows
-
Dataiku dss cli
How do I access the dataiku dsscli for running a cli command on a conda environment ?
