-
MS SQL SERVER CONNECTION
Hello Everyone, I created a connection with my Azure SQL DB using the MS SQL Server connector, the connection went well, but when I clicked on get table list, I got the following error message: Oops: an unexpected error occurred The connection is closed. Please see our options for getting help HTTP code: 500, type:…
-
column Index
I have a dataset which has Jan, Jan_1, Feb, Feb_1... I was to use Column index to pick the last column. Can you help using Column index without Python?
-
Access to webapp back-ends from outside of DSS (with Postman)
I found this article but i have some questions, hope someone can help me. I created a standard webapp with python server, im trying to access the endpoints from postman. I'm sending my project apikey as suggested by documentation but I'm receving this message: <body> <h1 style="margin-top: 50px">HTTP error 401 (Not…
-
Seeking Optimization Tips for DSS Flow and Spark Configuration
hello everyone, I am currently working on optimizing my DSS flow. I have a scenario that currently takes 20 minutes to execute, and I am looking to reduce this time to just 5 minutes. I would greatly appreciate any tips or strategies for optimization. Additionally, I am interested in understanding how to configure Spark…
-
Spark Configuration for optimization resource allocation
Hello, I am interested in understanding how to configure Spark settings to ensure optimal resource allocation. Specifically, I am looking for guidance on configuring parameters like spark.driver.cores, spark.dynamicAllocation.initialExecutors, spark.executor.cores, spark.dynamicAllocation.enabled, spark.executor.instances,…
-
pairwise distance for 2 geopoints
Is there any tool available to calculate pairwise distance. I have 2 different geo points available in dataframe .
-
Dynamic Column
I have an excel input file. Col A to Col T till row T26, I have data where Col T have latest month data. Col V to Col AO have second set of data till AO50. Now, its dynamic data, every month, a new column is added. For example, for next month, Col u will have Dec month's data. same with second set of data, I will be…
-
Delete records based on multiple JOINs
Newbie here. Trying to convert a SQL from HIVE that pulls records partly based on several JOIN conditions but limits those record based on other JOIN conditions. In SQL it is a "WHERE NOT EXISTS" condition. The following is the code - SELECT x FROM y, z Multiple left joins… (and) WHERE NOT EXISTS ( SELECT 1 FROM…
-
Avoid python recipe to change data types
Hello everyone, I would like to prevent python from inferring the data type of my dataframe during a python recipe . For example, I would like an id column to remain in string type rather than dataiku converting it to float. I could, for example, convert each of my columns manually, but this is tedious for datasets with…
-
Group by with empty value and with Null value
Hello everyone ! I have a dataset with empty values in one of the columns (col1) and I use a group by recipe on an other column (col2) without empty values with col1_distinct as aggregation. I get a volume of 21, 199 and 1608 for the 3 col2 fields. But I wanted to add a condition on col1 with a prepare recipe with a…
