-
Selecting the latest file added in a folder using Python
Hi All, I would appreciate it if someone can provide me with a python script to select (read_csv) the latest csv file in a SFTP folder? Currently I am using the following script to read csv files from a SFTP folder, however when we have multiple files added in different dates, I would like to select only the latest one…
-
Scenario Reporters
Currently using Scenario reporters to send data to a dataset with below configuration. { "flowname": "${scenarioName}", "status": "${outcome}", "summary": "${failedEventsSummary}" } The issue is failedEventsSummary is providing too much text. How can we get just the ERROR on why the scenario failed. Operating system used:…
-
How to upload TDE (tableau) file to DSS?
Hi, I want to upload a TDE file from tableau server to DSS. How should i go about it? Narayan
-
IF contains multiple conditions
colAB310 if(contains(toUppercase(col),"A"),"letters",if(contains(toUppercase(col),"B"),"letters","others")) The code above works but is it possible to shorten it to combine the logic for "A" and "B"? For example in the syntax below (albeit incorrect): if(contains(toUppercase(col),"A" or "B"),"letters","others")
-
How to establish connection to Cosmos DB in Dataiku
Could you please provide guidance on how to establish a connection between Dataiku and Cosmos DB for data integration and analysis purposes
-
"IF contains" on file path
* File_Path: \USER\FOLDER_A\ABC\FILE_A * Formula: if(contains(toUppercase(File_Path),"\ABC\"), "yes", "no") * This returns "Invalid Formula" message: Unexpected 'yes' (Paring error at offset ) I tested the formula using "ABC" instead "\ABC\" and it works as intended, however for my project retaining the slash symbols…
-
NLP Finding
I have to read the logs and find where the error has occurred so the item is marked as exception using NLP. can anyone help on this please.
-
Functionality questions on DataIKU
* Can Spark be configured for ML algorithms; it looks like current processing is in memory? * Is there Spark processing option available for K means clustering and PCA linear regression? * Is Light GBM available with Spark? * Is automated Hyperparameter tuning available in Dataiku? * Schema visibility: * Current DataIKU…
-
Sync-recipe to Snowflake
I have a flow that gets data from two Snowflake sources, then Python recipe checks the difference of the max(date columns) of both and extracts the rows that are missing from the other dataset. I first tried Syncing that back to snowflake (like updating the other source set with appending the missing rows) but encountered…
-
Regex function to return string between 2 characters
I'm trying to create a regex function that gives me the string between 2 characters I have the string below word1_word2_word3_word4_word5_word6_word7_word8_length_string.txt and I'm trying to return everything after the 7th instance of "_" and before ".txt" Desired output: word8_length_string is there a way to use a regex…
