-
Using date in DataIKU
Hi, Despite going through documentation multiple times, I still don't really understand how dates work in DSS. I'm importing dataset from a connection. Without turning on any of the options in Date & Time handling, this is how data looks like: It says that the data type is string, while in the database itself it is, in…
-
How to prevent DSS replace NA with null?
Hi, I'm using Python recipe to query and insert data to the output SQL Server dataset as below. import dataikuimport pandas as pdfrom dataiku import SQLExecutor2# Read recipe inputsp787PDMItem = dataiku.Dataset("_P787PDMItem_src")p787PDMItem_df = p787PDMItem.get_dataframe()# Initialize an empty DataFrame to collect all…
-
How to ingnore the Identity Column of SQL Server table in Dataiku
I have the SQL Server datasets as below TableSource has 2 columns [Name] string, [Address] string from connection 1 TableDest has 3 columns [Id] int identity, [Name], [Address] from connection 2 How to import data from TableSource to TableDest on Dataiku? Note: the connection1 and connection2 are on different SQL servers.
-
Using Script Scenario
The Webhook URL of a MS Teams channel can be used in the 'Reporter' to send scenario related messages to that Teams channel. However, I would like to know whether or not it possible to send messages to the same MS Teams channel programmatically using Dataiku Scenario API. I simply want to use two separate channels (mail…
-
Exclude specific dataset build from scenario
Hello, I need to exclude a part of my flow from being built during the execution of a scenario, because this part is to built only once in a while. For now, I have to do it in "Build only these items" mode and list all the datasets I actually want to build. Would it be a way to exclude the few datasets I DON'T want to…
-
starting a project with pyhon script that generate data, is it possible ?
Hello everyone, I'm new to dataiku. I developped a python script that is collecting data from differentes (web scrapping, local files...) sources before generating a pandas dataframe, then I performe my analysis on it. I would like to switch this project into dataiku. BUT, when I start a project, I need a dataset whereas I…
-
Dataset settings - userModified
Hello, When we look into jsons with configuration of each dataset, there's this setting "userModified" under "schema" section. Does anybody know if it's just a flag indicating whether user modified the schema or is it used for anything else? In one of our datasets, it was set to "true" by Dataiku, but after merging two…
-
Relationship between Dataiku partitioning vs S3 partitioning
Hi All, Suppose I have a dataset in S3 partitioned and stored as parquet. Suppose I read it into Dataiku as a partitioned dataset (or try to sync it from somewhere else). What is the relationship between the two partitions? If I use the same partition key in Dataiku as S3, will Dataiku recognize that and avoid…
-
DSS not finding python.exe with conda
Hi, I am trying to setup DSS 12 to use conda (python3.9) on windows. I included conda to my path. However, when setting a new code environment Managed by DSS using conda, the environment creation fails after initialisation and package download with the following error: "<path_to_env>\Scripts\python.exe": CreateProcess…
-
Custom scenario on variable change
Hello, my trigger does not launch with this trigger even though variables VAR_A and VAR_B are set to 1 : from dataiku.scenario import Trigger import dataiku PROJECT_ID = "RCU_V2" client = dataiku.api_client() project_handle = client.get_project(PROJECT_ID) variables = project_handle.get_variables() print(variables) t =…
