Update redshift table column with dataiku project variable.
Hi
I have one requirement where I want to update a column in redshift table with current date/time for every time a scenario is triggered in dataiku.
Can you please help me with code/explanation on how I can achieve that.
Best Regards,
Ankur.
Best Answer
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,226 Dataiker
Hi @Ankur30
,I think for this use case you use add reporters in a scenario to write to your table. This will automatically write the timestamp of when the scenario is completed.
1) Create or use an existing dataset with a timestamp column and any other columns you wish in this example I added outcome and scenario_name by updating the schema.
2) In Scenario under reporters configured correctly notably the timestamp column need to map to a timestamp column name in the dataset. The data needs to be defined in the format
{"column_name":"$variable_name"}
In the below example :
{"scenario_name":"${scenarioName}","outcome":"${outcome}"}
3) After Refreshing my sample I can see the last columns were added.
You can use a filesystem-like dataset and then sync to Redshift or use a redshift table directly.
Let me if this works for you and if this was what you were looking for.