Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Hello,
is there a way to check and validate data?
I have webshop traffic data in my spreadsheet on a daily basis. These are divided into our different channels like SEA, Price Search Engines, SEO and so on.
I'm looking for a way to check if there are major discrepancies in new data compared to previous ones.
In this way I would like to check whether we have tracking and/or transmission errors and then send a message to the responsible employee if necessary.
Maybe someone has a good idea how I can implement this.
Thanks and Greetings
Sascha
Hey @SaschaS ,
As a start you could try looking into the following:
1. Creating a scenario with specific data checks:Concept: Metrics & Checks — Dataiku Knowledge Base, and then send a email or use a webhook to send an alert (Concept: Scenarios — Dataiku Knowledge Base). In your scenario as a first step you could run the checks and only if the checks are approved continue to run the remainder of the flow or else send a message to users warning on a data change.
2. You could also take a look at the Dataiku Data Drift functionalities: Input Data Drift — Dataiku DSS 11.0 documentation
Hope this helps!
Best,
Kathy