Data validation compared to previous data

SaschaS

Hello,

is there a way to check and validate data?
I have webshop traffic data in my spreadsheet on a daily basis. These are divided into our different channels like SEA, Price Search Engines, SEO and so on.
I'm looking for a way to check if there are major discrepancies in new data compared to previous ones.
In this way I would like to check whether we have tracking and/or transmission errors and then send a message to the responsible employee if necessary.
Maybe someone has a good idea how I can implement this.

Thanks and Greetings
Sascha

kathyqingyuxu

Hey @SaschaS
,

As a start you could try looking into the following:

1. Creating a scenario with specific data checks:Concept: Metrics & Checks — Dataiku Knowledge Base, and then send a email or use a webhook to send an alert (Concept: Scenarios — Dataiku Knowledge Base). In your scenario as a first step you could run the checks and only if the checks are approved continue to run the remainder of the flow or else send a message to users warning on a data change.

2. You could also take a look at the Dataiku Data Drift functionalities: Input Data Drift — Dataiku DSS 11.0 documentation

Hope this helps!

Best,

Kathy

Xatorisan

What’s your take on balancing automation with manual oversight in data validation? Can't wait to hear your thoughts!