Database

swapnilnavale63

Hi,

I need to check how frequently my data is updating at database ,Is there any way to check at Dataiku ?

ben_p

Hi @swapnilnavale63
and welcome!

You could use a dataset check to count the number of rows in your database and then log this - what is the expected frequency of changes to the database and are you wanting to trigger some action when it updates?

Ben

Andrey

Hi,

Are you interested in some specific table/dataset? On the Dataiku side we track every time your dataset is built, one way to access this information is through the python API, for example:

import dataiku, dataikuapi
ds = dataiku.Dataset('mydataset')
build_history = ds.get_metric_history('Build date')['values']

Another way is to have an additional step that would log information after each time a table was written to.

You can achieve it by creating a table for logging and writing into it from a "Post-write statement" available in the Dataset -> Settings -> Advanced.

 

A third option is similar to the second but on the database side, you can create a database trigger on a table you're interested in and log information each time this table is modified, for example in Postgres:

https://www.postgresql.org/docs/9.1/sql-createtrigger.html

Regards

swapnilnavale63

Thank you for your detailed solution !