Hi,
I need to check how frequently my data is updating at database ,Is there any way to check at Dataiku ?
Hi @swapnilnavale63 and welcome!
You could use a dataset check to count the number of rows in your database and then log this - what is the expected frequency of changes to the database and are you wanting to trigger some action when it updates?
Ben
Hi,
Are you interested in some specific table/dataset? On the Dataiku side we track every time your dataset is built, one way to access this information is through the python API, for example:
import dataiku, dataikuapi
ds = dataiku.Dataset('mydataset')
build_history = ds.get_metric_history('Build date')['values']
Another way is to have an additional step that would log information after each time a table was written to.
You can achieve it by creating a table for logging and writing into it from a "" available in the Dataset -> Settings -> Advanced.
A third option is similar to the second but on the database side, you can create a database trigger on a table you're interested in and log information each time this table is modified, for example in Postgres:
https://www.postgresql.org/docs/9.1/sql-createtrigger.html
Regards
Thank you for your detailed solution !