Check out the first Dataiku 8 Deep Dive focusing on Productivity on October 29th Read More

Database

Level 2
Database

Hi,

I need to check how frequently my data is updating at database ,Is there any way to check at Dataiku ?

0 Kudos
3 Replies
Level 4

Hi @swapnilnavale63 and welcome!

You could use a dataset check to count the number of rows in your database and then log this - what is the expected frequency of changes to the database and are you wanting to trigger some action when it updates?

Ben

 

 

Dataiker
Dataiker

Hi,

Are you interested in some specific table/dataset? On the Dataiku side we track every time your dataset is built, one way to access this information is through the python API, for example:

 

import dataiku, dataikuapi
ds = dataiku.Dataset('mydataset')
build_history = ds.get_metric_history('Build date')['values']

 

 

Another way is to have an additional step that would log information after each time a table was written to.

You can achieve it by creating a table for logging and writing into it from a "" available in the Dataset -> Settings -> Advanced.

Screenshot 2020-07-22 at 11.56.07.png

A third option is similar to the second but on the database side, you can create a database trigger on a table you're interested in and log information each time this table is modified, for example in Postgres:

https://www.postgresql.org/docs/9.1/sql-createtrigger.html

 

Regards

 

Andrey Avtomonov
R&D Engineer @ Dataiku
Level 2
Author

Thank you for your detailed solution !

0 Kudos