Database

Options
swapnilnavale63
swapnilnavale63 Registered Posts: 5 ✭✭✭

Hi,

I need to check how frequently my data is updating at database ,Is there any way to check at Dataiku ?

Best Answers

  • ben_p
    ben_p Neuron 2020, Registered, Neuron 2021, Neuron 2022, Dataiku Frontrunner Awards 2021 Participant Posts: 143 ✭✭✭✭✭✭✭
    Answer ✓
    Options

    Hi @swapnilnavale63
    and welcome!

    You could use a dataset check to count the number of rows in your database and then log this - what is the expected frequency of changes to the database and are you wanting to trigger some action when it updates?

    Ben

  • Andrey
    Andrey Dataiker Alumni Posts: 119 ✭✭✭✭✭✭✭
    edited July 17 Answer ✓
    Options

    Hi,

    Are you interested in some specific table/dataset? On the Dataiku side we track every time your dataset is built, one way to access this information is through the python API, for example:

    import dataiku, dataikuapi
    ds = dataiku.Dataset('mydataset')
    build_history = ds.get_metric_history('Build date')['values']

    Another way is to have an additional step that would log information after each time a table was written to.

    You can achieve it by creating a table for logging and writing into it from a "" available in the Dataset -> Settings -> Advanced.

    Screenshot 2020-07-22 at 11.56.07.png

    A third option is similar to the second but on the database side, you can create a database trigger on a table you're interested in and log information each time this table is modified, for example in Postgres:

    https://www.postgresql.org/docs/9.1/sql-createtrigger.html

    Regards

Answers

Setup Info
    Tags
      Help me…