Indexing of tables in dataiku

akuma136
akuma136 Registered Posts: 2 ✭✭✭✭

My dataiku recipes are taking too long to run because of the absence of indexes in the parent datasets. How do i add indexing in the output data so that the next recipe in the flow can take advantage of the indexing. I need help urgently. Thanks in advance

Tagged:

Answers

  • Liev
    Liev Dataiker Alumni Posts: 176 ✭✭✭✭✭✭✭✭
    edited July 18

    Hi akuma,

    Should you need indexes on your tables, go under the dataset in question and follow these steps:

    Go to Settings > Advanced. There you will find pre and post write statements. As an example in the pre-write statement you could drop the index and post-write create it again. This needs to follow the specific SQL syntax of your database.

    So for postgres it would look like:

    Pre-write statements:


    DROP INDEX IF EXISTS some_index;

    Post-write statements:


    CREATE INDEX some_index ON "${projectKey}_table_name"(column);

  • akuma136
    akuma136 Registered Posts: 2 ✭✭✭✭
    Thank you. I will try using this.
Setup Info
    Tags
      Help me…