Survey banner
Switching to Dataiku - a new area to help users who are transitioning from other tools and diving into Dataiku! CHECK IT OUT

How to use custom DataSet plugin for data deletion

Level 3
How to use custom DataSet plugin for data deletion


  I have a custom dataset plugin to read data from a data store. I would like to know what is the best way to use the same custom plugin to support data deletion.

  I am re-using the existing custom plugin user interface to allow user input for for deletion. I am using the generate_rows method in Connector class for delete operation. But I am not happy with this approach.

Would anyone suggest a better way to support data deletion in a custom dataset plugin.

  Thanks, and below is my pseudo code:

def generate_rows(self, dataset_schema=None, dataset_partitioning=None,
partition_id=None, records_limit = -1):
The main reading method.

Returns a generator over the rows of the dataset (or partition)
Each yielded row must be a dictionary, indexed by column name.

The dataset schema and partitioning are given for information purpose.
operation_type = self.config.get('operation_type', '')
if operation_type == 'read':

elif operation_type == 'delete':
    return None
    raise ValueError('Invalid operation type.')



0 Replies