How to use custom DataSet plugin for data deletion

Dominic-S Registered Posts: 17


I have a custom dataset plugin to read data from a data store. I would like to know what is the best way to use the same custom plugin to support data deletion.

I am re-using the existing custom plugin user interface to allow user input for for deletion. I am using the generate_rows method in Connector class for delete operation. But I am not happy with this approach.

Would anyone suggest a better way to support data deletion in a custom dataset plugin.

Thanks, and below is my pseudo code:

def generate_rows(self, dataset_schema=None, dataset_partitioning=None,
partition_id=None, records_limit = -1):
The main reading method.

Returns a generator over the rows of the dataset (or partition)
Each yielded row must be a dictionary, indexed by column name.

The dataset schema and partitioning are given for information purpose.
operation_type = self.config.get('operation_type', '')
if operation_type == 'read':

elif operation_type == 'delete':
return None
raise ValueError('Invalid operation type.')

Setup Info
      Help me…