How to use custom DataSet plugin for data deletion
I have a custom dataset plugin to read data from a data store. I would like to know what is the best way to use the same custom plugin to support data deletion.
I am re-using the existing custom plugin user interface to allow user input for for deletion. I am using the generate_rows method in Connector class for delete operation. But I am not happy with this approach.
Would anyone suggest a better way to support data deletion in a custom dataset plugin.
Thanks, and below is my pseudo code:
def generate_rows(self, dataset_schema=None, dataset_partitioning=None, partition_id=None, records_limit = -1): """ The main reading method.
Returns a generator over the rows of the dataset (or partition) Each yielded row must be a dictionary, indexed by column name.
The dataset schema and partitioning are given for information purpose. operation_type = self.config.get('operation_type', '') if operation_type == 'read':