Survey banner
The Dataiku Community is moving to a new home! We are temporary in read only mode: LEARN MORE

How to use custom DataSet plugin for data deletion

Dominic-S
Level 3
How to use custom DataSet plugin for data deletion

Hi,

  I have a custom dataset plugin to read data from a data store. I would like to know what is the best way to use the same custom plugin to support data deletion.

  I am re-using the existing custom plugin user interface to allow user input for for deletion. I am using the generate_rows method in Connector class for delete operation. But I am not happy with this approach.

Would anyone suggest a better way to support data deletion in a custom dataset plugin.

  Thanks, and below is my pseudo code:

def generate_rows(self, dataset_schema=None, dataset_partitioning=None,
partition_id=None, records_limit = -1):
"""
The main reading method.

Returns a generator over the rows of the dataset (or partition)
Each yielded row must be a dictionary, indexed by column name.

The dataset schema and partitioning are given for information purpose.
operation_type = self.config.get('operation_type', '')
if operation_type == 'read':

    read_data(records_limit):
elif operation_type == 'delete':
    delete_data()
    return None
else:
    raise ValueError('Invalid operation type.')

 

    

0 Replies