How do I get the model to automatic re-train after the dataset is updated

Options
kid4869
kid4869 Registered Posts: 1

When the dataset is updated (including when the input schema also changes), how do I get the model to automatically adapt to this change and learn automatically ?

For now, what's happening is that since I also have tools like VIF to pre-filter the columns, when the data is updated, the columns that are filtered out will change.

But if this column changes, then when I use the model task that has been trained before, I get an error because it requires exactly the same information about the column as before.

Is there any way to adapt the model to this change for automation purposes?

Now I have to re-train the model in the original analysis after the data is updated, select the model and deploy it later to use the model's

Answers

  • Alexandru
    Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,209 Dataiker
    Options

    Hi @kid4869
    ,

    If the columns change especially but not the target, you need to detect the new schema of the datasets in the modeling tas using code e.g mltask.wait_guess_complete()
    See full example here:
    https://developer.dataiku.com/latest/concepts-and-examples/ml.html
    Once that is done, you can retrain your model with the new columns features.

    Note if the larger column changes you will need to create a new visual analysis entirely, retrain and deploy to the flow.

    Kind Regards,


Setup Info
    Tags
      Help me…