How to change the underlying table in a dataset?

pnkj
pnkj Registered Posts: 10 ✭✭

I am using this code to change the table associated with a dataset. The table is getting changed as I verified from the dataset settings, but I am getting this error 'Column name not matching between query and schema'. But when I click on Test Table in connection settings, the dataset is getting updated. I am using this code

import dataiku
client = dataiku.api_client()
project = client.get_project(dataiku.default_project_key())
dataset = project.get_dataset("dataset_name")
settings = dataset.get_settings()
settings.set_table(connection, schema, filename, catalog=None)
settings.save() ds = dataiku.Dataset("ds")
ds_df = dma_raw.get_dataframe()

After this when I am reading the dataset using it throws an error -

Exception: Reading dataset failed: b'Column name not matching between query and schema. Please check dataset schema.'

Answers

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,088 Neuron
    edited November 15

    If the tables are different then you need to update the dataset schema as well. Try running an autodetect after saving the settings:

    new_settings = dataset.autodetect_settings()
    new_settings.save()
    

Setup Info
    Tags
      Help me…