New to Dataiku DSS? Try out our NEW Quick Start Programs today and get onboarded on the product in just one hour! Let's go

Python API equivalent to "CHECK NOW" button

JohnB
Level 3
Python API equivalent to "CHECK NOW" button

The UI provides a "CHECK NOW" button on the dataset settings, schema tab

Can this be run in code?

0 Kudos
3 Replies
arnaudde
Dataiker
Dataiker

Hello,
You can achieve the same as the dataset schema tab "CHECK NOW" button with the following code sample:

import dataiku
client = dataiku.api_client()
p = client.get_project('PROJECT_KEY')
future = p.get_dataset('DATASET_NAME').test_and_detect()
future.wait_for_result()
try:
    warningLevel = future.get_result()['format']['schemaDetection']['warningLevel']
    print('Warning level:' + warningLevel)
except KeyError:
    print('No warning')

In the schemaDetection dict you have many informations like the reasons of the warning in "textReasons" and the type of the warning in "type". 

I hope it helps,
Arnaud

0 Kudos
JohnB
Level 3
Author

Hi Arnudde,

Thanks for the response.

Will this work with custom defined SQLServer datasets?

I get this exception on test_and_detect():

DataikuException: java.lang.ClassCastException: Cannot cast com.dataiku.dip.datasets.sql.ManagedSQLTableDatasetTestHandler to com.dataiku.dip.datasets.sql.ExternalSQLDatasetTestHandler

Actually I raised a support ticket on a similar question for which the response was that this was not currently possible with the API.

I would need something that also emulated the "Reload schema from table" button if a warning was found.

0 Kudos
arnaudde
Dataiker
Dataiker

Hi,

Do you get the same "Java.lang.ClassCastException" exception when using the "Check Now" button ?

You can achieve the same as the "Reload schema from table" button by extracting the detected schema from the result of "test_and_detect()" method and set it for the dataset with the DSSDataset.set_schema  method

detected_schema = future.get_result()['format']['schemaDetection'["detectedSchema"]

Hope it helps,

0 Kudos
A banner prompting to get Dataiku DSS