Discover this year's submissions to the Dataiku Frontrunner Awards and give kudos to your favorite use cases and success stories!READ MORE

Using Python API to retrieve last_build_time

Viet
Level 2
Using Python API to retrieve last_build_time

I am trying to retrieve info about last_build_time_start (end) of a dataset using APIs. On the Docs, in class dataikuapi.dss.dataset.DSSDataset, there is a function named get_info to get object DSSDatasetInfo , but I could not call it. 

1.PNG

Is it a bug or this class has been removed? If then, there are any other way to get these kind of information? There is a topic with a solution mentioning 'Internal Stats Dataset' but I don't know how to use that dataset.

Here is the information of the DSS I am using on my Chrome

2.PNG


Operating system used: debian (9)

5 Replies
CatalinaS
Dataiker
Dataiker

Hi @Viet,

I was able to retrieve last_build_time_start using below code:

 

import dataikuapi
host="http://localhost:11200"
apiKey = "*************"
project_key="*********"

def test_get_info(project_key):
    client = dataikuapi.DSSClient(host,apiKey)
    dataset = client.get_project(project_key).get_dataset("test")
    settings = dataset.get_settings()
    print(dataset)     
    print(dataset.get_info())
    print(dataset.get_info().get_raw())


test_get_info(project_key) 

 

The output of this code is:

 % python3.9 testdatasetDSS.py
<dataikuapi.dss.dataset.DSSDataset object at 0x10d022af0>
<dataikuapi.dss.dataset.DSSDatasetInfo object at 0x10c1e85e0>
{'type': 'UploadedFiles', 'name': 'test', 'analyses': [], 'charts': [], 'notebooks': [], 'worksheets': [], 'partitioned': False, 'dataset': {'type': 'UploadedFiles', 'managed': False, 'featureGroup': False, 'name': 'test', 'projectKey': 'MANAGEDFOLDERS', 'formatType': 'csv', 'checklists': {'checklists': []}, 'checks': [], 'customMeta': {'kv': {}}, 'flowOptions': {'virtualizable': False, 'rebuildBehavior': 'NORMAL', 'crossProjectBuildBehavior': 'DEFAULT'}, 'readWriteOptions': {'preserveOrder': False, 'writeBuckets': 1, 'forceSingleOutputFile': False, 'defaultReadOrdering': {'enabled': False, 'rules': []}}, 'formatParams': {'style': 'excel', 'charset': 'utf-8', 'separator': ',', 'quoteChar': '"', 'escapeChar': '\\', 'dateSerializationFormat': 'ISO', 'arrayMapFormat': 'json', 'hiveSeparators': ['\x02', '\x03', '\x04', '\x05', '\x06', '\x07', '\x08'], 'skipRowsBeforeHeader': 0, 'parseHeaderRow': True, 'skipRowsAfterHeader': 0, 'probableNumberOfRecords': 5, 'normalizeBooleans': False, 'normalizeDoubles': True, 'readAdditionalColumnsBehavior': 'INSERT_IN_DATA_WARNING', 'readMissingColumnsBehavior': 'DISCARD_SILENT', 'readDataTypeMismatchBehavior': 'DISCARD_WARNING', 'writeDataTypeMismatchBehavior': 'DISCARD_WARNING', 'fileReadFailureBehavior': 'FAIL', 'compress': ''}, 'partitioning': {'ignoreNonMatchingFile': False, 'considerMissingRequestedPartitionsAsEmpty': False, 'dimensions': []}, 'versionTag': {'versionNumber': 1, 'lastModifiedBy': {'login': 'admin'}, 'lastModifiedOn': 1659620057647}, 'creationTag': {'versionNumber': 0, 'lastModifiedBy': {'login': 'admin'}, 'lastModifiedOn': 1659620057476}, 'tags': [], 'params': {'uploadConnection': 'Default (in DSS data dir.)', 'notReadyIfEmpty': False, 'filesSelectionRules': {'mode':

 

I checked Dataiku API release 10 code and this contains get_info() method.

Can you please share more code that shows how you obtained the DSSDataset object?

What version of Dataiku API client do you use? Please confirm the version by running the following command using your version of Python:

python3.9 -m pip list |grep dataiku-api-client

 

0 Kudos
Viet
Level 2
Author

Hi Catalina,

Here is the codes. 

1.PNG

I was unable to run the code

2.PNG

But I have tried and succeeded with get_last_metric_value(). 

0 Kudos
CatalinaS
Dataiker
Dataiker
0 Kudos
Emma
Dataiker
Dataiker

Hey @Viet

In addition to the solution provided by Catalina, you asked about the Internal Stats dataset. 

This dataset is accessed through the Flow > + Dataset > Internal > Internal Stats. The info you're interested in is found in the "Objects state" view. See the screenshot below: 

Screen Shot 2022-08-04 at 9.13.47 AM.png

More information in the docs: https://doc.dataiku.com/dss/latest/connecting/internal-stats.html 

Hope that helps, 

Emma 

Viet
Level 2
Author

Great! Thanks

0 Kudos