New to Dataiku DSS? Try out our NEW Quick Start Programs today and get onboarded on the product in just one hour! Let's go

Python API and custom script scenario logs

Solved!
JohnB
Level 3
Python API and custom script scenario logs

Is there a way to get the log contents from the run of a scenario that is a custom script?

These do not appear in job logs so I can't do job.get_log()

0 Kudos
1 Solution
EmmaH
Dataiker
Dataiker

Hi John,

What steps are you running in your custom script? The sample code you provided should work for a custom python scenario as well, provided that you're creating jobs through the scenario. Unfortunately, we don't provide scenario logs, so you won't be able to receive logs on operations that don't trigger jobs.

View solution in original post

0 Kudos
4 Replies
CoreyS
Community Manager
Community Manager

Hi, @JohnB ! Can you provide any further details on the thread to assist users in helping you find a solution (insert examples like DSS version etc.) Also, can you let us know if you’ve tried any fixes already?This should lead to a quicker response from the community.

Looking for more resources to help you use DSS effectively and upskill your knowledge? Check out these great resources: Dataiku Academy | Documentation | Knowledge Base

A reply answered your question? Mark as ‘Accepted Solution’ to help others like you!
0 Kudos
JohnB
Level 3
Author

Hi Corey,

We are on version 8.0.3.

If I'm running a scenario that has steps, I can do something like this:

scenario = project.get_scenario("myscenario")
run = scenario.run_and_wait()
for step in run.get_details().steps:
    for job_id in step.job_ids:
        job = project.get_job(job_id)
        print(job.get_log())

When the scenario is a custom script I don't see any jobs that I can get logs from.

0 Kudos
EmmaH
Dataiker
Dataiker

Hi John,

What steps are you running in your custom script? The sample code you provided should work for a custom python scenario as well, provided that you're creating jobs through the scenario. Unfortunately, we don't provide scenario logs, so you won't be able to receive logs on operations that don't trigger jobs.

View solution in original post

0 Kudos
JohnB
Level 3
Author

Hi Emma,

The scenario is a custom python script.

It's one of a set of tests we perform on new code envs that have libraries that interact with bespoke Python interfaces. These would be outputting messages using the Python logger at the very least - even simple print statements e.g.

print("test")

..would not create a job so there is no log we can extract via the API.

Regards,

John

0 Kudos
A banner prompting to get Dataiku DSS