Python API and custom script scenario logs

Solved!
JohnB
Level 3
Python API and custom script scenario logs

Is there a way to get the log contents from the run of a scenario that is a custom script?

These do not appear in job logs so I can't do job.get_log()

0 Kudos
1 Solution
EmmaH
Dataiker

Hi John,

What steps are you running in your custom script? The sample code you provided should work for a custom python scenario as well, provided that you're creating jobs through the scenario. Unfortunately, we don't provide scenario logs, so you won't be able to receive logs on operations that don't trigger jobs.

View solution in original post

0 Kudos
4 Replies
CoreyS
Dataiker Alumni

Hi, @JohnB ! Can you provide any further details on the thread to assist users in helping you find a solution (insert examples like DSS version etc.) Also, can you let us know if youโ€™ve tried any fixes already?This should lead to a quicker response from the community.

Looking for more resources to help you use Dataiku effectively and upskill your knowledge? Check out these great resources: Dataiku Academy | Documentation | Knowledge Base

A reply answered your question? Mark as โ€˜Accepted Solutionโ€™ to help others like you!
0 Kudos
JohnB
Level 3
Author

Hi Corey,

We are on version 8.0.3.

If I'm running a scenario that has steps, I can do something like this:

scenario = project.get_scenario("myscenario")
run = scenario.run_and_wait()
for step in run.get_details().steps:
    for job_id in step.job_ids:
        job = project.get_job(job_id)
        print(job.get_log())

When the scenario is a custom script I don't see any jobs that I can get logs from.

0 Kudos
EmmaH
Dataiker

Hi John,

What steps are you running in your custom script? The sample code you provided should work for a custom python scenario as well, provided that you're creating jobs through the scenario. Unfortunately, we don't provide scenario logs, so you won't be able to receive logs on operations that don't trigger jobs.

0 Kudos
JohnB
Level 3
Author

Hi Emma,

The scenario is a custom python script.

It's one of a set of tests we perform on new code envs that have libraries that interact with bespoke Python interfaces. These would be outputting messages using the Python logger at the very least - even simple print statements e.g.

print("test")

..would not create a job so there is no log we can extract via the API.

Regards,

John

0 Kudos