Python API and custom script scenario logs

JohnB
JohnB Registered Posts: 32 ✭✭✭✭✭

Is there a way to get the log contents from the run of a scenario that is a custom script?

These do not appear in job logs so I can't do job.get_log()

Best Answer

  • EmmaH
    EmmaH Dataiker Posts: 4 Dataiker
    Answer ✓

    Hi John,

    What steps are you running in your custom script? The sample code you provided should work for a custom python scenario as well, provided that you're creating jobs through the scenario. Unfortunately, we don't provide scenario logs, so you won't be able to receive logs on operations that don't trigger jobs.

Answers

  • CoreyS
    CoreyS Dataiker Alumni, Dataiku DSS Core Designer, Dataiku DSS Core Concepts, Registered Posts: 1,150 ✭✭✭✭✭✭✭✭✭

    Hi, @JohnB
    ! Can you provide any further details on the thread to assist users in helping you find a solution (insert examples like DSS version etc.) Also, can you let us know if you’ve tried any fixes already?This should lead to a quicker response from the community.

  • JohnB
    JohnB Registered Posts: 32 ✭✭✭✭✭

    Hi Corey,

    We are on version 8.0.3.

    If I'm running a scenario that has steps, I can do something like this:

    scenario = project.get_scenario("myscenario")
    run = scenario.run_and_wait()
    for step in run.get_details().steps:
    for job_id in step.job_ids:
    job = project.get_job(job_id)
    print(job.get_log())

    When the scenario is a custom script I don't see any jobs that I can get logs from.

  • JohnB
    JohnB Registered Posts: 32 ✭✭✭✭✭

    Hi Emma,

    The scenario is a custom python script.

    It's one of a set of tests we perform on new code envs that have libraries that interact with bespoke Python interfaces. These would be outputting messages using the Python logger at the very least - even simple print statements e.g.

    print("test")

    ..would not create a job so there is no log we can extract via the API.

    Regards,

    John

Setup Info
    Tags
      Help me…