Submit your innovative use case or inspiring success story to the 2023 Dataiku Frontrunner Awards! LET'S GO

Run python script

Level 1
Run python script


I want to run other open source code in Python with Dataiku platform.
Most open-source code can be run through Python scripts like below link.
python --resume --lr=0.01

I want to know how to use python script in dataiku.

0 Kudos
1 Reply

One approach is to
1) import the remote git repo as a project library from git.
2) Then use a Notebook or Python recipe in DSS to run the "" with the arguments.
3) Note the code env used for the Python recipe/Notebook will need to have the required e.g torch,  torchvision 

Note the code may not be designed to be run as code directly e.g if interact with specific components, you may need to refactor or modify the code.
In your case, I found an issue with a line which I commend and set a fixed size as suggested here
Once resolving this, I was able to run the script but your sample command fails because there was no "checkpoint" directory which seems expected. 

import subprocess
import sys
import os.path as osp

def get_path_to_library_path(path_in_lib):
    for path_chunk in sys.path:
        searched_path = osp.join(path_chunk, path_in_lib)
        if osp.exists(searched_path):
            return searched_path
    raise Exception("Path %s not found in imported libraries" % path_in_lib)

path = get_path_to_library_path("pytorch-cifar/")
print("Path:", path)

current_python_path = sys.executable
print("Current Python Path:", current_python_path)

# Specify the command to execute
command = [current_python_path, path, '--resume', '--lr=0.01']

# Execute the command
    result =, check=True, text=True)
except subprocess.CalledProcessError as e:
    print(f"Subprocess execution failed with exit code {e.returncode}.")

# Properly exit the script

Hope that helps!