How to install customized wheel files in Dataiku (local & container)

iankush

Hi All,

I am looking for some help with my problem where I am trying to install customized wheel file on local & container image using resource initialization script. I am trying two options to achieve this task but it is running into issues. I am using Datetime package wheel file as a sample wheel file for this task.

1) Using Managed Folder hosted on S3 -- I am running below piece of code on jupyter notebook and it is returning exit code 1.

import dataiku
import sys
import subprocess

folder = dataiku.Folder("folder_id")
folder_info = folder.get_info()
access_info = folder_info["accessInfo"]
folder_base_url = 's3://%s%s' % (access_info['bucket'], access_info['root'])
print(folder_base_url)
call_com = [sys.executable,"-m","pip","install","DateTime-5.2-py3-none-any.whl","--extra-index-url=f{folder_base_url}"]
subprocess.run(call_com)

2. Using Resource initialization script -- In the resources section under code environment, I've added this script. In this option, I have already uploaded the datetime wheel file to S3 location and trying to refer it in resource initialization script but it is also running into issues.

import dataikuapi
import subprocess
import sys

call_components = [sys.executable, "-m", "pip", "install", "DateTime-5.2-py3-none-any.whl","--extra-index-url=https://<s3 path where the wheel file is present>"]
subprocess.run(call_components)

Could anyone help me with the right approach to download and install wheel files in dataiku. My requirement is to download and install my customized wheel file in both local and containers. Thanks in advance for the help

---

Operating system used: Linux

Turribeach

The traditional approach for this issue is to host the package in a local Python repository like Artifactory. If you can host it then copy the file to local storage in your container first the run pip with the local file:

https://stackoverflow.com/questions/15031694/installing-python-packages-from-local-file-system-folder-to-virtualenv-with-pip