How to setup local Python environment for dataiku? (pandas version conflict)

Haoran

【code】

import dataiku
import pandas as pd
print("pandas version: " + pd.__version__)

【result】

 C:\xxx\scripts\test.py 
WARNING:root:Could not import pandas. Pandas support will be disabled. To enable get_dataframe and other methods, please install the pandas package
pandas version: 2.3.3

 

【description】

I'm currently setting up a local Dataiku environment but have run into an issue. Although pandas is already installed—confirmed by successfully printing its version—I’m unable to import the dataiku module. Instead, I receive the following warning:

"WARNING:root: Could not import pandas. Pandas support will be disabled. To enable get_dataframe and other methods, please install the pandas package."

A colleague who is successfully using the Dataiku API locally mentioned they are using pandas version 2.2.2. I downgraded my pandas to match this version, but the problem persists.

Do you have any suggestions on how to resolve this? Thanks for your help.

Operating system used: win11 enterprise

Turribeach

Hi, the pandas version supported will depend on the Dataiku package version and Python version you have installed. Dataiku version you do not say what version you are trying to install but I can see you are trying to use Python 3.12. In order to crete a compatible code environment locally you should use a virtual code environment in the same way Dataiku does it. You should use a requirements.txt. My suggestion is that you first create the code environment in Dataiku with the "Install core packages" selected and the "Install Jupyter support" unselected. Then add whatever additional packages you need. Once you have a working code environment in Dataiku create a requirements.txt for your local setup by merging the Requested packages and the Base Packages used by Dataiku and adding the Dataiku API packages as needed. Then create your virtual environment locally using this requirements.txt.

Please note there are two different API packages and each has different uses. The following documentation page covers this in detail:

https://developer.dataiku.com/latest/tutorials/devtools/python-client/index.html

PS: If you are till having problems please post your requirements.txt so we can try to reproduce your issue.