Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
I'm trying to use Snowpark within Python recipe.
from dataiku.snowpark import DkuSnowpark
This works in DSS, but when I tried to run this code in local VS Code, it returns error "ModuleNotFoundError: No module named 'dataiku.snowpark'".
I want to edit and run .py script in local VS Code through Dataiku extension.
P.S. I already installed the API packages.
import dataiku, dataikuapi
Operating system used: Windows 10
You need to install the Dataiku internal API package for that to work. Have a look at this page:
I have already installed the two pakcages.
import dataiku, dataikuapi
I'm able to import dataiku, but not dataiku.snowpark.
This is an interesting one. It looks like the internal client package doesn't include the snowpark wrapper. From what I can see DSS is loading these from the installation directory. For instance in my system the files are in the following location:
/Users/my_user/Library/DataScienceStudio/kits/dataiku-dss-12.5.0-osx/python/dataiku/snowpark
So I fixed this by copying this snowpark directory to the directory where I got the internal client installed:
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/dataiku/
Then I got the Snowflake snowpark package installed (pip3 install snowflake-snowpark-python) and I was able to import dataiku.snowpark in my local Jupyter env:
Thank you, I'll take a try, it looks promissing!
Does it imply the package downloaded from this code below does not include Snowpark component?
pip install --trusted-host 10.xxx.xxx.xxx https://10.xxx.xxx.xxx/public/packages/dataiku-internal-client.tar.gz
Exactly.