Error in "from dataiku.snowpark import DkuSnowpark"

little_sun
little_sun Registered Posts: 5 ✭
edited July 2024 in Plugins & Extending Dataiku

I'm trying to use Snowpark within Python recipe. 

from dataiku.snowpark import DkuSnowpark

This works in DSS, but when I tried to run this code in local VS Code, it returns error "ModuleNotFoundError: No module named 'dataiku.snowpark'".

I want to edit and run .py script in local VS Code through Dataiku extension.

P.S. I already installed the API packages. 

import dataiku, dataikuapi

Operating system used: Windows 10

· Share on Facebook

Answers

  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023, Circle Member Posts: 2,722 Neuron

    You need to install the Dataiku internal API package for that to work. Have a look at this page:

    https://developer.dataiku.com/latest/tutorials/devtools/python-client/index.html#building-your-local-virtual-environment

    · Share on Facebook
  • little_sun
    little_sun Registered Posts: 5 ✭
    edited July 2024

    I have already installed the two pakcages. 

    import dataiku, dataikuapi

    I'm able to import dataiku, but not dataiku.snowpark.

    · Share on Facebook
  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023, Circle Member Posts: 2,722 Neuron

    This is an interesting one. It looks like the internal client package doesn't include the snowpark wrapper. From what I can see DSS is loading these from the installation directory. For instance in my system the files are in the following location:

    /Users/my_user/Library/DataScienceStudio/kits/dataiku-dss-12.5.0-osx/python/dataiku/snowpark

    So I fixed this by copying this snowpark directory to the directory where I got the internal client installed:

    /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/dataiku/

    Then I got the Snowflake snowpark package installed (pip3 install snowflake-snowpark-python) and I was able to import dataiku.snowpark in my local Jupyter env:

    Screenshot 2024-01-23 at 22.29.45.png

    · Share on Facebook
  • little_sun
    little_sun Registered Posts: 5 ✭
    edited July 2024

    Thank you, I'll take a try, it looks promissing!

    Does it imply the package downloaded from this code below does not include Snowpark component? 

    pip install --trusted-host 10.xxx.xxx.xxx https://10.xxx.xxx.xxx/public/packages/dataiku-internal-client.tar.gz

    · Share on Facebook
  • Turribeach
    Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023, Circle Member Posts: 2,722 Neuron

    Exactly.

    · Share on Facebook
Bookmark this topic

Categories

Setup Info
    Tags
      Help me…