ModuleNotFoundError: No module named 'dataiku.langchain'
Hi, I was recently trying to install the dataiku api locally for some testing purposes but was met with some errors.
Installation seemed to be working fine but the moment I tried to run this code:`from dataiku.langchain.llm import DKUChatModel` , it seemed to crash and be unable to find the dataiku.langchain module.
What seemed weird to me was that this import statement worked fine in the online dataiku DSS notebooks. Since I couldn't find anyone else on the forums with this issue, I decided to take a gamble and post my question here!
Sincerely wondering if there is any way to resolve this issue.
Thanks!
P.S.: The tutorial I was following was this basic one
Operating system used: Windows
Edit*
Here's the list dataiku libraries installed from pip list:
Package | Version
dataiku-api-client | 13.1.4
dataiku-internal-client| 13.1.4
Answers
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,162 NeuronCan you run “pip list” in your system and post all the lines that have Dataiku in them? Can you also post the output of “Testing your setup” section of the link you posted? Thanks
-
Sure! I've updated the original comment.
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,162 Neuron
Your post says you are in Dataiku version 12, why do you have the v13 packages installed?
-
When making the post, it didn't allow me to choose version 13, so I could only pick the latest one 😅, sorry about the confusion there
-
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,162 Neuron
What's the output of this?
import dataiku dataiku.set_remote_dss("https://dss.example", "YOURAPIKEY") client = dataiku.api_client() # Uncomment this if your instance has a self-signed certificate # client._session.verify = False info = client.get_auth_info() print(info) -
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,162 Neuron
Actually I just tested and I get the same error as you. It looks like the dataiku.langchain module is not part of the Dataiku internal client and it's only installed as part of the Dataiku installation. This might be because Dataiku expects you to run this code from inside Dataiku. Any reason as to you wanting to run it from your Windows PC?
-
Hmm, that's unfortunate if that's the case.
The reason I wanted to run it locally was because I was aiming to see whether it was possible to access the dataiku resources locally (i.e the knowledge bases as well as the llms) to build and host a REST API for private use (meaning that it would not be hosted in the cloud but on local premises).
Regardless, thank you so much for your help in clarifying my doubts!
Have a good day ahead 😊
P.S: if there's anyone out there who knows the solution in the future, do let me know :) -
Turribeach Dataiku DSS Core Designer, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 2,162 Neuron
Dataiku can easily be installed on-prem or even on Cloud (your own cloud that is). So you can easily have local access to it. Another option it will work is to write a Python recipe to interface with the LLM mesh and execute that recipe remotely using the Dataiku Python API and access the outputs as a result dataset. I also think you should be able to create a wrapper API over the LLM mesh as a custom Project Dataiku API which you can deploy to the Dataiku API node and again access remotely. But you will always need to go via Dataiku as that’s what the LLM mesh is giving you, an easy way to interface with LLM models.
Also keep in mind that the LLM mesh is simply a wrapper over other LLM models which are typically hosted via third party APIs. So you typically depend on those third party models and APIs, usually hosted on internet facing services.