ModuleNotFoundError: No module named 'dataiku.langchain'

Lincoln

Hi, I was recently trying to install the dataiku api locally for some testing purposes but was met with some errors.
Installation seemed to be working fine but the moment I tried to run this code:`from dataiku.langchain.llm import DKUChatModel` , it seemed to crash and be unable to find the dataiku.langchain module.

What seemed weird to me was that this import statement worked fine in the online dataiku DSS notebooks. Since I couldn't find anyone else on the forums with this issue, I decided to take a gamble and post my question here!

Sincerely wondering if there is any way to resolve this issue.

Thanks!

P.S.: The tutorial I was following was this basic one https://developer.dataiku.com/latest/tutorials/devtools/python-client/index.html

Operating system used: Windows

Edit*
Here's the list dataiku libraries installed from pip list:

Package                |  Version

dataiku-api-client     | 13.1.4

dataiku-internal-client|  13.1.4

Turribeach

Can you run “pip list” in your system and post all the lines that have Dataiku in them? Can you also post the output of “Testing your setup” section of the link you posted? Thanks

Lincoln

https://community.dataiku.com/discussion/comment/44582#Comment_44582

Sure! I've updated the original comment.

Turribeach

Your post says you are in Dataiku version 12, why do you have the v13 packages installed?

Lincoln

https://community.dataiku.com/discussion/comment/44584#Comment_44584

When making the post, it didn't allow me to choose version 13, so I could only pick the latest one 😅, sorry about the confusion there

Turribeach

What's the output of this?

import dataiku

dataiku.set_remote_dss("https://dss.example", "YOURAPIKEY")

client = dataiku.api_client()

# Uncomment this if your instance has a self-signed certificate
# client._session.verify = False

info = client.get_auth_info()
print(info)

Turribeach

Actually I just tested and I get the same error as you. It looks like the dataiku.langchain module is not part of the Dataiku internal client and it's only installed as part of the Dataiku installation. This might be because Dataiku expects you to run this code from inside Dataiku. Any reason as to you wanting to run it from your Windows PC?

Lincoln

https://community.dataiku.com/discussion/comment/44587#Comment_44587

Hmm, that's unfortunate if that's the case.

The reason I wanted to run it locally was because I was aiming to see whether it was possible to access the dataiku resources locally (i.e the knowledge bases as well as the llms) to build and host a REST API for private use (meaning that it would not be hosted in the cloud but on local premises).

Regardless, thank you so much for your help in clarifying my doubts!

Have a good day ahead 😊

P.S: if there's anyone out there who knows the solution in the future, do let me know :)

Turribeach

Dataiku can easily be installed on-prem or even on Cloud (your own cloud that is). So you can easily have local access to it. Another option it will work is to write a Python recipe to interface with the LLM mesh and execute that recipe remotely using the Dataiku Python API and access the outputs as a result dataset. I also think you should be able to create a wrapper API over the LLM mesh as a custom Project Dataiku API which you can deploy to the Dataiku API node and again access remotely. But you will always need to go via Dataiku as that’s what the LLM mesh is giving you, an easy way to interface with LLM models.

Also keep in mind that the LLM mesh is simply a wrapper over other LLM models which are typically hosted via third party APIs. So you typically depend on those third party models and APIs, usually hosted on internet facing services.