Storing and Retrieving Embeddings in Knowledge Bank via Python

misti

Hello Team,

I hope you are doing well.

I am currently working on a project in Dataiku 13.1.2, where I am generating embeddings using LLM Mesh in Python code. At present, I am storing these embeddings in a PostgreSQL dataset. However, I would like to store them directly into a Knowledge Bank using Python code.

Key Requirements:

• Store embeddings in a Knowledge Bank programmatically using Python (without using the "Embed Documents" recipe or any other recipe).
• Read data from the Knowledge Bank using Python.

Assistance Required:

1. Is it possible to write/update/store embeddings into a Knowledge Bank via Python code?
   • If yes, could you please share the relevant documentation link and a sample code snippet?
2. How can I retrieve/read data from the Knowledge Bank using Python?
   • If there is an API available, kindly provide the details and an example.

Your guidance on this would be highly appreciated. Looking forward to your response.

Regards,
Misti

Alexandru

Hi,
1) Can you write/update/store embeddings into a Knowledge Bank via Python code?

Currently no, you can only retrieve text embedding, but writes must be done with an embedding recipe.

https://developer.dataiku.com/latest/concepts-and-examples/llm-mesh.html#text-embedding
https://developer.dataiku.com/latest/api-reference/python/llm-mesh.html#dataikuapi.dss.llm.DSSLLMEmbeddingsQuery

2) Yes you can use a Knowledge bank from code
https://developer.dataiku.com/latest/concepts-and-examples/llm-mesh.html#using-knowledge-banks-as-langchain-retrievers

misti

Thank you so much Alexandru 😊