LLM Service Plugin?

importthepandas
LLM Service Plugin?

This might end up as a product idea if it is not already, but is there currently a way e.g. via plugins to incorporate an internal foundation model with its associated API endpoints? Essentially, we are wanting to create a new LLM connection.


Operating system used: Ubuntu 20

1 Reply
marcochang1028
Level 3

Hi 

Do you finally find a way to implement this method?

I only see an explanation from their knowledge base, but no any idea to realize it.

Users who have developed their own LLMs with API endpoints can integrate these custom services into Dataiku via the Custom LLM connection. Setting up a Custom LLM connection requires developing a specific plugin using the Java LLM component to map the endpoint’s behavior. 

Here is the original web page.

https://knowledge.dataiku.com/latest/ml-analytics/gen-ai/reference-llm-connections.html#custom-llm-c...

 

0 Kudos