LLM Service Plugin?

importthepandas

This might end up as a product idea if it is not already, but is there currently a way e.g. via plugins to incorporate an internal foundation model with its associated API endpoints? Essentially, we are wanting to create a new LLM connection.

---

Operating system used: Ubuntu 20

marcochang1028

Hi

Do you finally find a way to implement this method?

I only see an explanation from their knowledge base, but no any idea to realize it.

Users who have developed their own LLMs with API endpoints can integrate these custom services into Dataiku via the Custom LLM connection. Setting up a Custom LLM connection requires developing a specific plugin using the Java LLM component to map the endpoint’s behavior.

Here is the original web page.

https://knowledge.dataiku.com/latest/ml-analytics/gen-ai/reference-llm-connections.html#custom-llm-connections