LLM Service Plugin?
Options
importthepandas
Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS Core Concepts, Neuron, Dataiku DSS Adv Designer, Registered, Neuron 2023 Posts: 115 Neuron
This might end up as a product idea if it is not already, but is there currently a way e.g. via plugins to incorporate an internal foundation model with its associated API endpoints? Essentially, we are wanting to create a new LLM connection.
Operating system used: Ubuntu 20
Answers
-
marcochang1028 Partner, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 13 PartnerOptionsHi
Do you finally find a way to implement this method?
I only see an explanation from their knowledge base, but no any idea to realize it.
Users who have developed their own LLMs with API endpoints can integrate these custom services into Dataiku via the Custom LLM connection. Setting up a Custom LLM connection requires developing a specific plugin using the Java LLM component to map the endpoint’s behavior.
Here is the original web page.