How to use LLM Mesh work with LiteLLM

Abrarriadi

Hi,

I'm working on an Agentic Gen AI project using the crewai package, which uses LiteLLM as the engine to connect to various Gen AI models. I would like to use Dataiku LLM Mesh, but it seems that it's not compatible with the LiteLLM. I tried to use the DKULLM and DKUChatLLM, but both of them are not working. I'm on Dataiku v13.2.2

Can you please help?

Operating system used: Ubuntu 22.04

Alexandru

Hi,
Are you seeing a specific error? Could you open a support ticket so we can work with further on this

If LiteLLM proxy is OpenAI API compatible, you could try to use an OpenAI connection with “Custom URL”, the URL that exposes the model typically (typically ending in …/v1). Disable all models that may have been automatically suggested
Add custom models using the ID of the models you want to address.

Thanks