Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Hi ,
Is there a way to use LLM connections and models as a API node?
Operating system used: Windows
Hi,
Currently, you can use the LLM APIs directly in a custom python API endpoint.
This may evolve in the future.
You would need to leverage set_remote_dss https://developer.dataiku.com/latest/getting-started/outside-usage.html as API nodes are outside of DSS
For the host/API key note you can store these env variables to avoid storing in code
A) on static API node you can set env variables
B) Using Resource init script for the code env for API endpoints deployed on K8s.
from dataiku.code_env_resources import set_env_var
set_env_var(โDKU_API_KEYโ, โxxxxโ)
set_env_var(โDKU_DSS_URLโ, โhttps://xxxxxxxโ)
Then use read the env variables an use LLM APIs: https://developer.dataiku.com/latest/concepts-and-examples/llm-mesh.html
Hi Alex , I'm getting this error in the api designer :
No connection adapters were found for '127.0.0.1:51442/dip/publicapi/projects/TUT_REAL_TIME/llms/completions'
Hi @mebinjoy ,
Were you able to solve it? I have the exact same error.
Hi @mebinjoy,
Can you open a support ticket and share the Instance diagnostics and a zip of the api package from the version tab of API Designer?
Thanks