Running Hugging Face models in Dataiku

Jorge Carlos

Hi, I am trying to run some hugging face models in Dataiku DSS. The task requires to use an on-premise server where I have some useful GPUs to run these hugging face models. The server has an NVIDIA H100 Tensor Core GPU and I want to know:

1. If Dataiku DSS can be installed in that server, if there are no problems with architecture or something that Dataiku has issue with. I already looked for Dataiku requirements and I do not see any problem, but I just want to make sure that everything is clear.
2. According to documentation(Running Hugging Face models — Dataiku DSS 14 documentation), there is a pre-requisite to have a running Elastic AI Kubernetes cluster with NVIDIA GPUs. I am not sure if it is mandatory since I have a server with a GPU that I want to use to run these models, so running an elastic kubernetes cluster in the cloud may waste the potential of installing Dataiku in this server. Also I would like to know if there is no problem with running hugging face models in this type of GPU.

Thank you

Operating system used: Ubuntu

Operating system used: Ubuntu

Turribeach

1. I don't see any issues other than it can be tricky to get all the required GPU software installed and working correctly. There are numerous versions of each of the components around and they are not all compatible with each other. I covered some of the issues on this recent answer I wrote and it's linked post which I strongly suggest you read to get an idea. Getting the stack working properly is not a Dataiku issue so it isn't covered in the Dataiku documentation, you will have to look elsewhere.
2. I am not sure where this requirement to use a Kubernetes cluster for locally hosted Hugging Face models comes from. It may be from the assumption that you will be using public cloud resources therefore you wouldn't want to have the GPU enabled VM running 24x7. Since this is not the case you want to clarify with Dataiku. I can certainly confirm that you can run local models in GPU in Dataiku provided you meet you have a full configured / working GPU. It's possible however that the vLLM Dataiku is using might need Kubernetes, I do not have the answer for that. What I can tell you is that getting a fully configured GPU stack inside Kubernetes is even harder than outside it. This is a really advanced setup which you should ready try to avoid and you should follow the Cautions section in the page you linked.