Can I run AutoML on multiple instances?

s4gjoker
Level 1
Can I run AutoML on multiple instances?

I am using Dataiku 9, recently my server is overloaded when I use AutoML, because my data is quite big, I see DSS has a feature called cluster where I can add Hadoop clusters or K8S, can I install multiple DSS on K8S and train together, in general, I am looking for something like Spark, thanks

0 Kudos
1 Reply
AlexT
Dataiker

Hi,

There are several-push down compute options available see:  https://doc.dataiku.com/dss/latest/containers/index.html.

 This allows you to push down training of models and other jobs to at Kubernetes cluster. 

Also starting with DSS 9 we also introduced distributed hyperparameter computations "

https://knowledge.dataiku.com/latest/kb/analytics-ml/hyperparameter-distributed.html 

GPUs can be leveraged in containerized executions as well:

https://doc.dataiku.com/dss/latest/machine-learning/deep-learning/runtime-gpu.html 

Let me know if you have any additional questions.