Can I run AutoML on multiple instances?

s4gjoker

I am using Dataiku 9, recently my server is overloaded when I use AutoML, because my data is quite big, I see DSS has a feature called cluster where I can add Hadoop clusters or K8S, can I install multiple DSS on K8S and train together, in general, I am looking for something like Spark, thanks

Alexandru

Hi,

There are several-push down compute options available see: https://doc.dataiku.com/dss/latest/containers/index.html.

This allows you to push down training of models and other jobs to at Kubernetes cluster.

Also starting with DSS 9 we also introduced distributed hyperparameter computations "

https://knowledge.dataiku.com/latest/kb/analytics-ml/hyperparameter-distributed.html

GPUs can be leveraged in containerized executions as well:

https://doc.dataiku.com/dss/latest/machine-learning/deep-learning/runtime-gpu.html

Let me know if you have any additional questions.