about ML

[seungsulee]

Hi,

I'm querious about performance using EC2 type.

I created DSS Instance using 'm5.4xlarge' machine type.

I understood this type is related running etl& training model performance and

normally when i choose stronger machine type, performance will get better.

Q1. if i want to use small machine type to run ETL and 

use powerful machine type to train model, how could i seperate machine type for each task?

Q2. I couldn't find GPU instance list in machine type when i create DSS Instance.

I think this question is related to Q1.

If i want to use GPU instance to train AI/ML model, how?

Regards,

Daniel Lee

[AlexT]

Hi @seungsulee,

We would recommend leveraging Elastic AI in other words containers orchestrated by Kubernetes instead of the DSS instances to perform model training and recipes, Spark, etc. 

You can use GPUs on your K8s clusters instead of the DSS Server itself. 

https://doc.dataiku.com/dss/latest/containers/eks/unmanaged.html#using-gpus

In this case, you don't need a higher spec DSS machine but use ideally a managed Kubernetes cluster/s which you can scale according to your needs.