about ML
Hi,
I'm querious about performance using EC2 type.
I created DSS Instance using 'm5.4xlarge' machine type.
I understood this type is related running etl& training model performance and
normally when i choose stronger machine type, performance will get better.
Q1. if i want to use small machine type to run ETL and
use powerful machine type to train model, how could i seperate machine type for each task?
Q2. I couldn't find GPU instance list in machine type when i create DSS Instance.
I think this question is related to Q1.
If i want to use GPU instance to train AI/ML model, how?
Regards,
Daniel Lee
Answers
-
Alexandru Dataiker, Dataiku DSS Core Designer, Dataiku DSS ML Practitioner, Dataiku DSS Adv Designer, Registered Posts: 1,209 DataikerOptionsHi @seungsulee
,We would recommend leveraging Elastic AI in other words containers orchestrated by Kubernetes instead of the DSS instances to perform model training and recipes, Spark, etc.
You can use GPUs on your K8s clusters instead of the DSS Server itself.
https://doc.dataiku.com/dss/latest/containers/eks/unmanaged.html#using-gpus
In this case, you don't need a higher spec DSS machine but use ideally a managed Kubernetes cluster/s which you can scale according to your needs.