How to monitor jobs consumption resources

azamora
Level 2
How to monitor jobs consumption resources

Hi everyone,

Is there any way in Dataiku DSS that a user can see how many resources (memory, storage and CPU) are taking his active jobs ?

I have DSS installed in a Hortonworks cluster and as I am a Hortonworks admin I can see jobs resource consumption in Yarn, but I don't want to open Yarn for all users.

Therefore I need a way for users to be able to see how are their jobs consuming cluster resources.

Thanks,


Operating system used: Linux CentOS

0 Kudos
1 Reply
sergeyd
Dataiker

Hi @azamora 

There is no UI metrics/monitoring for this but you may want to have a look at CRU (Compute Resource Usage) that we added in DSS8: 

https://doc.dataiku.com/dss/latest/operations/compute-resource-usage-reporting.html

 

0 Kudos