Get job memory details [Python API]

RajeshRana84
Level 2
Get job memory details [Python API]

Good Day All,

I am looking to get the following things for any job :

1. Total EKS Memory (ex. 20 GB)

2. Total EKS CPU  (ex. 10 CPU)

3. Spark Driver RAM (ex. 4 GB)

4. Spark Driver CPU (ex. 2 CPU)

5. Is it a Local job or not ?

6. Is it using SPARK or not ?

 

I know how to get Job_logs for any job but will I find all the above details in the logs or Is there any other way to get all the above details for a particular job ?

 

Thanks,
Rajesh

0 Kudos
1 Reply
Clรฉment_Stenac

Hi,

Dataiku has the ability to retrieve all of that information, but that requires some setup. Could you please get in touch with your Dataiku Customer Success Manager, who can help set up a session to go over this?

0 Kudos