Sign up to take part
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
Registered users can ask their own questions, contribute to discussions, and be part of the Community!
As a follow up to https://community.dataiku.com/t5/Setup-Configuration/Feature-request-Monitor-memory-usage-per-scenar... it sure seems like this has been implemented via https://doc.dataiku.com/dss/latest/operations/monitoring.html#configure-dss-to-push-metrics, which I've done. I'm now seeing this in job logs:
{
"context": {
"type": "JOB_ACTIVITY",
"authIdentifier": "ADataScientistUser",
"projectKey": "MY_PROJECT",
"jobId": "Build_shap_values_unpivoted_2021-01-14T09-22-01.850",
"activityId": "compute_shap_values_unpivoted_NP",
"activityType": "recipe",
"recipeType": "python",
"recipeName": "compute_shap_values_unpivoted"
},
"type": "LOCAL_PROCESS",
"id": "JwY8E2ga2tS4V42W",
"startTime": 1610616122774,
"localProcess": {
"pid": 67709,
"commandName": "/mnt/secondary/dataiku_data/bin/python",
"cpuUserTimeMS": 654790,
"cpuSystemTimeMS": 64300,
"cpuChildrenUserTimeMS": 0,
"cpuChildrenSystemTimeMS": 0,
"cpuTotalMS": 719090,
"cpuCurrent": 0.9978004399120176,
"vmSizeMB": 82066,
"vmRSSMB": 80657,
"vmHWMMB": 80657,
"vmRSSAnonMB": 80623,
"vmDataMB": 80694,
"vmSizePeakMB": 82066,
"vmRSSPeakMB": 80657,
"vmRSSTotalMBS": 42283649,
"majorFaults": 18,
"childrenMajorFaults": 0
}
}
Question is: where are these details going? I can't seem to find any references to these processes or recipes or activities or whatnot in my Carbon (via dkumonitor) server. If it is being pushed to Carbon/statsd, what's the prefix and/or search query? If it's pushed elsewhere, where might that be?
Hi @mklaber
Those are two separate things:
compute-resource-usage
topic. For more details looks at this documentation page