As a follow up to https://community.dataiku.com/t5/Setup-Configuration/Feature-request-Monitor-memory-usage-per-scenar... it sure seems like this has been implemented via https://doc.dataiku.com/dss/latest/operations/monitoring.html#configure-dss-to-push-metrics, which I've done. I'm now seeing this in job logs:
{
"context": {
"type": "JOB_ACTIVITY",
"authIdentifier": "ADataScientistUser",
"projectKey": "MY_PROJECT",
"jobId": "Build_shap_values_unpivoted_2021-01-14T09-22-01.850",
"activityId": "compute_shap_values_unpivoted_NP",
"activityType": "recipe",
"recipeType": "python",
"recipeName": "compute_shap_values_unpivoted"
},
"type": "LOCAL_PROCESS",
"id": "JwY8E2ga2tS4V42W",
"startTime": 1610616122774,
"localProcess": {
"pid": 67709,
"commandName": "/mnt/secondary/dataiku_data/bin/python",
"cpuUserTimeMS": 654790,
"cpuSystemTimeMS": 64300,
"cpuChildrenUserTimeMS": 0,
"cpuChildrenSystemTimeMS": 0,
"cpuTotalMS": 719090,
"cpuCurrent": 0.9978004399120176,
"vmSizeMB": 82066,
"vmRSSMB": 80657,
"vmHWMMB": 80657,
"vmRSSAnonMB": 80623,
"vmDataMB": 80694,
"vmSizePeakMB": 82066,
"vmRSSPeakMB": 80657,
"vmRSSTotalMBS": 42283649,
"majorFaults": 18,
"childrenMajorFaults": 0
}
}
Question is: where are these details going? I can't seem to find any references to these processes or recipes or activities or whatnot in my Carbon (via dkumonitor) server. If it is being pushed to Carbon/statsd, what's the prefix and/or search query? If it's pushed elsewhere, where might that be?
