DSS Memory Management Webapp consumption buildup

Ddur

Hi, so we have quite a number of webapp backends runing on production server and this leads to out of memory situations. We would like to tend to this issue by implementing a backend cleanup routine for long running webapps. Is there a way to get a timestamp for webapp start ? Or is there a better way to go around for this cleanup routine ?

Also we are using dataiku version 13 but this is not available in dropdown menu. Thanks.

Thanks!

Turribeach

This will get you what you want and more. For running apps you can get the PID as well so if you combine this output with a Shell recipe you could even get how much memory each Webapp is using. Ultimately though you should really look to move these Webapps to Kubernetes to isolate them and protect DSS from memory issues.

import dataiku

import datetime


client = dataiku.api_client()

project_key = dataiku.default_project_key()

project = client.get_project(project_key)

project_webapps = project.list_webapps()

for webapp in project_webapps:

    webapp_id = webapp['id']

    webapp_name = webapp['name']

    webapp_running = webapp['backendRunning']

    if webapp_running:

        webapp = project.get_webapp(webapp_id)

        webapp_start_time = datetime.datetime.utcfromtimestamp(int(webapp.get_state().state['futureInfo']['startTime']) / 1000).strftime("%d-%b-%Y %H:%M:%S")

        webapp_running_time = str(datetime.timedelta(seconds=(webapp.get_state().state['futureInfo']['runningTime'])))

        webapp_pid = webapp.get_state().state['futureInfo']['payload']['extras']['pid']

    else:

        webapp_start_time = ''

        webapp_running_time = ''

        webapp_pid = ''

    print(f"Webapp name: {webapp_name} - Webapp ID: {webapp_id} - Running: {webapp_running} - Start Time: {webapp_start_time} - Running Time: {webapp_running_time} - PID: {webapp_pid}")