Cleaning Share Drive

[sj0071992]

Hi Team,

I am using Share drive to read files in Dataiku and i want the files which are older than 30 days to get removed from share drive.

Can we handle this from Dataiku?

Thanks in Advance

[AlexT]

Hi,

There is no built-in functionality for this in DSS. Usually, storage providers like S3 would have lifecycle rules you could leverage for this. 

If you don't currently have such native functionality you may be able to leverage managed folders API in DSS in a Scenario to delete files older than x days. 

import dataiku
import dataikuapi
from datetime import datetime, timedelta

client = dataiku.api_client()
project_key = "DKU_TUTORIAL_FIRSTPLUGIN"
odb_id = "eZBgqcZ3"

folder = dataikuapi.dss.managedfolder.DSSManagedFolder(client, project_key, odb_id)
contents = folder.list_contents()

thirty_days_ago = int((datetime.now() - timedelta(days=30)).timestamp() * 1000)

print(thirty_days_ago)

for item in contents['items']:
    if item['lastModified'] < thirty_days_ago:
        folder.delete_file(item['path'])